AI the Docs 2025 conference

Event overview

I recently attended the AI the Docs event which focused on the intersection of artificial intelligence and technical documentation, exploring how AI is transforming documentation creation, consumption, and management. The event took place over two days with multiple speakers and presentations.

This blog post summarizes the key themes, notable presentations, and practical recommendations from the event. I’ve used the LLM Claude Sonnet 3.7 to parse my raw notes and help me write this summary.

Key themes

1. AI as documentation consumer

• LLMs as runtime interpreters: Documentation is no longer just for human readers but is being parsed, vectorized, and interpreted by AI systems
• Machine-friendly content: Technical content needs to be structured in ways that machines can effectively extract and process information
• Content-first approach: Plaintext rules; fancy formatting and layouts are lost in translation to AI systems

2. Documentation architecture for AI

• Semantic structure: Using Markdown, clean HTML, and semantic markup to make content AI-accessible
• Chunking content: Proper organization with headings, lists, and examples improves AI comprehension
• Metadata management: Incorporating dates and other metadata helps AI provide more accurate responses
• Pronoun management: Replace pronouns at the beginning of chunks with explicit subjects for clarity

3. RAG (Retrieval Augmented Generation)

• Reducing hallucinations: RAG was highlighted as the best current approach to minimize AI hallucinations
• Vector databases: Embedding documentation in vector databases for improved retrieval
• Contextual understanding: Knowledge graphs give content structure, making it “intelligent content”
• Architecture components:
  • Pre-processing content from knowledge bases
  • Splitting articles into smaller chunks (approximately 500 tokens)
  • Creating embeddings to convert text into vector representations
  • Calculating distances between question embeddings and stored embeddings
  • Using text completion APIs to generate responses based on retrieved context

4. Evolving role of technical writers

With AI consuming and acting on documentation, the role of the writer becomes more strategic. It’s not just about about using clear phrasing or formatting. The role becomes more focused on information architecture, intentionality, and responsibility.

• AI workflow managers: Technical writers are becoming architects of AI interactions
• Truth keepers: Writers serve as guardians of truth in AI interactions
• Content self-healing: Documentation as tests, automation to ensure content remains accurate and up-to-date

Technical writers are key contributors to the design of AI-powered user experiences.

Presentations I enjoyed

Roy Derks (IBM): Effectively use AI Agents to Maintain Your Docs

• Distinguished between business and developer audiences with different documentation needs

• Showed how they created AI-generated narrative videos to explain APIs and software

• Used documentation as the single source of truth (SSOT) for AI avatars .

• Building MCP servers for improved LLM interactions

• Microsoft Learn’s implementation of MCP: https://github.com/MicrosoftDocs/mcp

• Tools like MCP Inspector (“Like Postman for LLMs”): https://modelcontextprotocol.io/docs/tools/inspector

Elmer Thomas, Maria Bermudez (Twilio): The Robots are coming for your job, and that’s okay

The talk introduced “Docs Buddy” which described how to use agents for specific tasks. Instead of one mega-bot, they built six single-purpose agents:

• Automated Editor - Grammar, formatting, accuracy
• Image Alt-Text - Instant accessibility wins
• Jargon Simplifier - Plain English from dev-speak
• SEO Metadata - Title & description, safe chars
• Docs Outline Builder - Nav & structure (soon)
• Slack BatBot - Triage help-channel requests

Selvaraaju Murugesan (kovai.co): Producing AI-friendly content: Emerging best practices

Head of Data Science at Kovai.co presented emerging best practices for AI-friendly content:

• Detailed the changing trends in documentation audience: from humans to AI systems
• Explained RAG architecture and implementation concepts
• Shared practical learnings from implementing AI documentation systems:
  • Shorter how-to guides (under 12 steps) work better than longer ones due to token limits
  • Business glossary terms should be used consistently and in context, not isolated
  • FAQs should be integrated within articles rather than collected separately
  • Acronyms should be fully described and defined in business glossaries
  • Time-related terms like “new” and “latest” should be avoided as RAG lacks time awareness
  • Duplicate or generic article titles should be avoided
• Presented evaluation methods for measuring AI response quality, including accuracy metrics

Practical recommendations from Selvaraaju Murugesan’s talk

1. Audit docs for LLM readiness: Clean markup, proper chunking, semantic structure
2. Build prompt libraries: Capture effective prompts for internal tools and doc generation
3. Experiment with embedded search: Combine technologies like LangChain with API docs
4. Start small with chatbots: Implement RAG for a single doc set and refine
5. Measure what matters: Track successful queries, not just page views
6. Structure content for AI consumption:
   • Use detailed content with a few well-placed FAQs (as “cheat sheets” for AI)
   • Implement proper information architecture with clear, specific headings
   • Avoid generic titles like “Fee” or “Pricing”
   • Use semantic HTML tags like <code> for code snippets and <abbr> for acronyms
   • Include descriptive metadata (creation date, last updated date, article title)
   • Format tables properly with no null values and use binary formats when applicable
   • Ensure articles remain within reasonable token limits (ideally one chunk per concept)

Some tools and technologies mentioned

• LangGraph & LangChain: For building AI workflows
• MCP Inspector: Testing tool for Model Context Protocol
• Langflow: Visual programming for LLMs
• Lorelai: AI platform for documentation
• Astro Docs: Documentation platform
• n8n: Workflow automation tool
• OpenAI Evals: Framework for evaluating AI response quality
• RAGAS: Evaluation metrics for RAG pipelines
• Superface: API documentation and agent framework
• Doc Detective: Tool for detecting documentation issues
• Docs Buddy: AI agents for documentation tasks
• Kopyst: AI-powered documentation platform
• Apigee: Platform for developing, managing, and securing APIs
• The OpenAPI Imperative: OpenAPI specification for AI documentation

Resources shared

Multiple resources were shared during the event, including:

• GitHub repositories for MCP and AI tools
• Research papers on AI reasoning models, e.g. The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
• Documentation best practices for AI consumption
• Specialized tools for API documentation and specification checking
• Evaluation frameworks: https://github.com/openai/evals and https://blog.langchain.dev/evaluating-rag-pipelines-with-ragas-langsmith/

Conclusions

The event highlighted the transformative impact of AI on documentation practices, emphasizing that documentation is increasingly being consumed by machines rather than humans directly, requiring new approaches to content creation and management. The focus is shifting towards making documentation machine-friendly, with an emphasis on structured, semantic content that can be effectively processed by AI systems.