Discovering Athena: Letting Go of Content to Unlock Knowledge
Everything old is new again β with a twist of AI
Filter by Category
Filter by Author
Everything old is new again β with a twist of AI
Posted by Will Sansbury
In the modern world of continuous delivery, "releases" are no longer the important mega-event they once were. So why are we still planning everything around them?
Posted by Will Sansbury
Everything old is new again β with a twist of AI
I started my career as a technical writer. When I was introduced to AuthorIT and the concept of content reuse β write once, publish everywhere βΒ I was obsessed. With reuse, technical communication teams could increase the quality and quantity of what they produced significantly. But it required a dedicated team of specialists, which put it out of reach for most companies.
I didn’t expect to revisit that problem twenty-something years later. At Rhythm Software, a 35-person association management software company, we get a lot of the same questions β and the answers all live in people’s heads. Our CEO had charged the team with finding a way to capture that knowledge to power an AI chatbot that could deflect support tickets. We’d been at it for a while, trying the obvious approaches β long-form prose, recorded walkthroughs, transcripts fed to the AI. Nothing was working well enough.
Two days ago the framing shifted: what if instead of capturing knowledge as prose, we captured it as structured records? I didn’t find a solution so much as stumble into a system.
We’re calling it the ATHENA Knowledge Management System.
We’re a small team β we knew we couldn’t take on anything that required sustained effort from the people who were already at capacity. So everything we tried was optimized for low overhead. We started with written explanations. When that proved too slow, we tried having people record video walkthroughs instead β talk through a module, transcribe it, feed the transcript to the AI. Lower cost of capture, same idea.
It failed on two fronts.
The first problem was that even with video, we were still asking busy people to produce something comprehensive on a topic they lived in day to day. The recordings were useful but incomplete β you get what someone thinks to mention, not necessarily everything that matters.
The second problem was quality. Even when we had decent prose, the chatbot answers weren’t great. An LLM retrieving a long narrative document and trying to answer a specific question from it produces vague, inconsistent responses. The document might explain how auto-renewal works in general β but if the answer to “why isn’t my membership auto-renewing?” depends on three specific configuration fields all being true simultaneously, a prose paragraph probably doesn’t surface that cleanly.
Prose is written for human readers navigating a document. It’s not structured for a machine trying to answer a precise question. I needed something different.
Then I looked more carefully at something we already had. Rhythm’s API documentation is some of the best I’ve encountered in my career β exhaustively detailed, consistently structured, covering every endpoint, every field, every status value across all fifteen modules. A huge amount of product knowledge is already encoded in it. If I could harvest that, I’d have a foundation that didn’t depend on anyone finding time to write or record anything.
The question was how. Feeding raw API docs into an LLM had the same problem as prose β too much noise, not enough signal. But what if instead of transforming API docs into long-form explanations, I transformed them into something more granular? Not documents, but discrete facts. Typed records with defined fields. Something structured enough that a machine could reason about it precisely.
That’s where the schema came from.
I landed on seven record types that together cover everything you’d want to know about a software product:
These aren’t articles. They’re not written for any audience. They’re facts, captured in a form that a machine can query and reason about. An LLM reading a Rule record doesn’t get a paragraph explaining the rule to a customer β it gets the condition, the behavior, the exceptions, and the common misconceptions. It figures out how to explain that to whoever’s asking.
Every record also carries governance metadata: an owner, a status (needs-review, current, deprecated), a last reviewed date, and a review frequency. Knowledge has an expiry date. The system should know what needs to be checked.
I didn’t arrive at that schema alone β Claude helped me work through it, challenging my initial thinking and suggesting record types I hadn’t considered. With the structure defined, the next question was: how quickly could it actually be populated?
Rhythm publishes OpenAPI specs for all fifteen modules of our platform, so I started there. I fed those specs into Claude and asked it to translate them into structured knowledge records β not to summarize the documentation, but to extract the facts and represent them in the schema I’d defined. Concepts, rules, state machines, configurations, procedures. Claude did the translation; I validated the output.
The speed was honestly shocking. Within hours I had a structural foundation across all fifteen modules β 847 records. I stored everything in a headless CMS β a highly configurable content store that exposes everything via API. That API supports MCP, a protocol that lets Claude connect to the records directly as a live data source. Once I had that connection in place, I had a working chatbot prototype answering real product questions. The whole thing came together in about three hours.
Those records are also cross-referenced β 1,042 links connecting related records across modules. A chatbot answering “why isn’t my membership auto-renewing?” needs more than a definition of auto-renewal. It needs to know that auto-renewal depends on three specific configuration fields all being true simultaneously. That’s a relationship, not just a fact β and without those connections, you get an AI that answers questions in isolation rather than one that reasons about how things fit together.
The OpenAPI specs gave me the structural layer fast β endpoints, fields, status values, workflows. But API docs describe what the product does. They don’t capture how it behaves in practice. The gotchas. The edge cases. The things you only learn from actually using it.
That knowledge exists β it’s just scattered. It lives in screen recordings and demo transcripts, in internal documents and Slack threads, in support calls and the heads of people who’ve been working with the product for years. The challenge isn’t that it doesn’t exist β it’s that it’s unstructured and inaccessible.
So I built a second ingestion pattern alongside the API specs one. For each source β a recording, a transcript, an internal document, a Slack thread β I feed it through Claude with the existing records as context and ask it to extract knowledge nuggets that aren’t already captured. Claude maps each finding to the appropriate record type β a new Rule, an augmented Concept, a missing Configuration β and flags it for review.
I tested it with screen recordings of someone walking through the Accreditation module. The results were immediate. The recordings surfaced things that aren’t anywhere in the API documentation:
Every one of those is the kind of thing that causes a support ticket. None of them are in the specs.
Each source type tends to produce different kinds of knowledge. Demos surface behavioral context. Support calls surface failure modes. Internal docs surface decisions and policies. That’s why the multi-source approach produces something richer than any single source could.
The API specs give you the skeleton. Everything else gives it life.
The knowledge layer is live and the chatbot prototype is working. Next step is an internal pilot β putting it in front of implementation and support staff with real questions and watching where it falls short.
The schema has already evolved once from real use. Someone posted a question in our internal #knowledge channel asking about our WCAG 2.1 compliance posture. I went to add the answer and realized I had nowhere to put it β it wasn’t product behavior, it wasn’t a configuration, it was a business decision that affects how we build and what we commit to customers. So I added an eighth record type: Policies. Things like accessibility commitments, data retention rules, support response standards.
That question didn’t just reveal a gap in our knowledge β it revealed a gap in the schema itself. Which is exactly the kind of signal the system is designed to surface. The schema isn’t finished. It grows as real questions expose what’s missing.
To make that signal flow reliably, I vibe-coded an internal chat interface β a React app that connects to the knowledge system and gives team members two ways to interact with it. They can ask a question and get an answer drawn from the published records, or they can proactively share something they know isn’t documented yet. In both cases, the bot captures what they share, structures it into a draft record, and queues it for the knowledge manager to review. Nothing goes live without human sign-off, but the capture happens in the conversation itself β no separate tool, no form to fill out, no ticket to file.
I’m not a developer. But with Claude doing the heavy lifting, building and iterating a working solution took a few hours.
Here’s what using it actually looks like.
Correcting a wrong answer:
Team member: How do I create a site visit?
Athena: (answers from published records)
Team member: That’s mostly right, but you also need to make sure the Visit Type has been set up in Program Settings first β otherwise that dropdown will be empty and there’s no way to proceed.
Athena: Got it β I’ve added that as a prerequisite to the “Create a Visit” procedure. Here’s how it currently reads for published users:
(Athena’s current answer about creating a site visit)
And here’s how it will read once the knowledge manager reviews and publishes the draft:
(The same answer, now with the Visit Type prerequisite included)
Does the updated version look correct?
The bot answers the original question twice β once from the published record, once from the draft β so the team member can confirm the correction landed before the session ends.
Sharing something that isn’t documented yet:
Team member: When the portal evaluation checkbox is checked but no team members have been assigned yet, the system lets you save the visit but the evaluations never actually get sent. It’s a common source of confusion.
Athena: That sounds like a known error β I’d capture it as an Error / Exception. A couple of quick questions: does this come up frequently in support tickets? And is there a specific error message, or does it just silently not send?
Team member: Pretty frequent. No error β it just silently does nothing.
Athena: Got it. I’ve created a draft Error / Exception: “Portal Evaluations Not Sent When No Team Members Are Assigned” β marked as In Progress and ready for the knowledge manager to review.
The knowledge never existed in any document. Now it does β structured, typed, and queued for review. The team member just had a conversation.
Every documentation system I’ve ever worked with has the same fundamental problem: it starts accurate and drifts. The product changes, the docs don’t, and eventually they’re worse than useless β they’re actively misleading. The only fix anyone has ever found is to throw human effort at it on a recurring basis, which is expensive and doesn’t scale.
I was stuck on that reality when a radical thought hit me: what if the chatbot itself is how the knowledge stays current?
When a staff member gets a wrong or incomplete answer, they don’t file a ticket or flag it for someone to review later. They just correct the bot in the conversation β “actually, that’s not right, here’s how it works.” The chatbot takes the correction, immediately structures it into a proposed knowledge record update, and surfaces it to the knowledge manager for review. The knowledge manager iterates if needed and approves it. The record gets updated. The system gets smarter.
The knowledge manager is the human in the loop β nothing gets published without their sign-off β but the heavy lifting of translating a correction into a structured update happens automatically. The staff member just has a conversation. The chatbot and the knowledge manager do the rest.
Every interaction is a potential improvement. The more people use it, the more corrections surface, the more it reflects reality. It doesn’t decay over time. It compounds.
And that one role β a knowledge manager reviewing a stream of AI-generated updates triggered by real questions, not scheduled review cycles nobody has time for β makes the company’s knowledge genuinely accessible at scale. Support stops answering the same questions repeatedly. The people who know the product inside and out stop carrying the invisible labor of filling in everyone else’s gaps. The knowledge becomes infrastructure: available to anyone, at any time, without the tax on the people who hold it.
I spent the early part of my career building structured content systems β and they worked. DITA and AuthorIT delivered on their promise. The same content really did render cleanly to a help site, a PDF, and an in-app tooltip. But it required a dedicated team of trained writers to make it happen. That was fine for large enterprises with documentation budgets. For everyone else, it was out of reach.
What I stumbled into with Athena goes further than what those systems could do β and turns out to be practical for teams of any size.
Capture knowledge once as structured facts β not content, not prose, just facts β and let the LLM render it for whoever’s asking. The same verified records that power the chatbot can render a KB article for a customer, a training module for a new consultant, an RFP response for a sales team, an in-app help system for a portal user. You capture the knowledge once, and the presentation takes care of itself.
Together with a feedback loop that keeps the knowledge current, you get something that hasn’t really existed before: a single canonical knowledge layer that serves every audience, in every format, and gets more accurate the more it’s used. Not by design β that’s just what happened when the pieces came together.
Any organization with complex knowledge and multiple audiences has this problem. The gap between what you know and what you can make usable has always been the bottleneck.
That gap just got a lot smaller.
For those curious about the architecture: the whole system runs on three pieces that connect cleanly.
Sanity is the knowledge store. It’s a highly configurable headless CMS with a flexible schema and a full API β which matters because the records need to be queryable by both humans (via Sanity Studio, where the knowledge manager works) and machines (via API, where Claude retrieves them). One thing worth noting: the Sanity Studio interface is defined in code, which would normally put it out of reach for someone without an engineering background. I’m a product leader with more coding familiarity than most, but I’m not a developer. Claude wrote the schema configuration. That matters for who this approach is available to β you don’t need an engineering team to build it.
MCP (Model Context Protocol) is the bridge. Sanity exposes an MCP server that lets Claude connect directly to the knowledge records as a live data source β not a static document dump, but a structured, queryable store. When the chatbot answers a question, it’s retrieving the most current version of the relevant records in real time.
Claude does two distinct jobs. On ingestion, it acts as a translator β taking a source (an OpenAPI spec, a transcript, a Slack thread) and extracting structured knowledge records in the defined schema. On retrieval, it acts as a renderer β taking structured records and translating them into whatever the audience needs: a conversational answer, a KB article draft, a training outline. The same model, doing opposite things, with the knowledge layer in between.
The insight that made this work is that separating ingestion from retrieval β using Claude for both but keeping the structured records as the stable layer in the middle β is what gives the system its flexibility. Change the audience, change the format, change the channel. The knowledge stays the same.
In product development, thereβs a perpetual tension between designers, developers, and product managers. By learning to balance the concerns of all of the parts of the team, we can craft truly great products.
I rarely disagree* with Harvard Business Review. But then I read this: A meeting-free day or even half-day may be your ideal, but you may never have this type of time. Waiting for a slice of project nirvana keeps you from getting started when you can. A better approach is to accept and work within […]