Improve AI Answers About Your Product

A developer asks an assistant to write an integration against your API. The generated example uses a parameter you renamed last month. You fix the example in a support reply, but the old parameter is still in your quickstart, so the next person can hit the same problem.

You can improve the material an assistant retrieves even when you don't control the assistant. Start with the pages behind a wrong answer, check them against the current product, and give the correction a path through review and publication.

Find the source behind the wrong answer

Ask for the prompt, the generated answer, any cited URLs, and the product version the customer is using. Keep credentials and customer data out of a shared investigation. If the tool offers a source view, open it. A familiar citation can point to an old version or a page that doesn't support the claim.

For a renamed parameter, search your own content for both names. Check the reference, quickstart, tutorials, help center, and copyable examples. A reference page can be correct while an older tutorial still teaches the wrong request.

Record what you find in a small issue:

• The incorrect instruction and the page that contains it
• The supported version and the correct behavior
• The release or merged pull request that changed it
• An owner who can confirm the correction

If the source is already correct, preserve the failing example for a separate assistant evaluation. Retrieval and generation can still go wrong with accurate documentation. Updating a page cannot guarantee that a model will fetch it or use it correctly.

Give each task a clear, current page

A useful quickstart says which SDK version it uses, lists the prerequisites, and includes a runnable request. The response should show enough detail for a reader to tell whether it worked.

Suppose an SDK replaces project_id with workspace_id. Update the explanation and every copyable request together. If the old option remains valid for an older release, keep that guidance under a clearly marked version. A page that mixes both names without explaining the version boundary is hard for a person or an assistant to use.

Use headings that describe the task: "Create an API key" or "Retry a rate-limited request." Put an important qualification next to the example it changes. A warning about required permissions at the bottom of another page is easy to miss when a tool retrieves only one section.

Keep a canonical URL for the current guide. Redirect retired pages when there is a replacement, and label archived versions. Treat copied content on a marketing site or help center as part of the same maintenance job.

Publish formats your tools can use

HTML remains useful for readers and tools that browse the web. Markdown can offer the same instructions with less page furniture. Generate both from the same source so a corrected command reaches each output.

The llms.txt proposal describes a small Markdown overview with links to detailed content. If you publish one, link to the pages a developer needs first: setup, authentication, version policy, and core tasks. Check the links after a route change. Publishing the file does not guarantee that a particular assistant will read it or cite you.

Structured JSON is useful when you have a consumer that needs predictable fields, such as a page title, version, section text, and canonical URL. Decide which fields the consumer uses before building another output to maintain.

An MCP integration lets a compatible client call tools exposed by a server. For documentation, those tools can search and read your current pages. The user still needs a compatible client and the connection configured. Keep private content behind its existing access checks, and test that an unauthorized reader cannot retrieve it through a tool.

Thally's Agent Layer provides documentation formats and retrieval tools. Evaluate them with a real task: retrieve the authentication guide, inspect its canonical URL, and compare the returned example with the published page. Source links make an answer easier to check; they don't prove that every sentence is correct.

Connect code changes to content review

After a merge that changes user-visible behavior, name the affected pages while the context is fresh. The pull request author can identify a changed default or permission rule more easily than someone reconstructing it weeks later.

For each proposed correction, attach the relevant change and quote the old instruction. A reviewer should be able to compare the old sentence, proposed sentence, and product behavior without searching through an entire repository.

Thally's public Track preview analyzes recent merged pull requests using read access. It can give you candidates to investigate. Cloud's separately authorized automation can prepare documentation update PRs for review. Check the scope of the connection and the changes covered before assuming an analysis includes a whole release.

An internal refactor may need no public update. Record that decision when it helps the release owner distinguish reviewed changes from overlooked ones.

Review, publish, and try the task again

Have someone who understands the feature confirm the behavior. Have the content owner check whether a reader can follow the page from the stated prerequisites. Run copyable commands in an appropriate test environment and remove any credentials before saving results.

After approval, check the deployed page and its alternate outputs. A source edit can be correct while the live site still serves an older build. Open the canonical URL, inspect the relevant example, and confirm that Markdown or tool responses contain the same correction.

Finally, repeat the original question in the assistant you are evaluating. Record the date, tool, version, sources shown, and whether the example works. One successful answer is a useful check of that case. Keep a small set of recurring customer tasks and repeat them after important releases to see where errors return.

Start with the next wrong answer you receive. Fixing its source, verifying publication, and retrying the task gives you a repeatable process for improving what customers and their assistants can learn about your product.

Sources

• The llms.txt proposal
• Model Context Protocol architecture
• Thally Agent Layer
• Thally Track
• Thally documentation automation