Dev Tools
Fact-check documents with LiteLLM and Nimble
Nimble search runs one query per factual claim found in a document, and the retrieved pages become the evidence a model rules on. Nimble is registered in LiteLLM as the nimble/search provider, so the search calls and both model calls go through one interface and report their cost in the same units. A cheap model pulls the claims out, a strong model judges them, and each verdict names the URL that decided it.
Quick Start
Inputs
- A document Any markdown or text file whose factual claims need checking, such as a blog draft, a research summary, a press release, or an AI-generated report. The shipped example is a 333-word article.
Outputs
- The document annotated claim by claim, each marked supported, contradicted, or unverifiable
- The deciding source URL behind every verdict
- A correction for each contradicted claim
- A list of what was set aside as opinion, and what the sources could not settle
- Search spend, token spend, and cost per verified claim, reported separately
- Reports in HTML and Markdown
How it works
A 5-phase pipeline.
- Separate claim from opinion A cheap Claude model reads the document and returns every checkable factual claim as structured JSON, setting opinion, prediction, and recommendation aside.
- One search per claim Nimble search runs a query for each claim, concurrently, returning ranked results with title, url, and full page content.
- Trim each page down Nimble returns whole pages as markdown, which can run to hundreds of thousands of characters, so each result is cut to a claim-relevant window by rarity-weighted keyword scoring after nav menus and tables are stripped.
- Judge against the sources A strong Claude model rules on each claim, returning supported, contradicted, or unverifiable, the URL that decided it, and the correct figure where the document is wrong.
- Annotate and price it The document is rendered back marked up claim by claim, with search spend and token spend accounted separately and a cost per verified claim.
Stack
Nimble primitives plus the full runtime stack.
Nimble APIs
What it does
- search Returns live web results for each claim, with structured page content used as the evidence behind every verdict. Registered inside LiteLLM as the nimble/search provider.
3rd Party Tools
Role
- litellm One interface for the search provider and both models, with an optional proxy that puts every call on a single dashboard and ledger.
- claude-haiku-4-5 Pulls the checkable claims out of the document, once per run.
- claude-opus-5 Judges each claim against the retrieved pages, once per claim.
- python 3.10 to 3.14 The range LiteLLM supports.
Reach out if you have any questions.
Talk to an Expert