May 7, 2026

Building Data Apps with Nimble and Claude: Practices That Work

min read

Copied!

Tom Shaked

No items found.

May 7, 2026

Building Data Apps with Nimble and Claude: Practices That Work

min read

Copied!

Tom Shaked

No items found.

Table of Contents

Connect with Nimble

Connect on Slack

Building Data Apps with Nimble and Claude: Practices That Work

Claude is one of the most capable reasoning models available. Nimble is one of the most powerful web data platforms. Put them together and you can build data apps that collect from anywhere on the web, reason over what they find, and surface results that would be impossible to get any other way.

But getting there isn't always straightforward.

The mistakes are quiet — wrong configuration, missing resume logic, brittle logic — and they tend to surface at the worst time. This guide is the shortcut: hard-won patterns for making the most of this stack without the trial and error.

Setting up Nimble with Claude

Nimble's CLI installed, an API key set, and the Nimble plugin running in Claude Code. That gives Claude direct access to the full Nimble API — inline, without leaving the coding environment.

Get an API key from Account Settings > API Keys.

Install the CLI:

npm i -g @nimble-way/nimble-cli

Set your key:

export NIMBLE_API_KEY="your-api-key-here"

To persist across sessions, add it to ~/.claude/settings.json:

{ "env": { "NIMBLE_API_KEY": "your-api-key-here" } }

Install the Claude Code plugin:

claude plugin marketplace add Nimbleway/agent-skills && \\\\ claude plugin install nimble@nimble-plugin-marketplace

Three components install:

ComponentWhat it doesnimble-web-expert skillOne-off extractions, searches, and agent calls from a Claude conversationnimble-agent-builder skillBuild, test, and publish reusable extraction agentsMCP serverFull Nimble API access — search, extract, map, crawl, agents

1. Test inline before you write a script

Don't write a script against a response format you've only read in the docs.

Test the call inline first. One call tells you what the response actually looks like: which keys exist, what format the data arrives in, whether the driver tier gets through.

nimble --transform "data.parsing" agent run \\ --agent amazon_pdp \\ --params '{"asin": "B0DLKFK6LR"}'

When one inline call returns a clean response and you understand the shape, move to Python. The inline test becomes the extraction function — no surprises at call 50.

For anything involving multiple candidate tools or models, test all of them before committing. A model that silently returns empty results, a driver tier that gets blocked, a response key that doesn't exist for a particular site — find these with one call, not midway through a 300-call run.

2. Know your tool before you commit to it

Nimble has several extraction approaches and driver tiers. The wrong choice doesn't produce an error — it produces empty data or a challenge page with a 200 status code.

Driver tier determines access. vx6 handles most static sites. JavaScript-rendered pages and sites with active detection need vx10 or vx10-pro.

Agent vs. extract vs. batch are different tools. Agents return structured, parsed data for specific platforms. Extract returns raw HTML or markdown from any URL. Batch runs extract across up to 1,000 URLs in parallel. Using extract on a page that has a dedicated agent means writing a parser you didn't need to write. Using sequential extract calls for 500 URLs means waiting hours you didn't need to wait.

SDK vs. raw requests behave differently. For agents that run as browser sessions, the SDK handles session management and timing internally. Raw requests don't. A hard 60-second timeout on requests will cut off a call that would have completed at 62 seconds.

3. Define your schema before you collect

The schema is the contract between collection and analysis. Define it first.

Write out the exact fields you need, their types, and where they come from. If you're pulling from two sources, normalize them to the same shape before they leave the collection layer.

LISTING_SCHEMA = { "source": str, # origin platform "url": str, # direct link to listing "price": int, # in cents "title": str, "category": str, "fetched_at": str, # ISO timestamp }

This prevents discovering mid-analysis that two sources use different field names for the same concept, and realizing after a full fetch run that you forgot a field needed for scoring.

If a source can't provide a field, decide upfront what the default is — null, a sentinel value, an inferred default. Don't let that decision happen inside the parser.

<aside>💡

When building your own custom web search agents with Nimble’s agent builder skill, you can define your data schema in-agent - thus returning the exact schema you need directly from the data collection phase.

</aside>

4. Decouple collection from analysis

Store raw responses before transforming them. Always.

Collection and analysis have different failure modes. If your analysis logic breaks, you want to fix and rerun it without going back to the source. That's only possible if the raw data is on disk.

def save_response(item_id, agent, raw_text, fmt): path = RESPONSES_DIR / f"{item_id}.json" payload = json.loads(path.read_text()) if path.exists() else {"id": item_id, "responses": {}} payload["responses"][agent] = {"raw": raw_text, "format": fmt} path.write_text(json.dumps(payload, indent=2))

Tag the format at collection time, not during analysis. If sources produce different output shapes, that metadata belongs with the raw response from the moment it arrives. Downstream code should never have to guess what it's looking at.

Reprocessing 100 cached responses takes seconds. Re-fetching 100 live pages takes an hour.

5. Write scripts that are safe to re-run

A long-running fetch script isn't done until it's resumable.

Save each response immediately when it arrives. Skip any item already cached before making the API call. A crash loses at most the calls in flight. A re-run picks up where things stopped.

def already_cached(item_id): path = RESPONSES_DIR / f"{item_id}.json" if not path.exists(): return False data = json.loads(path.read_text()) return all(agent in data.get("responses", {}) for agent in AGENTS) pending = [item for item in all_items if not already_cached(item["id"])]

This also makes partial runs safe. Add a new agent to an existing dataset, run the same script — it skips what's done and fetches what's missing.

6. Monitor long-running jobs

A script running for 30 minutes with no output is indistinguishable from a stuck script.

A background monitor thread costs almost nothing:

import threading, time def monitor(stop_event, interval=30, warn_after=90): while not stop_event.is_set(): time.sleep(interval) now = time.time() with lock: slow = [(agent, item_id, now - t0) for agent, item_id, t0 in in_flight.values() if now - t0 > warn_after] print(f"[{completed}/{total}] in-flight: {len(in_flight)}") for agent, item_id, age in slow: print(f" ⚠ SLOW ({age:.0f}s) [{agent}] {item_id}") stop = threading.Event() threading.Thread(target=monitor, args=(stop,), daemon=True).start()

Soft warning: log the call if it's taking longer than expected. Hard timeout: kill it and record the failure. Log the specific agent and item ID — you're debugging a known call, not guessing from a count.

7. Use batch for scale, S3 for delivery

More than a handful of URLs? Use the Batch Extract API. Up to 1,000 URLs per request, processed in parallel. You get a batch_id back immediately — no thread pool, no retry logic.

from nimble_python import Nimble nimble = Nimble(api_key="YOUR_API_KEY") response = nimble.extract_batch( inputs=[{"url": url} for url in listing_urls], shared_inputs={ "render": True, "driver": "vx10-pro", "formats": ["html"], "country": "US", } ) batch_id = response.batch_id

Poll with the lightweight progress endpoint:

while True: progress = nimble.batches.progress(batch_id) print(f"{progress.completed_count}/{progress.total} ({progress.progress:.0%})") if progress.completed: break time.sleep(15)

For jobs longer than a few minutes, add S3 delivery. Results land in your bucket as each task completes — no polling loop, no retrieval code, no 7-day expiration:

response = nimble.extract_batch( inputs=[{"url": url} for url in listing_urls], shared_inputs={ "render": True, "formats": ["html"], "storage_type": "s3", "storage_url": "s3://your-bucket/results/", "storage_compress": True, } )

Add a callback_url and you get a push notification when the batch finishes — no polling loop needed at all:

8. Use AI for semantic problems

Rule-based parsing works until it meets real data. String similarity scores two answers as 20% identical because one is terse and one is verbose. A regex grabs the first bold word — a table header, a transition word — instead of the conclusion.

When the question is "do these mean the same thing" or "what is this document's position on X," use a model. Claude Haiku handles these tasks well and costs almost nothing:

import anthropic client = anthropic.Anthropic(api_key="YOUR_API_KEY") def semantic_label(text_a, text_b): msg = client.messages.create( model="claude-haiku-4-5-20251001", max_tokens=100, messages=[{"role": "user", "content": f'Do these two statements express the same position? Answer with just "yes" or "no".\\n\\nA: {text_a}\\nB: {text_b}'}] ) return msg.content[0].text.strip().lower()

Roughly $0.002 per call. Less than the time spent debugging a rule-based approach that still fails on edge cases.

The split: regex and string matching for structural extraction — finding fields, parsing known formats, splitting on delimiters. Models for interpretation — normalizing freeform text, judging equivalence, extracting positions from unstructured prose.

Wrapping Up

These patterns won't cover every edge case, but they'll eliminate the most common ones. Test inline before scripting, store raw data before transforming it, use batch for anything at scale, and reach for a model when rules start breaking down.

Get those right and the rest is just building.