Dev Tools
Query live web data from Databricks SQL
Installs Nimble's search, extract, and Web Search Agent APIs as Unity Catalog table functions in Databricks. From there, any SQL query can pull live web search results, fetch and parse any URL, or run a Web Search Agent against a column of inputs — without leaving the warehouse or building a separate ingestion pipeline. Enriches Delta tables in place and powers Databricks Genie agents with fresh web data on every query.
Inputs
- Databricks workspace A serverless SQL warehouse with outbound networking enabled. The table functions are installed once with the Databricks CLI and a Nimble API key stored as a workspace secret, after which any SQL user can call them.
- A SQL query Once installed, call nimble_search, nimble_extract, or nimble_agent_run in any FROM clause with your query string, URL, or agent parameters.
Outputs
- Web search results as SQL rows: title, description, url, and content
- Parsed page content from any URL as markdown or HTML
- Nimble agent catalog with input schemas as queryable rows
- Structured agent results as a VARIANT column
- Delta tables enriched with live web data
- Genie tools for natural-language web queries
How it works
A 6-step setup and query flow.
- Authenticate the CLI The Databricks CLI authenticates to the workspace and creates a secret scope that holds the Nimble API key, so the key never appears in a function body or call site.
- Enable egress A one-time Workspace Preview toggle and a cold warehouse restart let the serverless SQL warehouse make outbound HTTPS calls, which the Python UDTFs need to reach the Nimble API.
- Deploy the table functions deploy_sql.py posts each SQL file to the warehouse, creating the nimble_integration catalog and installing the five table functions as re-runnable definitions.
- Query live web data Calling nimble_search or nimble_extract in a SQL FROM clause runs a live Nimble request and returns web results or parsed page content as typed rows; nimble_agent_run does the same for any Nimble agent, such as google_maps_search.
- Persist to Delta Wrapping any Nimble call in CREATE TABLE AS SELECT lands the web data as a governed Delta table that inherits Unity Catalog access controls, lineage, and audit logs.
- Register with Genie create_genie_space.py registers each function as a Genie tool, using the function and column comments as its spec, so a Genie space can answer natural-language web queries.
Stack
Nimble primitives plus the full runtime stack.
Nimble APIs
What it does
- search Powers nimble_search; returns live web results (title, description, url, content) for any query, with focus mode, depth, and result-count options.
- extract Powers nimble_extract; fetches and parses any URL into clean markdown or HTML, including JavaScript-heavy pages.
- Nimble Agents API Powers nimble_agent_run, nimble_agent_list, and nimble_agent_describe; runs any managed agent, such as google_maps_search, and returns structured results as a navigable VARIANT column.
3rd Party Tools
Role
- Databricks Serverless SQL Hosts the Unity Catalog Python UDTFs and SQL wrapper functions and runs every Nimble call from inside the warehouse.
- Unity Catalog Governs the nimble_integration catalog, enforcing access controls and lineage and exposing functions as Genie tools.
- Databricks CLI v0.205+ Authenticates to the workspace and deploys the SQL files through the Statements API.
- Python UDTF runtime Runs the HTTP call inside the warehouse and yields typed rows back to SQL.
- Databricks Genie Registers the table functions as agent tools for natural-language web search.
Reach out if you have any questions.
Talk to an Expert