Cut the Page. Get the Payload.

Web API Now Supports Direct Access to XHR Endpoints

Sometimes, the data you need isn’t in the HTML—it’s behind the scenes, loaded through XHR calls once the page renders.

If you’ve ever opened DevTools, watched dozens of requests fire off in the background, and wished you could just hit one of those directly… now you can.

Nimble Web API now supports direct access to internal XHR endpoints. No rendering. No parsing. Just the structured data the frontend is already pulling.

Why We Built It

Previously, accessing XHR data required rendering the full page, enabling render: true, and capturing network traffic via network_capture. While powerful, this approach had tradeoffs:

• Heavier requests
• Longer render times
• More complexity

But in most cases, you already know the exact XHR endpoint the site is hitting. So why not just call it directly?

Now, you can do exactly that—with a single API call.

How to Use It

Here’s what a basic request looks like:

import requests, base64

auth = base64.b64encode(b"user@example.com:your_password").decode()

res = requests.post(
    "https://api.webit.live/api/v1/realtime/web",
    headers={
        "Authorization": f"Basic {auth}",
        "Content-Type": "application/json"
    },
    json={
        "url": "https://www.lowes.com/wpd/5001374857/productdetail/1875/Guest",
        "country": "US",
        "is_xhr": "true"
    }
)

print(res.json())

No rendering. No JavaScript execution. You get back exactly what the frontend sees: a structured JSON payload from the server that includes:
‍

{
  "productDetails": {
    "5001374857": {
      "product": {
        "omniItemId": "5001374857",
        "additionalBadges": [
          "Only at Lowes"
        ],
        "brand": "Charbroil",
        "brandId": "2077",
        "banners": [
          "LowesCA",
          "USHI"
        ],
        "barcode": "047362336507",
        "barcodes": [
          {
            "code": "047362336507",
            "qty": "1",
            "hierarchyLevel": "EACH",
            "consumerUnit": "true"
          }
        ],
        "categories": {
          "103460": "GAS_GRILL"
      },
      "location": {
        "price": {
          "type": "STORE",
          "displayText": "STORE",
          "pricingDataList": [
            {
              "usageType": "SELLING",
              "displayType": "WAS",
              "priceType": 2,
              "basePrice": 249,
              "finalPrice": 199,
              "retailPrice": 199,
              "savings": {
                "totalPercentage": 20,
                "endDate": 1750219200000,
                "endDateTime": "2025-06-18T00:00:00.000Z",
                "saveComponent": [
                  {
                    "type": "WASNOW",
                    "percentage": 20,
                    "dollarDiscount": 50
                  }
                ],
                "totalSaving": 50
              },
              "priceDisplayCode": 9,
              "dotd": false,
              "sourceSystemEventId": "785292567"
            }
          ]
        },
        ...

The full response includes much, much more data that we’ve truncated for clarity.

Real-World Example: Product Data from Lowe’s

Let’s say you’re tracking home improvement product availability or pricing from Lowe’s.

The product page itself loads a lot of assets—images, ads, layout scripts, and dozens of DOM manipulations. But the actual product data? It comes from a single XHR call, something like:

https://www.lowes.com/wpd/5001374857/productdetail/1875/Guest

How to Find XHR Endpoints (Using Chrome DevTools)

If you’re not sure where the data is coming from, here’s a quick way to find it:

1. Open the site in Chrome
2. Right-click → Inspect, then go to the Network tab
3. Set the filter to XHR
4. Refresh the page or interact with the element you want
5. Watch for requests that return JSON (check the Response tab)
6. Copy the request URL — that’s your XHR endpoint
7. Plug it into the Web API with "is_xhr": true

Pro tip: you can inspect headers and payloads to simulate different request types (POST, custom headers, cookies, etc.)

When to Use It

This feature is ideal for:

• Product feeds & pricing — Hit internal APIs used by eComm frontends
• Structured content — Reviews, metadata, spec sheets, listings
• Performance-sensitive workflows — When rendering adds unnecessary delay
• Data pipelines — Feeding raw structured JSON directly into a model or database

Best Practices

• Use it only when you’ve confirmed the endpoint is stable
• Avoid relying on undocumented internal APIs without fallback logic
• Combine with country / geo targeting when the API varies by region

Skip the Render. Get to the Data.

This feature unlocks a direct path to the structured data you care about. No DOM traversal. No JS overhead. Just clean, fast, frontend-level data access—at scale.

Read the full docs to get started →

‍