Fetch best practices

This page covers when to use the "pro" mode, when to render JavaScript, when to extract raw HTML or image URLs, when to ask Fetch for typed JSON, and how to pair Fetch with Search.

Choosing a retrieval mode

Use "standard" for mode on regular pages. It is the default and the most cost-effective option. Use "pro" for mode on hard-to-retrieve pages. Pro fetch delivers significantly higher success rates on hard-to-retrieve pages. It has a higher per-call cost. mode and renderJs are independent. Choose mode based on how difficult the page is to access, and choose renderJs based on whether the page needs JavaScript to display its content.

{
  "url": "https://example.com/hard-to-retrieve-page",
  "mode": "pro",
  "renderJs": false
}

When to render JavaScript

Many modern sites load content via JavaScript. Within an agentic pipeline, setting renderJs to true is the safer default: it ensures the full content of the page is extracted and provided to the agent. Setting renderJs to false is appropriate when targeting a known set of static pages, once the specific site has been confirmed to return full content without JavaScript rendering. The default is false and the no-JS rate is cheaper ($0.001 vs $0.005 per call). Indicators that renderJs should be true:

The returned markdown is substantially shorter than the live page.
The output contains repeated boilerplate such as “Loading…” or “JavaScript is required”.
Sections visible in a browser are missing entirely from the returned markdown.

Pairing with Search

A common pattern uses Search to find candidate URLs and Fetch to retrieve them in full when the agent needs the entire page rather than the snippets returned by Search.

search_results = client.search(
    query="Datadog pricing tiers and per-host costs",
    depth="standard",
    output_type="searchResults",
)

for result in search_results.results[:3]:
    page = client.fetch(url=result.url, render_js=True)
    # feed page.markdown into your LLM, or pass schema to get typed JSON in data

Selection of pages to fetch can be done:

agentic: ask the agent to fetch the most relevant pages based on the page snippets returned by Search.
programmatic: set maxResults in Search and fetch all URLs.

Working with raw HTML and images

Fetch returns clean markdown by default. Two flags add adjacent representations of the same page when the markdown alone is insufficient.

`extractImages`

Set extractImages to true to additionally return a list of image URLs found on the page (product photos, charts on a financial page, recipe images). Adds latency; enable only for workflows that consume image URLs.

`includeRawHtml`

Set includeRawHtml to true for:

workflows that need to operate on the full page HTML;
pages whose structure (complex tables, embedded widgets) is erased during markdown conversion.

Both flags default to false and can be combined with renderJs set to true. They are independent of schema.

Structured output

Use schema when you already have a URL and want typed JSON from this page. Default Fetch stays markdown-only; schema is the switch that adds data. Put the shape and per-field meaning in the schema. Field descriptions tell the model what to look for. schema without instructions is valid. Use instructions for global rules the schema cannot express:

“public list prices only”
“express monetary values in USD”
“if a role is listed in several cities, emit one item per city”

instructions only steers schema fill. It does not trim or rewrite markdown. instructions without schema returns a 400. Maximum 4,000 characters. Keep the schema a JSON Schema object (type "object"). Keep it shallow: primitive fields and one level of arrays are more reliable than deep nesting. If a field is not on the page, it is omitted from data — including fields marked required. The model does not invent values. Fetch with schema reads this URL only. It does not follow links and it does not crawl a site. Adding a schema costs a flat $0.001 on top of whatever the call would have cost as markdown. See the Fetch pricing table for every combination.

Job	Use
One known URL, markdown	Fetch
One known URL, typed JSON from that page	Fetch with `schema`
Many rows from a listing page, possibly following links	Extract (closed beta)
No URL yet, structured JSON from the web	Search with `outputType` `"structured"` and `structuredOutputSchema`

Search keeps structuredOutputSchema. Fetch uses schema and instructions.

{
  "url": "https://www.linkup.so/careers",
  "schema": {
    "type": "object",
    "properties": {
      "jobs": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "position": {
              "type": "string",
              "description": "The job title or position"
            },
            "city": {
              "type": "string",
              "description": "The city where the job is located"
            }
          }
        }
      }
    }
  },
  "instructions": "If a position is listed in multiple cities, emit one jobs item per city so jobs[].city is a single city."
}

Common pitfalls

Bad → Fix pairs grounded in the documented constraints of Fetch (HTML and PDF, 20 MB HTML / 100 MB PDF caps, anonymous, optional JavaScript rendering for HTML pages). SPA fetched without renderJs. The markdown comes back near-empty because the content is rendered client-side.

Bad

{ "url": "https://app.example.com/dashboard", "renderJs": false }

Fix

{ "url": "https://app.example.com/dashboard", "renderJs": true }

Passing an unsupported binary URL. Fetch supports HTML and PDF. ZIPs, images, videos, and other binary content return a 400 error.

Bad

{ "url": "https://example.com/archive.zip" }

Fix

{ "url": "https://example.com/whitepaper.pdf" }

Expecting Fetch to retrieve content behind a login wall. The endpoint is anonymous and returns what a logged-out visitor would see.

Bad

{ "url": "https://app.example.com/private/report" }

Fix

{ "url": "https://example.com/public/press-release" }

Sending instructions without schema. Structured output has no prose mode. instructions only steers schema fill, so the request is rejected.

Bad

{
  "url": "https://example.com/pricing",
  "instructions": "Return list prices in USD"
}

Fix

{
  "url": "https://example.com/pricing",
  "schema": {
    "type": "object",
    "properties": {
      "plans": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": { "type": "string", "description": "Plan name" },
            "priceUsd": { "type": "number", "description": "Public list price in USD" }
          }
        }
      }
    }
  },
  "instructions": "Use public list prices only and express monetary values in USD."
}

Expecting Fetch to follow links or fill fields that are not on the page. Fetch reads this URL only. Missing fields are omitted from data, not invented. For many rows across a listing, use Extract.

Get Started

Endpoints

Platform

Guides

Tutorials

FAQ

Choosing a retrieval mode

When to render JavaScript

Pairing with Search

Working with raw HTML and images

`extractImages`

`includeRawHtml`

Structured output

Common pitfalls

Resources

​Choosing a retrieval mode

​When to render JavaScript

​Pairing with Search

​Working with raw HTML and images

​extractImages

​includeRawHtml

​Structured output

​Common pitfalls

​Resources

Choosing a retrieval mode

When to render JavaScript

Pairing with Search

Working with raw HTML and images

`extractImages`

`includeRawHtml`

Structured output

Common pitfalls

Resources