Search best practices

This page covers how to pick a depth, how to phrase queries for each of "fast", "standard", and "deep", how to choose an output type, and how to apply source and date filtering on the Search endpoint.

Choosing depth

The depth parameter controls if / how agentic search is used within the search.

Use case	Recommended setting
Keyword style lookup at sub-second latency	`"fast"`
Instruction-based retrieval comparable to one or a few Google searches	`"standard"`
Breadth across adjacent keywords (news, research)	`"standard"` with explicit “run several searches with adjacent keywords”
Scrape one URL provided in the query and run a search	`"standard"`
Scrape several known URLs and run several searches	`"deep"`
Find a URL, then scrape it	`"deep"`

Rule of thumb: chat or keyword lookup → "fast"; if one or a few parallel Google searches would answer the question → "standard"; if a human would open multiple tabs or follow leads → "deep".

"standard" and "deep" can scrape URLs provided in the query. "standard" accepts one URL; "deep" accepts several and scrapes them with JavaScript rendering.

Per-depth guidance

fast
standard
deep

"fast" is keyword-only and bypasses the LLM entirely.It returns sub-second results for keyword-shaped queries where latency is the binding constraint. Currently in beta.

Behaviour

Single-pass, keyword-like search with no LLM involvement.
No query interpretation, no reformulation, no evaluation.
The query string is passed to the index as-is.
No in-query URL scraping, no sub-searches, no iteration.

When to use it

Conversational AI use cases where low latency is critical.
High-volume, low-latency pipelines.
Lookups for one specific piece of information

When not to use it

Anything that requires reading a page (use "standard" for one URL, "deep" for several).
Any query whose intent depends on instruction parsing, ordering, or multi-step retrieval — "fast" will treat the instructions as keywords.

Query shape

Keep queries short and keyword-shaped:

NVIDIA Q4 2024 revenue

Current EUR/USD exchange rate

"standard" runs a single iteration of agentic search.The agent interprets the query, can split it into parallel sub-searches when appropriate, and can scrape one URL provided in the query string.Outputs from one iteration are not reused in another (an extracted URL cannot be passed back into a follow-up step within the same call).Latency is approximately 1–3 seconds.

When to use it

Queries answerable by one well-targeted search.
Breadth across adjacent keywords for research and news, with an explicit instruction such as “run several searches with adjacent keywords”.
A single known URL that needs to be combined with a search.
High-volume or low-latency use cases where the answer is likely found in one pass.

Adjacent searches for breadth

"standard" can run multiple searches in parallel within a single call. The agent fans out automatically when a query implies it, but stating the dimensions explicitly gives the caller direct control over coverage.

Use case	Prompt
Company research	Build a profile on `{company}`. Run several searches to map (i) its products, (ii) its team, and (iii) its business model.
News	Find recent news about OpenAI. Run several searches with adjacent keywords.
Trends	What are people saying about AI agents on Twitter and Reddit? Run several searches.

For coverage-style workloads (news, trends), bare adjacent-search prompts are enough; the agent picks the angles. For workloads where specific dimensions matter (company research, competitive analysis), naming the dimensions in the prompt produces more reliable coverage.

In-query URL scraping (one URL per call)

"standard" can scrape one URL provided in the query. A combined search-and-scrape call:

Scrape the website linkup.so.
Also run a search to find articles, news, and posts mentioning linkup.so clients.
Based on the content from the website and from the search,
return a list of clients that use Linkup, with the source for each.

For workloads with several known URLs and adjacent searches, several "standard" calls or one "deep" call works best.

"deep" runs several iterations of agentic search.It supports sequential instructions, where outputs from one step are used in the next (for example, search first, then scrape a discovered URL).Latency is approximately 5–30 seconds.

When to use it

Sequential work where uterior steps depends on the first (locate a URL, then read it; find a list of pages, then scrape each).
Queries that explicitly require multiple searches and multiple scrapes.
Cases where information is not reliably found in a single pass.
Workloads where the requirements are undetermined and reliability matters more than latency.

Sequential ordering

Use instruction-style prompts. "deep" follows ordering literally, so name the steps:

First find Datadog's pricing page URL.
Then scrape that URL.
Then return plan names, per-host prices, and included features as JSON.

For chained URL discovery (a common LinkedIn-style two-step pattern):

First find LinkedIn posts on context engineering.
Then, for each URL, extract the LinkedIn comments.

Compound prompts that decompose the work and assign a role to the agent also work well in "deep":

Your role is to map a company's value proposition from its website.

Inputs: {company_name}, {company_website}.

- First, find and scrape the homepage and primary product pages.
- From each page, extract: headline claims, customer benefits,
  differentiator language, and CTAs.
- Then, synthesize the extracted data into a summary of the value
  proposition.
- Avoid vague marketing fluff. Focus on concrete external value claims.

In-query URL scraping (chained scrapes)

"deep" can scrape multiple URLs in a single call, including URLs discovered during earlier iterations. This is essential for:

Detailed answers that rely on full pages rather than search snippets.
Returning a list of precise items found on a page (product characteristics, PDF links, downstream URLs).
Tasks requiring prices, images, or specifications that only appear on the source page.

Chain with Fetch

An alternative to in-query URL scraping: use Search to find candidate URLs, then call Fetch on the most relevant ones. This gives the caller direct control over which pages get scraped and how their content is processed downstream.

search = client.search(query="Datadog pricing tiers", depth="standard", output_type="searchResults")
for r in search.results[:3]:
    page = client.fetch(url=r.url, render_js=True)
    # process page yourself

Prompting best practice

"standard" and "deep" mode use agentic search and can follow instruction-style queries. "fast" ignores natural language instructions. Queries should be split between:

What the search must retrieve: agentic search will optimize searches to find those elements.
How the results should be reasoned over: for "sourcedAnswer" and "searchResults", how to use the data to answer a question.

Original phrasing	Recommended phrasing	Why
”How to estimate the annual IT costs of Total SA?"	"Find data sources that quantify Total SA’s IT spend (annual reports, tech-vendor case studies, IT services contracts mentioning Total SA). For each, extract the figure and the year.”	The first phrasing requests an answer; the second specifies retrievable evidence the agent can locate.
”Tell me about the company linkup.so"	"Find the homepage, product pages, and about page for linkup.so. Extract: what the company does, target customers, pricing model, and known investors.”	The first phrasing is unscoped; the second names targets and extraction fields.

How to construct a query

A retrieval query has four components. The same shape applies across "standard" and "deep", with longer instructions and explicit ordering on "deep".

Role

From which perspective should the agent think.Example: “You are an expert GTM consultant”

Scope

Where the agent should look.Example: “On the company domain {company_domain}, analyze homepage, about, and blog”

Method

What to extract.Example: “Include products, business model, target market, value proposition”

Format

Shape of the answer ("sourcedAnswer" or "structured").Example: “Concise, business-oriented prose”

Output type selection

Downstream consumer	Recommended setting
LLM that will reason over the sources	`"searchResults"`
End user, displayed directly	`"sourcedAnswer"`
Code that parses fields	`"structured"` (with `structuredOutputSchema`)

For "structured", see the structured output tutorial for JSON-schema mechanics.

Source filtering

The full filtering parameter list lives on the Search overview.

Use includeDomains (up to 100) and excludeDomains (unlimited) for control over sources. See the filtering tutorial for more.
Use fromDate and toDate (ISO 8601, YYYY-MM-DD) to restrict the index window. Note that some webpages (product pages, news) might have metadata publish date different from their latest update date, which makes filtering unstable.

LinkedIn data extraction

LinkedIn extraction is only available on the Search endpoint. The Fetch endpoint does not retrieve LinkedIn content.

The Search endpoint can extract structured data from LinkedIn profile and company pages, and can surface posts by keyword.

Target	Query formulation
Person or company profile	`{linkedin_url}` + `Return the profile details.`
Person or company posts	`{linkedin_url}` + `Return the recent posts.`
Person or company comments	`{linkedin_url}` + `Return the comments.`
Topic search	`Search for LinkedIn posts on {keyword}.`

{linkedin_url} is a person URL (linkedin.com/in/{slug}) or a company URL (linkedin.com/company/{slug}).

LinkedIn data extraction only works with the exact LinkedIn profile or company URL. Shortened links, search-result fragments, or partial slugs will not return reliable data.

When the URL isn’t known up front, use "deep" to find the profile and scrape it in the same call.

First find the LinkedIn profile for {person_name} at {company}.
Then scrape that URL and return the profile details.

Common pitfalls

Bad → Fix pairs grounded in observed integration failures. Each pair targets a single decision: rewrite the prompt, not the surrounding code.

Reasoning instead of retrieving
Unscoped 'tell me about' prompts
Dates in the query string
Instruction prompts on fast
Using Fetch for LinkedIn

Bad

How to estimate Total SA's annual IT spend?

Fix

Find Total SA's annual reports and IT-services contracts that mention IT
spend. For each source, extract the disclosed IT-spend figure and the year,
with the citation URL.

Bad

Tell me about the company linkup.so.

Fix

Find the linkup.so homepage, product pages, and about page. Extract: what
the company does, target customers, pricing model, and known investors.

Use fromDate / toDate instead of embedding date ranges in the query.

Bad

What AI funding rounds happened between 2025-01-01 and 2025-03-31?

Fix

List European AI seed rounds. (with fromDate=2025-01-01, toDate=2025-03-31)

"fast" is keyword-only and ignores everything beyond the literal query string.

Bad

First find Datadog's pricing page. Then scrape it. Then return plan names.
(depth: fast)

Fix

First find Datadog's pricing page. Then scrape it. Then return plan names.
(depth: deep)

Fetch does not authenticate, so LinkedIn URLs return the login-wall view an anonymous visitor would see. Route LinkedIn URLs through Search instead — see LinkedIn data extraction above.

Bad

POST /fetch
{ "url": "https://www.linkedin.com/in/{slug}", "renderJs": true }

Fix

POST /search
{
  "query": "https://www.linkedin.com/in/{slug}\nReturn the profile details.",
  "depth": "standard"
}

Resources

Search overview

Parameters, modes, output types, pricing, and the minimal first call.

Filtering tutorial

Domain and date filtering: how to combine includeDomains, excludeDomains, fromDate, and toDate.

Structured output

JSON-schema mechanics for outputType set to "structured".

Prompt Optimizer

Rewrite a draft prompt into the retrieval-shaped form the Search endpoint expects. Templates available.

Errors and rate limits: Errors · Rate limits

Get Started

Endpoints

Platform

Guides

Tutorials

FAQ

Choosing depth

Per-depth guidance

Behaviour

When to use it

When not to use it

Query shape

When to use it

Adjacent searches for breadth

In-query URL scraping (one URL per call)

When to use it

Sequential ordering

In-query URL scraping (chained scrapes)

Chain with Fetch

Prompting best practice

Output type selection

Source filtering

LinkedIn data extraction

Common pitfalls

Resources

Search overview

Filtering tutorial

Structured output

Prompt Optimizer

Get Started

Endpoints

Platform

Guides

Tutorials

FAQ

Documentation Index

​Choosing depth

​Per-depth guidance

​Behaviour

​When to use it

​When not to use it

​Query shape

​When to use it

​Adjacent searches for breadth

​In-query URL scraping (one URL per call)

​When to use it

​Sequential ordering

​In-query URL scraping (chained scrapes)

​Chain with Fetch

​Prompting best practice

​Output type selection

​Source filtering

​LinkedIn data extraction

​Common pitfalls

​Resources

Search overview

Filtering tutorial

Structured output

Prompt Optimizer

Choosing depth

Per-depth guidance

Behaviour

When to use it

When not to use it

Query shape

When to use it

Adjacent searches for breadth

In-query URL scraping (one URL per call)

When to use it

Sequential ordering

In-query URL scraping (chained scrapes)

Chain with Fetch

Prompting best practice

Output type selection

Source filtering

LinkedIn data extraction

Common pitfalls

Resources