API and Data Integration: A Practical Reference Guide

You're wiring AI visibility data into Looker on a Monday morning. The API responds, the dashboard loads, and then the numbers don't reconcile because one connector grouped citations by URL while another grouped them by domain. A webhook retries after your receiver times out, a cursor is reused incorrectly, and a nightly export arrives with timestamps interpreted in the wrong locale. The connection technically works, but the data product doesn't.

Reliable API and data integration depends on more than authentication and endpoint selection. You need explicit contracts, stable identifiers, useful payload shapes, controlled retries, semantic mapping, and a recovery path for every partial failure. The patterns below treat integration as production infrastructure, with examples for analytics, automation, warehouse ingestion, and AI visibility workflows.

How to Use This API and Data Integration Reference

Start with the operational question, not the endpoint list. If your platform team is wiring visibility data into Looker, begin with the read and export surfaces, then move to the destination recipe that matches BigQuery or your reporting stack. If you're exploring how AI systems present sources and brands, a plain-English question platform can help you frame the questions your analytics pipeline needs to answer.

This guide works as a documentation-style reference rather than a narrative tutorial. Jump to the area that matches your task:

• Integration surface: Find read APIs, write APIs, webhooks, or bulk exports.
• Data type: Locate prompts, citations, mentions, or share-of-voice fields.
• Destination stack: Use the BigQuery, Sheets, Slack, and reverse ETL patterns.
• Failure mode: Go directly to rate limits, signing errors, schema mismatches, or truncated exports.

The examples use consistent conventions. Request and response blocks show only the fields that matter for the integration decision. Timestamps use ISO-8601 in UTC, identifiers are opaque strings, and retry guidance assumes an exponential backoff window rather than immediate repeated requests. Webhook examples include the exact signing header and validation approach needed by the consumer.

Practical rule: Treat every payload as a contract. Preserve unknown additive fields, validate required fields, and keep the original event body available for replay.

API descriptions are easier to share and test when teams use machine-readable contracts. The OpenAPI ecosystem supports consistent API descriptions across teams and tooling, while UNECE guidance recommends modeling APIs with JSON Schema even when a later exchange uses XML, reinforcing schema-first design at the data-contract layer (UNECE API technical specification). Keep the quick-reference material at the end open while building. It consolidates endpoints, fields, and operational limits into a lookup you can return to during implementation.

Integration Surfaces and How They Are Organized

An AI visibility platform usually exposes four integration surfaces, and each serves a different operational job. Separating them early prevents a common mistake, using a high-frequency read API for historical warehouse loading or expecting a webhook to provide a complete audit trail.

Surface	Primary Use Case	Typical Latency	Reference Section
Read APIs	Prompt metrics, citations, mentions, and share-of-voice aggregates	Minutes for polled reads	Read Endpoints for Prompts, Citations, and Share of Voice
Write APIs	Prompt collections, annotations, alert rules, and rescans	Request-dependent	Write Endpoints, Webhooks, and Event Payloads
Signed webhooks	Rank changes, new citations, and triggered alerts	Seconds for event delivery	Write Endpoints, Webhooks, and Event Payloads
Bulk exports	CSV snapshots and warehouse ingestion	Hours for export workflows	CSV Export Patterns for Analytics Stacks

Read surfaces

Use read APIs when the consumer needs fresh per-prompt signals, filtered analysis, or a reconciliation job. A dashboard might request a date range and engine filter, while a warehouse job walks cursors until it has a complete partition.

Write surfaces

Write APIs belong in workflows that change configuration or metadata. Prompt tagging, alert registration, and rescan requests should be idempotent, authenticated with narrowly scoped credentials, and recorded with request identifiers.

Event delivery

Signed webhooks fit user-facing alerts and operational dashboards. They remove the need to ask repeatedly whether a prompt changed, but they introduce ordering, replay, signature validation, and dead-letter handling.

Bulk export

CSV snapshots are often cheaper and simpler for backfills. They're naturally suited to idempotent loads because a file can be staged, checked, and merged without making the destination depend on request timing.

For a broader vocabulary around REST, event, composite, and other API patterns, keep this API types reference guide nearby. Choose read APIs or webhooks when the workflow depends on current prompt-level movement. Choose exports when the priority is historical completeness, repeatable loading, or dashboard backfill.

Authentication, Base URLs, and Request Formatting

Use workspace-scoped bearer tokens for normal API access. Give downstream reporting consumers a secondary read-only key when they don't need to create alerts, edit prompt collections, or trigger rescans. Rotate credentials by creating the replacement first, deploying it through the secret manager, verifying traffic, and revoking the old token only after the consumer has switched.

The three base URLs are:

• Production, `
• EU region, `
• Sandbox, `

The sandbox resets weekly and rejects webhook signatures intentionally, so it's suitable for local request testing, not end-to-end signed-event validation.

Every request should include the standard headers:

• Authorization: Bearer <token>
• X-LLMrefs-Client: <application-name>/<version>
• X-Request-Id: <opaque-request-id>
• Accept: application/json
• Content-Type: application/json for requests with a body

A paginated read can look like this:

curl --request GET \
  --url '' \
  --header 'Authorization: Bearer llmrefs_token' \
  --header 'X-LLMrefs-Client: looker-sync/1.0' \
  --header 'X-Request-Id: 7c8f2b1e' \
  --header 'Accept: application/json' \
  --header 'If-Modified-Since: 2026-09-11T00:00:00Z'

If-Modified-Since keeps routine polling inexpensive when the collection hasn't changed. Don't add a trailing slash to collection endpoints, because /v1/prompts/ returns 404. Lowercase query parameters exactly, and keep cursor values opaque rather than parsing or generating them.

For mobile clients, TLS pinning should follow your organization's certificate-rotation policy. Pinning can reduce interception risk, but hardcoded pins can also take an app offline when certificates change. Use a managed trust strategy with a tested rotation path, and never embed bearer tokens in a mobile binary.

Teams comparing token patterns can also browse the auth docs, and teams handling search-driven workflows may find the Brave Search API key guide useful when separating search credentials from analytics credentials.

Read Endpoints for Prompts, Citations, and Share of Voice

Read endpoints should map cleanly to analytical grains. /v1/prompts is prompt-grain data, /v1/citations is citation-grain data, /v1/share-of-voice is aggregate metric data, and /v1/mentions captures brand or competitor appearances. Keep those grains separate in the warehouse. A citation array nested inside a prompt record is convenient for a response consumer, but it's not a substitute for a child citation table.

A prompt request might be:

GET /v1/prompts?date_from=2026-09-01&date_to=2026-09-11&engine=perplexity&limit=100

{
  "data": [
    {
      "prompt_id": "prm_opaque_01",
      "prompt_text": "Which analytics platforms track AI search visibility?",
      "engine": "perplexity",
      "run_id": "run_opaque_91",
      "observed_at": "2026-09-11T09:00:00Z"
    }
  ],
  "pagination": {
    "next_cursor": "opaque_cursor",
    "has_more": true
  }
}

Citations use filters such as prompt_id, engine, date_from, date_to, limit, and cursor:

GET /v1/citations?prompt_id=prm_opaque_01&engine=perplexity&limit=100

{
  "data": [
    {
      "citation_id": "cit_opaque_02",
      "prompt_id": "prm_opaque_01",
      "run_id": "run_opaque_91",
      "citation_url": "https://example.com/guide",
      "citation_domain": "example.com",
      "engine": "perplexity"
    }
  ],
  "pagination": {
    "next_cursor": null,
    "has_more": false
  }
}

Aggregates need an explicit grain

Share-of-voice requests should state the comparison window and engine:

GET /v1/share-of-voice?date_from=2026-09-01&date_to=2026-09-11&engine=chatgpt&limit=100

{
  "data": [
    {
      "prompt_id": "prm_opaque_01",
      "engine": "chatgpt",
      "sov_score": 0.42,
      "mentions_count": 3,
      "period_start": "2026-09-01T00:00:00Z",
      "period_end": "2026-09-11T23:59:59Z"
    }
  ],
  "pagination": {
    "next_cursor": null,
    "has_more": false
  }
}

When results exceed 1,000 rows, continue with next_cursor until has_more is false. Don't assume an empty array means failure. A narrow engine and date combination, or a prompt filter that doesn't overlap the selected period, can legitimately return no rows.

citation_url values aren't deduplicated across engines. Group by citation_domain downstream when the business question concerns source visibility, and retain the raw URL when the question concerns the exact cited page. This distinction prevents an engine-specific URL variation from inflating source counts.

Write Endpoints, Webhooks, and Event Payloads

Write operations change state, so they need stronger controls than reads. Use POST for creating prompt collections, alert rules, and webhook subscriptions, and PUT for replacing or updating a known resource. Send an idempotency key with alert creation so a network timeout doesn't create duplicate rules when the client retries.

A webhook subscription payload should retain the selected event types and the signing secret returned by the platform:

{
  "url": "https://analytics.example.com/hooks/llmrefs",
  "events": [
    "prompt.rank_changed",
    "citation.added",
    "alert.triggered"
  ],
  "secret": "webhook_secret",
  "version": "2026-01"
}

The receiver validates the raw request body with HMAC-SHA256, using X-LLMrefs-Signature and the timestamp header before parsing JSON. The signature must be calculated over the exact bytes received, not a reserialized object.

import hashlib
import hmac

def valid_signature(raw_body, timestamp, received_signature, secret):
    signed = f"{timestamp}.{raw_body.decode('utf-8')}".encode("utf-8")
    expected = hmac.new(
        secret.encode("utf-8"),
        signed,
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, received_signature)

Persist event_id before enqueuing work. That single design choice makes duplicate delivery harmless.

{
  "event_id": "evt_opaque_01",
  "type": "prompt.rank_changed",
  "created_at": "2026-09-11T09:00:00Z",
  "data": {
    "prompt_id": "prm_opaque_01",
    "engine": "chatgpt",
    "previous_rank": 4,
    "current_rank": 2
  }
}

citation.added carries citation-level data, while alert.triggered carries the alert identifier and the condition that fired. Keep the envelope stable and treat fields inside data as event-specific.

Delivery retries follow 1 second, 5 seconds, 30 seconds, 5 minutes, 30 minutes, and 2 hours, after which the event moves to /v1/webhooks/dlq. The consumer should return a success response only after durable persistence. Queue downstream enrichment separately so a slow warehouse doesn't make the webhook receiver appear unhealthy.

Polling Versus Webhooks for the Same Workflow

Suppose a team tracks share-of-voice movement across 50 prompts and 6 engines, refreshing hourly. That produces 300 calls per hour and 216,000 calls per month, using the stated workload assumptions. The math is simple, but the operational cost isn't. Cursor management, repeated response parsing, rate-limit bursts, and reconciliation logic all sit inside the polling service.

Identical cursors may produce cache hits, but a cache hit still leaves the client responsible for freshness decisions and error handling. Refreshing at minute granularity increases the chance of 429 responses, especially when multiple workers start together.

The webhook design subscribes to prompt.rank_changed and updates only affected prompt-engine pairs. The same workflow can reduce monthly requests to under 5,000, but that figure is a workload outcome rather than a universal platform limit. The trade-off is real: the consumer must handle event ordering, duplicate delivery, retries, dead-letter replay, and gaps caused by receiver downtime.

Use this rubric:

• Polling: Backfills, scheduled reconciliation, audit trails, and recovery after an event gap.
• Webhooks: Alerts, live dashboards, Slack notifications, and user-facing status changes.
• Hybrid: Webhooks for current state, periodic reads for verification and historical completeness.

The Slack and Zapier integration guide is useful when the final action is a notification rather than a warehouse write. The key is to keep the webhook handler thin, durable, and replayable.

CSV Export Patterns for Analytics Stacks

CSV exports work well when the destination values completeness over immediacy. Keep four artifacts separate: the prompts export, citations export, daily share-of-voice export, and long-format event export. The prompts file is generally wide and prompt-oriented. The event file is long and event-oriented, which makes it better for incremental analysis.

Preserve the source column order and header names. Treat commas, quotes, and line breaks inside text fields as normal CSV content, not delimiters. Load with a parser that honors quoted fields rather than splitting lines manually.

Export	Format	Key Columns	Join Key
Prompts	Wide prompt-level rows	prompt_id, prompt_text, engine, run_id, observed_at	prompt_id plus run_id
Citations	Child citation rows	citation_id, prompt_id, citation_url, citation_domain, engine	prompt_id plus run_id
Daily share of voice	Aggregate rows	prompt_id, engine, sov_score, mentions_count, period fields	prompt_id plus period
Events	Long event rows	event_id, type, created_at, payload fields	event_id

A useful star schema keeps prompt-level dimensions in a prompt or prompt-run table, engine as a dimension, and citations as a child fact table. Don't concatenate multiple citations into one string. That destroys page-level joins and makes domain grouping harder.

For BigQuery, stage the file before loading:

LOAD DATA INTO `project.analytics.llmrefs_prompts`
FROM FILES (
  format = 'CSV',
  uris = ['gs://bucket/llmrefs/prompts/*.csv'],
  skip_leading_rows = 1,
  field_delimiter = ',',
  quote = '"'
);

In Google Sheets, IMPORTDATA is convenient for a filtered snapshot:

=IMPORTDATA("https://exports.example.com/llmrefs/share-of-voice.csv")

Use a named range or a stable staging tab so header refreshes don't break downstream formulas. Store timestamps as UTC strings until the transformation layer assigns a timezone. Preserve language and locale fields where available, otherwise a French result can be incorrectly merged with an English result that uses a translated prompt or different citation context.

Connector Recipes for BigQuery, Sheets, and Reverse ETL

A connector is production-ready when it has a destination contract, an identity key, and a recovery path. The following recipes keep transformation logic visible rather than hiding it inside an opaque sync.

BigQuery ingestion

1. Write prompts and citations CSV files to a Cloud Storage staging bucket.
2. Validate headers, row counts, quoting, and timestamp parsing.
3. Load into staging tables.
4. Run a scheduled MERGE into curated tables using prompt_id plus run_timestamp.
5. Partition or cluster around the query patterns used by Looker.
6. Grant the loader only the Cloud Storage object and BigQuery job permissions it needs.

The frequent production failures are duplicate files, partially uploaded objects, and schema additions that break strict loaders. Use object-level completion markers and preserve the raw file for replay.

Sheets synchronization

1. Pull only the share-of-voice slice required by the reporting audience.
2. Write into a staging tab through the Sheets API.
3. Replace the named range after the write succeeds.
4. Refresh every six hours, or at the cadence appropriate to the dashboard.
5. Apply conditional formatting to flag coverage drops above the team's chosen threshold.
6. Keep the service account limited to the target spreadsheet.

Sheets is excellent for review and lightweight collaboration, but it's a poor system of record. Concurrent edits, formula drift, and accidental column movement are more dangerous there than in a warehouse.

Reverse ETL to HubSpot

1. Build curated prompt cohorts in BigQuery.
2. Select stable cohort identifiers and the properties HubSpot should receive.
3. Use Hightouch or Census to sync those fields into HubSpot custom properties.
4. Restrict the destination token to the required CRM objects.
5. Log source row identity, sync status, and destination response.
6. Quarantine records with invalid taxonomy or stale source data.

Reverse ETL is useful when analysts need warehouse-derived visibility signals inside revenue workflows. It fails when teams push raw event volume directly into CRM records. Curate first, then activate.

For a broader analytics implementation, the enterprise SEO analytics guide provides useful context for connecting visibility metrics to reporting operations.

Rate Limits, Retries, and Governance Guardrails

A retry loop should protect both systems. Use a base delay of 500 milliseconds, full jitter, a maximum wait of 30 seconds, and no more than 7 attempts for a retryable request. The sleep before attempt n can be calculated as:

delay = random(0, min(30s, 0.5s × 2^(n-1)))

For a 10,000-request nightly batch, don't launch every request at once. Partition work, cap concurrency according to the workspace tier, honor Retry-After, and record exhausted attempts for replay. The exact tier budget must come from the workspace contract, not from an assumption embedded in application code.

Tier	Requests/min	Burst	Retry Base	Max Attempts
Workspace-defined	Contract value	Contract value	500 ms	7
Read-only consumer	Contract value	Contract value	500 ms	7
Bulk loader	Contract value	Contract value	500 ms	7

The API integration guide cited in the brief recommends retrying only retryable errors, honoring Retry-After, adding random jitter, and capping total retry time (B2B SaaS API rate-limit guidance). A separate performance guide suggests comparing production APIs against 200 to 500 ms p50, under 2 seconds p95, and published limits commonly between 60 and 1,000 requests per minute (B2B data API latency and rate-limit guide). These are evaluation targets and market observations, not substitutes for a workspace's actual quota.

Governance starts before the request leaves the service. Redact PII, version prompt taxonomies, and prevent an old prompt_id from unnoticeably representing a new campaign. Add idempotency keys to writes, validate schemas before loading, and store request_id, event_id, endpoint, status, retry count, schema version, and failure class in the warehouse. Integration complexity deserves deliberate planning, since one 2026 report found integrations took 3.2 times longer than initial engineering estimates and 68% of developers said integrations exceeded estimates (2026 API integration complexity report).

Troubleshooting Common Integration Failures

Start with the observable response, then inspect the request identity and scope.

• 401: An invalid or expired token is the usual pattern. Check the deployed secret version, send a harmless authenticated GET, and complete rotation before revoking the old credential.
• 403: A workspace transfer or project scope mismatch can leave a valid token without access. Compare token scope with the destination project rather than regenerating keys blindly.
• 422: Validate JSON types. A prompt_id sent as an integer instead of an opaque string should fail schema validation, and the fix belongs in the serializer.
• 429: Large historical backfills often create bursts. Reduce concurrency, honor Retry-After, and resume from the last durable cursor.
• Webhook 5xx or timeout: Return success only after persisting the event. Receivers that don't return a 200 response within 10 seconds can trigger redelivery, so move slow processing to a queue.

For HMAC mismatches, compare the raw body, timestamp, selected secret version, and received signature. A rolled signing secret that wasn't deployed to every consumer is a common cause. For truncated CSV responses, inspect proxy buffering and content-length behavior, discard the incomplete object, and reload from a completed staged file rather than appending uncertain rows.

Quick Reference of Endpoints, Fields, and Limits

Keep this compact map beside the implementation ticket.

Method	Path	Primary Use Case	Required Scope
GET	/v1/prompts	Prompt records and runs	Read prompts
GET	/v1/citations	Citation records	Read citations
GET	/v1/share-of-voice	Aggregate visibility metrics	Read metrics
GET	/v1/mentions	Brand and competitor mentions	Read mentions
POST	/v1/prompt-collections	Create collections	Write prompts
PUT	/v1/prompt-collections/{id}	Update collections	Write prompts
POST	/v1/alerts	Create alert rules	Write alerts
PUT	/v1/alerts/{id}	Update alert rules	Write alerts
POST	/v1/webhooks	Register event delivery	Manage webhooks
GET	/v1/webhooks/dlq	Retrieve failed events	Read webhook queue

Canonical fields include prompt_id, prompt_text, run_id, engine, citation_url, citation_domain, sov_score, mentions_count, event_id, type, and created_at. Treat identifiers and envelope fields as stable contract fields. Treat additive fields inside event payloads as forward-compatible, and ignore unknown fields without dropping the raw object.

Workspace quotas, burst envelopes, maximum CSV row counts, pagination defaults, and webhook delivery commitments should come from the documented tier contract. The implementation should still enforce the operational controls covered above: 500 ms retry base, 30-second cap, 7-attempt maximum, HMAC-SHA256 validation, and the retry sequence of 1 second, 5 seconds, 30 seconds, 5 minutes, 30 minutes, and 2 hours for webhook delivery.

Use polling for backfill and reconciliation. Subscribe to webhooks for rank changes, new citations, alerts, and other events that require prompt user-facing action.

---

LLMrefs provides AI visibility data across prompts, citations, mentions, share of voice, and the answer engines your audience uses, with API and CSV paths for analytics workflows. Visit LLMrefs to connect those signals to your warehouse, dashboards, and operational systems, then build an integration that's observable, replayable, and useful to the people making decisions.