Advanced Analytica Advanced Analytica: IBOM
← All connectors
Google Search Console logo

Source connector

Google Search Console

Set up Google Search Console as a source and move its data into your chosen destination.

Version 2.1.21 certified generally available

Google Search Console

Use this page to set up the Google Search Console source connector and find reference information.

Prerequisites

  • Google Account
  • One or more verified Google Search Console properties. You need each property’s exact identifier. URL-prefix properties look like https://example.com/. Domain properties look like sc-domain:example.com. See Google’s guide to property types.
  • Google Search Console API enabled for your project (Airbyte Open Source only)

Set up the connector

Step 1: Set up Google Search Console

Choose one of the following methods to authenticate the Google Search Console connector:

You can authenticate with your Google Account using OAuth. You must own the Google Search Console property or have permission to view it. Follow Google’s instructions to confirm that your account has the required Owner or Full User permissions. This option is recommended for Airbyte Cloud users because it simplifies setup. It also lets you authenticate the connection directly from the Airbyte UI.

To authenticate with OAuth in Airbyte Open Source, create an authentication app and obtain these credentials and tokens:

  • Client ID
  • Client Secret
  • Refresh Token
  • Access Token

See in Google’s documentation for instructions on creating an OAuth app and obtaining these credentials for Google APIs.

You can authenticate using a JSON key file linked to a Google service account. This option is recommended for Airbyte Open Source users. Follow these steps to create a service account and generate the JSON key file:

  1. Open the Service Accounts page.
  2. Select an existing project or create a new one.
  3. At the top of the page, click + Create service account.
  4. Enter a name and description for the service account, then click Create and Continue.
  5. Under Service account permissions, select the roles to grant the service account, then click Continue. The Viewer role is recommended.
    • Optional: Under Grant users access to this service account, specify which users or groups can use and manage the service account.
  6. Go to the API Console/Credentials and click the email address of your new service account.
  7. In the Keys tab, click + Add key, then click Create new key.
  8. Select JSON as the Key type. This generates and downloads the JSON key file you’ll use for authentication. Click Continue.

:::caution This file is your only copy of the JSON service key. You can’t download it again, so store it securely. :::

:::note You can return to the API Console/Credentials to manage your service account or generate more JSON keys. For details about service account credentials, see Google’s IAM documentation. :::

Delegate domain-wide authority

Domain-wide delegation lets service accounts access users’ data across a Google Workspace environment through ‘impersonation’. Certain use cases require this authority. One example is a service account needing broad access across several users and services within a domain.

:::note Only your Google Workspace domain’s super admin can enable domain-wide delegation of authority for a service account. :::

Follow the steps in Google documentation to enable delegated domain-wide authority. Grant the service account this OAuth scope:

  • https://www.googleapis.com/auth/webmasters.readonly

For more information, see this Google article.

Step 2: Set up the connector in Airbyte

Airbyte Cloud

  1. Log into your Airbyte Cloud account.
  2. Click Sources, then click + New source.
  3. On the Set up the source page, select Google Search Console from the Source type dropdown.
  4. Enter a name for the Google Search Console connector.
  5. For Website URL Property, enter each Search Console property exactly. Use sc-domain:example.com for Domain properties. For URL-prefix properties, enter the full URL, including its protocol and trailing slash, such as https://www.example.com/. You can add multiple properties.
  6. For Start Date, the default is 2021-01-01. Use the provided datepicker or enter a date in the YYYY-MM-DD format. The connector replicates data created on or after this date.
  7. Authenticate the connection:

For Airbyte Open Source:

  1. Go to the Airbyte Open Source dashboard.
  2. Click Sources, then click + New source.
  3. On the Set up the source page, select Google Search Console from the Source type dropdown.
  4. Enter a name for the Google Search Console connector.
  • For Airbyte Cloud:
    • Select Oauth from the Authentication dropdown. Then click Sign in with Google to authorise your account.
  • For Airbyte Open Source:
    • (Recommended) Select Service Account Key Authorization from the Authentication dropdown, then enter the Admin Email and Service Account JSON Key. For the key, paste the JSON key you obtained during service account setup. It should begin with {"type": "service_account", "project_id": YOUR_PROJECT_ID, "private_key_id": YOUR_PRIVATE_KEY, ...}
    • Select Oauth from the Authentication dropdown. Then enter your Client ID, Client Secret, Access Token and Refresh Token.
  1. (Optional) For End Date, enter a date in the YYYY-MM-DD format. The connector replicates data created between the defined Start Date and End Date. Leave this blank to replicate everything from the Start Date to the present.
  2. (Optional) For Custom Reports, provide an array of JSON objects representing custom reports for querying the API. See the Custom reports section below to learn how to create these reports.
  3. (Optional) For Data Freshness, choose whether to include “fresh” data that Google hasn’t finalised and might change. For Incremental sync mode, leave the default value of final. See the Data Freshness section below for details about this parameter.
  4. (Optional) For Number of Concurrent Threads, set the number of worker threads used during sync. The default is 40. Consider reducing this value when syncing many site URLs or encountering rate limit errors.
  5. (Optional) Enable Always Use Aggregation Type Auto if your search analytics streams fail with a 400 error about an unsupported aggregationType. This setting replaces the default aggregation type with auto. It resolves the error for certain site configurations.
  6. (Optional) For Search Analytics API Requests Per Minute, set the maximum requests per minute for Search Analytics API calls. The default is 1200, matching Google’s documented maximum quota. Lower this value if you encounter rate limit errors. Most new Google Cloud projects start with a quota of 60 requests per minute. Check your Google Cloud Console quotas for your actual limit. See the Rate Limiting section below for more information.
  7. Click Set up source and wait for the tests to finish.

Supported sync modes

The Google Search Console source connector supports these sync modes:

:::note The cursor’s granularity is one day. Incremental Sync in Append mode might therefore duplicate data. :::

Supported streams

Entity-relationship diagram (ERD)

Connector configuration

Custom reports

Custom reports let you query the API and group results using your chosen dimensions. Results follow the order in which you provide these dimensions. Create each custom report as follows:

  1. Click Add under the Custom Reports section.
  2. Enter the report’s Name. This becomes the stream name.
  3. Select one or more Dimensions from the available dropdown list.

The available Dimensions are:

  • country
  • date
  • device
  • page
  • query

For example, use this custom report to group API results first by country and then by date:

  • Name: country_date
  • Dimensions: [“country”, “date”]

For technical reasons, date is the default dimension and always appears in your query. Specify it to change the grouping order. The primary key includes your custom dimensions, the default dimension, site_url and search_type.

The UI Custom report builder transforms your information into a custom stream using its Name.

Use the Google APIS Explorer to build and test your reports.

Data freshness

The Data Freshness parameter controls the freshness, or finality, of the queried data.

  • final: The query includes only finalised, stable data. Google has processed and verified this data, so it’s unlikely to change. This option returns the definitive statistics and information Google has analysed and confirmed.
  • all: The query returns finalised data and what Google calls “fresh” data. Fresh data is more recent but hasn’t completed the full processing and verification applied to finalised data. This option provides more current insights, but the data might change as Google continues processing and analysing it.

:::caution For Incremental Sync mode, leave this parameter at its default value of final. The all option might create discrepancies between your destination table and finalised Google Search Console data. :::

Data type map

Integration TypeAirbyte Type
stringstring
numbernumber
arrayarray
objectobject

Limitations and troubleshooting

Expand to see details about Google Search Console connector limitations and troubleshooting.

Connector limitations

Rate limits

The Search Console API applies two distinct quota types to Search Analytics queries. The connector retries automatically after exceeding either quota, using different backoff periods. See Google’s Usage Limits documentation for full details.

QPS (queries per minute) quota: Limits API requests per minute. Google documents a maximum of 1,200 requests per minute for each user and site. However, most new Google Cloud projects start at 60 requests per minute. When you exceed this quota, the connector retries after a 60-second backoff.

Load quota: Limits the internal resources that queries consume. Complex queries consume more load, especially when grouping or filtering by both page and query dimensions. Queries covering long date ranges also consume more load. Google measures load quota across 10-minute short-term and daily long-term windows. When exceeded, the connector retries after a 15-minute backoff. This matches Google’s recommendation to wait 15 minutes before retrying.

To check your current QPS quota limits:

  1. Go to your Google Cloud Console.
  2. Go to APIs & Services, then Quotas.
  3. Search for “Search Console API”.
  4. Find “Requests per minute per user” to view your current limit.

For higher QPS limits, enable billing on your Google Cloud project or request a quota increase through Google Cloud Console. Then set the connector’s Search Analytics API Requests Per Minute to match your actual quota.

If you reach load quota limits, simplify your queries. Don’t group or filter by page and query together. Use shorter date ranges and avoid querying the same data repeatedly.

Data retention

Google Search Console retains website data for only the previous 16 months. You can’t access data from before this period. For more information, see Google’s documentation on data freshness and availability.

Retired search appearance values

The search_analytics_keyword_page_report, search_analytics_keyword_site_report_by_page and search_analytics_keyword_site_report_by_site streams separate their data by search_appearance. Google retires search appearance types over time. FAQ rich results stopped appearing in Search in May 2026. Google’s FAQ structured data documentation announced the removal of FAQ rich result data from the Search Console API in August 2026.

The connector discovers appearance values from the API during every sync instead of hard-coding them. A retired value therefore requires no configuration change. Rows for that value stop arriving, while other appearance values continue syncing normally. No stream or field is added or removed.

Plan for these two effects:

  • search_appearance forms part of these streams’ primary key, and syncs don’t delete rows. Records already synced for a retired appearance value remain in your destination indefinitely. When you aggregate across search_appearance, these rows continue contributing to totals.
  • Re-syncing a historical window might no longer reproduce rows for a retired value. A full refresh can therefore return fewer records than the original sync.

Troubleshooting

Site URL errors

The connection test reads the sites stream for every configured property. An incorrect Website URL Property value therefore fails during setup, rather than during a sync. The sites and sitemaps streams report these errors as configuration errors:

  • Configured site URL is not a verified Search Console property in this account. The value doesn’t match a property in your authenticated Search Console account. Compare it with the property list in Search Console. Common errors include a missing trailing slash or using http instead of https. Other errors include a missing or extra www.. You might also have entered a plain domain instead of the sc-domain: form for a Domain property.
  • Configured site URL is not accessible with the account's Search Console permissions. The property exists, but the authenticated account can’t read it. Ask a property owner to add the Google account for OAuth authentication. For service account authentication, ask them to add the service account email as a user on that property.

Other issues

  • Find common Google Search Console source connector issues on the Airbyte Forum.

IP allow list

If you use Airbyte Cloud and restrict access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review
VersionDatePull RequestSubject
2.1.212026-11-1089240Update dependencies
2.1.202026-11-0591063Clarify Website URL Property formats for Domain and URL-prefix properties
2.1.192026-11-0591068Emit dimension values exactly as Google returns them, including quoted and numeric-looking queries
2.1.182026-10-2289665Fix 400 errors on the search_analytics_site_report_by_site stream by using aggregationType auto for Google News and Discover search types; keep numeric-looking search queries as strings in keyword streams
2.1.172026-10-1388645Update dependencies
2.1.162026-10-0687882Update dependencies
2.1.152026-09-2987207Update dependencies
2.1.142026-09-2286665Update dependencies
2.1.132026-09-1586094Update dependencies
2.1.122026-09-0975289Improve error messages for unverified or inaccessible site URLs on the sites and sitemaps streams
2.1.112026-09-0885529Update dependencies
2.1.102026-08-1884615Update dependencies
2.1.92026-08-1183959Update dependencies
2.1.82026-07-2883194Update to CDK 7.23.8 (fixes AirbyteCustomCodeNotPermittedError for bundled custom components) and remove the temporary Cloud version override
2.1.72026-07-281082Roll Cloud back to 2.1.5 — 2.1.6 is built on SDM 7.23.7, which breaks bundled custom components
2.1.62026-07-2882942Update dependencies
2.1.52026-07-2182440Update dependencies
2.1.42026-07-1481851Update dependencies
2.1.32026-06-3081094Update dependencies
2.1.22026-06-2380489Update dependencies
2.1.12026-06-1679906Update dependencies
2.1.02026-06-1279146Handle 403 “Search Analytics load quota exceeded” as rate-limited with a 15-minute backoff instead of config error
2.0.42026-06-0979344Update dependencies
2.0.32026-06-0178545Adds jitter to backoff strategy
2.0.22026-05-2178289Reduce search_analytics_* paginator page_size from 25000 to 5000 to lower peak worker memory on high-volume tenants. See airbytehq/oncall#12246 for context.
2.0.12026-05-1878157Restore default_concurrency to num_workers-driven configuration ({{ config.get('num_workers', 3) }}) and keep max_concurrency at 100; cancels the Phase 1 concurrency tuning rollout introduced in 2.0.1-rc.1
2.0.02026-05-0476306Append search_appearance to existing primary keys of keyword report streams; drop duplicate unfiltered keyword rows caused by empty/null searchAppearance partitions; fix multi-site duplication by removing the cartesian-product over site_urls on keyword streams; wrap dimensionFilterGroups.filters in an array to match the Google Search Console API spec. This is a breaking change for the search_analytics_keyword_page_report, search_analytics_keyword_site_report_by_page, and search_analytics_keyword_site_report_by_site streams. See the migration guide for details.
1.10.332026-04-2877276Update dependencies
1.10.322026-04-2176624Update dependencies
1.10.312026-04-1476190Add access_token to extract_output and complete_oauth_output_specification to fix OAuth secretId 422 regression
1.10.302026-04-1376276Rename “concurrent workers” to “concurrent threads” in connector spec
1.10.292026-04-0175582Add oauth_connector_input_specification with granular scopes
1.10.282026-03-3175699Update dependencies
1.10.272026-03-3075426Guard numeric metric fields against complex types to prevent serialization failures and deadlocks
1.10.262026-03-2474935Update dependencies
1.10.252026-03-1074685Update dependencies
1.10.242026-02-2473935Update dependencies
1.10.232026-02-1773372Update dependencies
1.10.222026-02-1072683Update dependencies
1.10.212026-01-2071877Update dependencies
1.10.202026-01-1471722Update dependencies
1.10.192025-12-1870517Update dependencies
1.10.182025-12-0970444Add user-configurable API rate limit option for Search Analytics streams
1.10.172025-11-2570059Update dependencies
1.10.162025-11-1869392Update dependencies
1.10.152025-10-2968771Update dependencies
1.10.142025-10-2168231Update dependencies
1.10.132025-10-1468004Update dependencies
1.10.122025-10-0767263Update dependencies
1.10.112025-09-3065384Update dependencies
1.10.102025-09-1666010Update to CDK v7
1.10.92025-08-0964584Update dependencies
1.10.82025-08-0264187Update dependencies
1.10.72025-07-2663877Update dependencies
1.10.62025-07-2363720Ignore 403 “insufficient permission” errors
1.10.52025-07-1963471Update dependencies
1.10.42025-07-1563309Adds type property to config_normalization_rules in manifest
1.10.32025-07-1263094Update dependencies
1.10.22025-07-0562613Update dependencies
1.10.12025-06-2862194Update dependencies
1.10.02025-06-2361425Migrate to manifest-only
1.9.42025-06-2161789Update dependencies
1.9.32025-06-1861706Fix record serialization bug
1.9.22025-06-1460663Update dependencies
1.9.12025-06-1061514Promoting release candidate 1.9.1-rc.1 to a main version.
1.9.1-rc.12025-06-1061508Add API budget, reduce concurrency levels, and catch 403 rate limiting errors
1.9.02025-06-0661408Migrate custom reports streams to low-code
1.8.02025-06-0661335Migrate SearchAnalyticsAllFields, SearchAnalyticsByDate, SearchAnalyticsByDevice, SearchAnalyticsByPage, and SearchAnalyticsByQuery to low-code
1.7.02025-06-0561332Migrate search_by_keyword streams to low-code
1.6.02025-06-0461380Promoting release candidate 1.6.0-rc.1 to a main version.
1.6.0-rc.12025-06-0260928Migrate search_analytics_by_country, sites, and sitemaps to low-code
1.5.182025-05-2858121Prevent KeyError raise on check connetion when account does not have access to any site. Better Error message
1.5.172025-05-1054946Update dependencies
1.5.162025-05-0959737Promoting release candidate 1.5.16-rc.1 to a main version.
1.5.16-rc.12025-05-0259140Migrate to CDK V6
1.5.152025-02-2254426Update dependencies
1.5.142025-02-1553781Update dependencies
1.5.132025-02-0152854Update dependencies
1.5.122025-01-2552303Update dependencies
1.5.112025-01-1851629Update dependencies
1.5.102025-01-1151129Update dependencies
1.5.92025-01-0450919Update dependencies
1.5.82024-12-2850579Update dependencies
1.5.72024-12-2150040Update dependencies
1.5.62024-12-1449173Update dependencies
1.5.52024-11-2543730Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
1.5.42024-09-0645196Fix request body for report streams by keyword
1.5.32024-08-0343067Update dependencies
1.5.22024-07-2742786Update dependencies
1.5.12024-07-2042142Update dependencies
1.5.02024-07-1742073Migrate to CDK v1.8.0
1.4.132024-07-1341734Update dependencies
1.4.122024-07-1041440Update dependencies
1.4.112024-07-0941164Update dependencies
1.4.102024-07-0640981Update dependencies
1.4.92024-06-2740215Replaced deprecated AirbyteLogger with logging.Logger
1.4.82024-06-2640532Update dependencies
1.4.72024-06-2540312Update dependencies
1.4.62024-06-2240077Update dependencies
1.4.52024-06-1739516Update state handling for incremental streams
1.4.42024-06-0439059[autopull] Upgrade base image to v1.2.1
1.4.32024-05-2438649Update deprecated auth package
1.4.22024-04-1936639Updating to 0.80.0 CDK
1.4.12024-04-1236639Schema descriptions
1.4.02024-03-1936267Pin airbyte-cdk version to ^0
1.3.72024-02-1235163Manage dependencies with Poetry
1.3.62023-10-2631863Base image migration: remove Dockerfile and use the python-connector-base image
1.3.52023-09-2830822Fix primary key for custom reports
1.3.42023-09-2730785Do not migrate config for the newly created connections
1.3.32023-08-2929941Added primary key to each stream, added custom_report config migration
1.3.22023-08-2529829Make Start Date a non-required, added the suggested streams, corrected public docs
1.3.12023-08-2429329Update tooltip descriptions
1.3.02023-08-2429750Add new Keyword-Site-Report-By-Site stream
1.2.22023-08-2329741Handle HTTP-401, HTTP-403 errors
1.2.12023-07-0427952Removed deprecated searchType, added discover(Discover results) and googleNews(Results from news.google.com, etc.) types
1.2.02023-06-2927831Add new streams
1.1.02023-06-2627738License Update: Elv2
1.0.22023-06-1327307Fix data_state config typo
1.0.12023-05-3026746Remove authSpecification from connector spec in favour of advancedAuth
1.0.02023-05-2426452Add data_state parameter to specification
0.1.222023-03-2022295Update specification examples
0.1.212023-02-1422984Specified date formatting in specification
0.1.202023-02-0222334Turn on default HttpAvailabilityStrategy
0.1.192023-01-2722007Set AvailabilityStrategy for streams explicitly to None
0.1.182022-10-2718568Improved config validation: custom_reports.dimension
0.1.172022-10-0817751Improved config validation: start_date, end_date, site_urls
0.1.162022-09-2817304Migrate to per-stream state.
0.1.152022-09-1616819Check available site urls to avoid 403 error on sync
0.1.142022-09-0816433Add custom analytics stream.
0.1.132022-07-2114924Remove additionalProperties field from specs
0.1.122022-05-0412482Update input configuration copy
0.1.112022-01-059186Fix incremental sync: keep all urls in state object
0.1.102021-12-239073Add slicing by date range
0.1.92021-12-229047Add ‘order’ to spec.json props
0.1.82021-12-218248Enable Sentry for performance and errors tracking
0.1.72021-11-267431Add default end_date param value
0.1.62021-09-276460Update OAuth Spec File
0.1.42021-09-236394Update Doc link Spec File
0.1.32021-09-236405Correct Spec File
0.1.22021-09-176222Correct Spec File
0.1.12021-09-226315Verify access to all sites when performing connection check
0.1.02021-09-035350Initial Release
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.