GitLab
Use this guide to set up the GitLab source connector and find reference information.
Prerequisites
- A GitLab account on GitLab.com or a self-hosted GitLab instance using the v4 API
For Airbyte Cloud:
- OAuth (recommended) or a personal access token with the
read_apiscope
For Airbyte Open Source:
- A personal access token with the
read_apiscope
Setup guide
Step 1: Set up GitLab
Create a GitLab account or set up a self-hosted GitLab instance.
If you use a personal access token, generate one in Settings > Access Tokens with the read_api scope. The read_api scope provides read-only API access. This includes all groups, projects, the container registry and the package registry.
Step 2: Set up the connector
For Airbyte Cloud:
- Log into your Airbyte Cloud account.
- In the left navigation bar, click Sources. Then click + new source in the top-right corner.
- On the source setup page, select GitLab from the Source type dropdown. Enter a name for this connector.
- Click Authenticate your GitLab account and select OAuth or Personal Access Token for authentication.
- Log in and authorise the GitLab account.
- API URL (Optional) - Enter your self-hosted GitLab instance’s host, or use
gitlab.com(default). - Start date (Optional) - Enter the date to start replicating stream data, using the
YYYY-MM-DDT00:00:00Zformat. - Groups (Optional) - Enter a list of GitLab group paths, such as
airbytehqfor one group. - Projects (Optional) - Enter a list of GitLab project paths, such as
airbytehq/airbyte. - Number of Concurrent Threads (Optional) - Set the number of concurrent threads used for syncing. Higher values can speed up syncs but may reach rate limits. The default is 4, and the allowed range is 2 to 25. Adjust this value for your GitLab instance’s rate limits.
- Click Set up source.
For Airbyte Open Source:
- Go to the Airbyte UI and click Sources > + New source.
- Select GitLab and enter a name for the connector.
- Authenticate with a Personal Access Token.
- API URL (Optional) - Enter your self-hosted GitLab instance’s host, or use
gitlab.com(default). - Start date (Optional) - Enter the date to start replicating stream data, using the
YYYY-MM-DDT00:00:00Zformat. - Groups (Optional) - Enter a list of GitLab group paths, such as
airbytehqfor one group. - Projects (Optional) - Enter a list of GitLab project paths, such as
airbytehq/airbyte. - Number of Concurrent Threads (Optional) - Set the number of concurrent threads used for syncing. Higher values can speed up syncs but may reach rate limits. The default is 4, and the allowed range is 2 to 25. Adjust this value for your GitLab instance’s rate limits.
- Click Set up source.
Choose groups and projects
Groups and Projects control how much data you sync. Almost every stream is scoped to a group or project.
- Leave both blank to discover every group your token can see and sync their projects.
- Set Groups to sync those groups, their descendant groups and their projects.
- Set both to use Projects as a filter for projects discovered from your groups. If nothing matches, the connector syncs the project paths in Projects directly. This fallback lets you sync a project outside the listed groups.
Use full paths instead of display names or numeric IDs. Use airbytehq for a group and airbytehq/airbyte for a project.
API URL format
Enter a host in API URL, not a full API endpoint. The connector adds /api/v4/. Therefore, gitlab.company.org and https://gitlab.company.org both work. A value containing a path, such as gitlab.company.org/api/v4, fails validation. Only the http and https schemes are accepted. Airbyte Cloud rejects http.
Supported sync modes
The GitLab source connector supports these sync modes:
Supported streams
The connector outputs these streams:
- Branches
- Commits (Incremental)
- Deployments (Incremental)
- Epic Issues (GitLab Premium and Ultimate only)
- Epics (GitLab Premium and Ultimate only)
- Group Issue Boards
- Group Labels
- Group Members
- Group Milestones (Incremental)
- Groups
- Issues (Incremental)
- Jobs (Incremental; child of Pipelines, with one request per pipeline)
- Merge Request Commits (Incremental; child of Merge Requests, with one request per merge request)
- Merge Requests (Incremental)
- Pipelines (Incremental; includes child pipelines, which GitLab only returns when queried with
source=parent_pipeline) - Pipeline Trigger Jobs (Incremental; bridge jobs that trigger downstream pipelines, child of Pipelines, with one request per pipeline)
- Pipelines Extended (Incremental; child of Pipelines, with one request per pipeline)
- Project Labels
- Project Members
- Project Milestones (Incremental)
- Projects
- Releases
- Tags
- Users
Limitations
API version
The connector uses GitLab API v4. It works with GitLab.com and self-hosted GitLab instances.
Incremental sync window
Most incremental streams request data in 180-day windows. A first sync of a long-lived project can therefore issue many requests. Deployments and group and project milestones aren’t windowed. They filter on updated_at using updated_after and updated_before. They ignore Start date to read all history. merge_request_commits reads commits for merge requests returned by the incremental merge_requests query. commits filters on created_at using GitLab’s since and until parameters. If Start date is blank, other incremental streams start from 2014-01-01. For most projects, this effectively includes all history. Set a start date to shorten the initial sync.
Incremental deployments and milestones
Deployments and milestones filter on updated_at through updated_after and updated_before. They ignore Start date, so Full Refresh output remains unchanged. Deployments are requested with order_by=updated_at. Boundary records may be read again. Use Incremental | Append + Deduped to avoid duplicates.
Incremental merge request commits
The connector only reads commits for merge requests returned by the incremental merge_requests query. It uses merge_request_updated_at as the cursor. It won’t pick up commit changes that don’t update the parent merge request’s updated_at.
Incremental pipeline child streams
jobs, pipelines_extended and pipeline_trigger_jobs each make one request per pipeline. In Incremental mode, these requests only cover pipelines returned by the incremental pipelines query. These pipelines have an updated_at at or after the previous sync’s cursor. All three streams use pipeline_updated_at as their cursor. This value is the pipeline’s updated_at returned by pipelines. Keep these points in mind:
- The connector won’t pick up changes that don’t update the parent pipeline’s
updated_at. Examples include erasing a job or job artefacts expiring. Use Full Refresh for these streams if you need those changes. - When you retry a job, GitLab creates an attempt under a new job
id. Thejobsstream returns every attempt, including earlier attempts of retried jobs. A pipeline can therefore contain several rows with the samename. To keep only each job’s latest attempt, retain the highestidfor eachpipeline_idandname. Deleted pipelines and their jobs aren’t removed from the destination. - In
pipeline_trigger_jobs,downstream_pipelinefields such asstatusreflect the downstream pipeline when the upstream pipeline last changed. A trigger job withoutstrategy: dependfinishes when the downstream pipeline is created. Later downstream status changes don’t update the upstream pipeline, so the connector doesn’t read the row again. For the current downstream status, joindownstream_pipeline_idto thepipelinesstream. Alternatively, use Full Refresh forpipeline_trigger_jobs. - The cursor has one-second precision, and GitLab’s
updated_afterfilter includes the boundary. Each sync requests the latest updated regular and child pipeline in each project again. This means up to two pipelines per project. Use Incremental | Append + Deduped to avoid duplicate rows. - A project first appearing after the initial incremental sync doesn’t start from Start date. It starts from the latest
pipelinescursor, minus the previous sync’s duration. This applies to projects added to the connector configuration. It also applies to projects created in or moved into a configured group. The connector doesn’t read older pipelines in that project. Refresh these streams to backfill them. - Changing a stream’s sync mode clears its saved state. After switching from Full Refresh, the first Incremental sync reads every pipeline from Start date. With Incremental | Append, earlier Full Refresh rows remain in the destination. Use Append + Deduped or clear the stream when you switch.
- In Full Refresh mode, a failed attempt can resume without reading every pipeline again. The destination must support refreshes. You must also refresh the schema after upgrading to 4.5.0. This behaviour also covers existing Full Refresh connections that never switch to Incremental. An attempt resumes from each project’s last completed 180-day
pipelineswindow. Some pipelines are therefore read twice. Use Full Refresh | Overwrite + Deduped to avoid duplicate rows. Each new Full Refresh sync still starts from Start date. - Pipelines use page-number pagination. A pipeline updated during pagination can shift another pipeline to an earlier page. The connector might then miss that pipeline. Its child records remain skipped until the pipeline is updated again.
Child pipelines on large instances
Since version 4.4.41, pipelines reads each project twice. It reads once for regular pipelines and once with source=parent_pipeline for child pipelines. Existing connections retain their per-project cursor for regular pipelines. They automatically backfill child pipelines from Start date.
The backfill isn’t guaranteed when the stream tracks more than 10,000 partitions. This is roughly 5,000 projects because each project now has two partitions. Above this limit, the connector stores one shared cursor for the whole stream. It doesn’t read child pipelines older than that cursor. For a connection this large, reset four streams once after upgrading: pipelines, pipelines_extended, jobs and pipeline_trigger_jobs. This reset loads historical child pipelines.
Rate limits
The connector respects per-endpoint rate limits based on GitLab.com’s documented defaults. This covers Groups API, Members API, Projects API and general authenticated traffic:
| Endpoint | Rate limit |
|---|---|
GET /groups | 200 requests/min |
GET /groups/:id | 400 requests/min |
GET /groups/:id/descendant_groups | 200 requests/min |
GET /projects/:id | 400 requests/min |
GET /groups/:id/members/all | 200 requests/min |
GET /projects/:id/members/all | 200 requests/min |
| All other authenticated requests | 2,000 requests/min |
From October 19, 2026, GitLab.com also applies subscription-tier rate limits. Free namespaces allow 5,000 requests hourly and 100 requests per minute for each user. Premium and Ultimate follow in January 2027. See rate limits on GitLab.com. Syncs may pause for long periods while the connector waits for these limits to reset.
Administrators may configure different rate limits for self-hosted GitLab instances. The connector also reads the RateLimit-Remaining and RateLimit-Reset response headers. When GitLab returns HTTP 429, the connector waits for the period in Retry-After, up to about one hour. It then retries. If the header is missing, the connector retries with exponential backoff. If rate limit errors persist, create an issue.
Adjust Number of Concurrent Threads to control the connector’s number of parallel requests. Lower this value if other integrations share your API quota or you experience rate limiting.
API error handling
| Response | Behavior |
|---|---|
| 401 Unauthorized | The sync fails with Unable to refresh the access_token, please re-authenticate in Sources > Settings. Re-authenticate the source, or replace an expired or revoked personal access token. |
| 403 Forbidden | The connector skips the group, project, or resource and continues. Nothing fails, so partial data is the only symptom. |
| 404 Not Found | The sync fails with Groups and/or projects that you provide are invalid or you don't have permission to view it. Check the paths in Groups and Projects for typos and confirm your token can see them. |
| 500 Internal Server Error | Retried. |
The connector silently skips 403 responses. Missing data therefore usually indicates a permissions gap, not a bug. Check that your token can access every expected group and project.
Connection checks read the groups endpoint. A check can succeed while individual project streams return no data. This happens when the token can list groups but can’t read their projects.
IP allow list
If you use Airbyte Cloud and restrict access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.
Changelog
Expand to review
| Version | Date | Pull Request | Subject |
|---|---|---|---|
| 4.6.0 | 2026-10-16 | 89052 | Add incremental sync support to the deployments, group_milestones, project_milestones, and merge_request_commits streams |
| 4.5.7 | 2026-11-03 | 90577 | Update dependencies |
| 4.5.6 | 2026-10-27 | 89911 | Update dependencies |
| 4.5.5 | 2026-10-22 | 88981 | Normalize epics due_date_from_milestones and start_date_from_milestones from datetime strings to YYYY-MM-DD dates (typed date columns are unchanged; raw JSON values change); add work_item_id, imported and imported_from to the epics schema |
| 4.5.4 | 2026-10-20 | 89088 | Include earlier attempts of retried jobs in the jobs stream |
| 4.5.3 | 2026-10-20 | 89217 | Update dependencies |
| 4.5.2 | 2026-10-13 | 88467 | Honor GitLab Retry-After headers for HTTP 429 rate limits and raise heartbeat timeout to 90 minutes |
| 4.5.1 | 2026-10-13 | 88629 | Update dependencies |
| 4.5.0 | 2026-10-07 | 87655 | Add incremental sync support to the pipelines_extended, jobs, and pipeline_trigger_jobs streams. Failed Full Refresh attempts of these streams can now resume, which can write duplicate rows under Full Refresh | Overwrite |
| 4.4.43 | 2026-10-06 | 87865 | Update dependencies |
| 4.4.42 | 2026-09-29 | 87138 | Update dependencies |
| 4.4.41 | 2026-09-24 | 86933 | Include child pipelines in the pipelines stream (and therefore pipelines_extended and jobs) and add the pipeline_trigger_jobs stream |
| 4.4.40 | 2026-09-22 | 86599 | Update dependencies |
| 4.4.39 | 2026-09-15 | 86063 | Update dependencies |
| 4.4.38 | 2026-09-08 | 85485 | Update dependencies |
| 4.4.37 | 2026-08-18 | 84568 | Update dependencies |
| 4.4.36 | 2026-08-11 | 83941 | Update dependencies |
| 4.4.35 | 2026-07-28 | 82903 | Update dependencies |
| 4.4.34 | 2026-07-21 | 82429 | Update dependencies |
| 4.4.33 | 2026-07-14 | 81830 | Update dependencies |
| 4.4.32 | 2026-06-30 | 81041 | Update dependencies |
| 4.4.31 | 2026-06-23 | 80476 | Update dependencies |
| 4.4.30 | 2026-06-17 | 77516 | Lower default concurrency from 8 to 4; set HTTPAPIBudget to documented GitLab.com rate limits |
| 4.4.29 | 2026-06-16 | 79874 | Update dependencies |
| 4.4.28 | 2026-06-09 | 79295 | Update dependencies |
| 4.4.27 | 2026-06-02 | 78724 | Update dependencies |
| 4.4.26 | 2026-04-28 | 77265 | Update dependencies |
| 4.4.25 | 2026-04-21 | 76608 | Update dependencies |
| 4.4.24 | 2026-04-13 | 76276 | Rename “concurrent workers” to “concurrent threads” in connector spec |
| 4.4.23 | 2026-04-10 | 76210 | Promote 4.4.23-rc.5 to GA: HTTPAPIBudget with concurrency 8 |
| 4.4.23-rc.5 | 2026-04-01 | 75994 | Revert concurrency to 8 with HTTPAPIBudget for progressive rollout tuning |
| 4.4.23-rc.4 | 2026-03-31 | 75915 | Re-enable HTTPAPIBudget with concurrency 10 for rate limit protection during progressive rollout |
| 4.4.23-rc.3 | 2026-03-27 | 75537 | Increase default concurrency from 8 to 10 for progressive rollout tuning |
| 4.4.23-rc.2 | 2026-03-25 | 75480 | Comment out HTTPAPIBudget to isolate concurrency tuning during progressive rollout |
| 4.4.23-rc.1 | 2026-03-09 | 70858 | Add HTTPAPIBudget and concurrency_level for improved sync performance |
| 4.4.22 | 2026-02-24 | 73774 | Update dependencies |
| 4.4.21 | 2026-02-17 | 73401 | Update dependencies |
| 4.4.20 | 2026-02-10 | 71398 | Update dependencies |
| 4.4.19 | 2026-02-05 | 72262 | Update CDK version from 7.6.1 to 7.6.5 |
| 4.4.18 | 2025-12-18 | 70698 | Update dependencies |
| 4.4.17 | 2025-12-11 | 70839 | Change check stream from projects to groups to fix timeout for large GitLab organizations |
| 4.4.16 | 2025-11-25 | 69875 | Update dependencies |
| 4.4.15 | 2025-11-18 | 69411 | Update dependencies |
| 4.4.14 | 2025-10-29 | 69030 | Update dependencies |
| 4.4.13 | 2025-10-21 | 68297 | Update dependencies |
| 4.4.12 | 2025-10-14 | 68000 | Update dependencies |
| 4.4.11 | 2025-10-07 | 67266 | Update dependencies |
| 4.4.10 | 2025-09-30 | 66076 | Update dependencies |
| 4.4.9 | 2025-09-10 | 65997 | Update dependencies |
| 4.4.8 | 2025-08-26 | 65556 | Update dependencies |
| 4.4.7 | 2025-08-23 | 65362 | Update dependencies |
| 4.4.6 | 2025-08-09 | 64579 | Update dependencies |
| 4.4.5 | 2025-08-02 | 64222 | Update dependencies |
| 4.4.4 | 2025-07-26 | 63913 | Update dependencies |
| 4.4.3 | 2025-07-19 | 63489 | Update dependencies |
| 4.4.2 | 2025-07-18 | 63309 | Adds type property to config_normalization_rules in manifest |
| 4.4.1 | 2025-07-12 | 61471 | Update dependencies |
| 4.4.0 | 2025-07-07 | 62831 | Promoting release candidate 4.4.0-rc.1 to a main version. |
| 4.4.0-rc.1 | 2025-07-02 | 62480 | Migrate to manifest-only |
| 4.3.7 | 2025-06-02 | 57264 | Update dependencies |
| 4.3.6 | 2025-03-29 | 54945 | Update dependencies |
| 4.3.5 | 2025-02-23 | 51695 | Update dependencies |
| 4.3.4 | 2025-01-15 | 44671 | Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64 |
| 4.3.3 | 2024-08-18 | 44207 | Update dependencies |
| 4.3.2 | 2024-08-12 | 43856 | Update dependencies |
| 4.3.1 | 2024-08-04 | 43058 | Update dependencies |
| 4.3.0 | 2024-07-31 | 42920 | Migrate to CDK v4.1.0 |
| 4.2.2 | 2024-07-28 | 42601 | Update dependencies |
| 4.2.1 | 2024-07-21 | 42295 | Update dependencies |
| 4.2.0 | 2024-07-17 | 42085 | Migrate to CDK v2.4.0 |
| 4.1.0 | 2024-07-17 | 42021 | Migrate to CDK v1.8.0 |
| 4.0.8 | 2024-07-14 | 41835 | Update dependencies |
| 4.0.7 | 2024-07-11 | 41470 | Update dependencies |
| 4.0.6 | 2024-07-10 | 41100 | Update dependencies |
| 4.0.5 | 2024-07-07 | 40894 | Update dependencies |
| 4.0.4 | 2024-06-26 | 40417 | Update dependencies |
| 4.0.3 | 2024-06-23 | 40102 | Update dependencies |
| 4.0.2 | 2024-05-07 | 36637 | Schema descriptions and CDK 0.80.0 |
| 4.0.1 | 2024-04-23 | 37505 | Set error code 500 as retryable |
| 4.0.0 | 2024-04-01 | 35989 | Migrate to low-code |
| 3.0.0 | 2024-02-15 | 34548 | Fix merge_request_commits stream to return commits for each merge request |
| 2.1.2 | 2024-02-13 | 35167 | Manage dependencies with Poetry. |
| 2.1.1 | 2024-01-16 | 34203 | prepare for airbyte-lib |
| 2.1.0 | 2024-01-09 | 33676 | Add fields to Commits (extended_trailers), Groups (emails_enabled, service_access_tokens_expiration_enforced) and Projects (code_suggestions, model_registry_access_level) streams |
| 2.0.0 | 2023-10-26 | 31700 | Add correct date-time format for Deployments, Projects and Groups Members streams |
| 1.8.4 | 2023-10-19 | 31599 | Base image migration: remove Dockerfile and use the python-connector-base image |
| 1.8.3 | 2023-10-18 | 31547 | Add validation for invalid groups_list and/or projects_list |
| 1.8.2 | 2023-10-17 | 31492 | Expand list of possible error status codes when handling expired access_token |
| 1.8.1 | 2023-10-12 | 31375 | Mark start_date as optional, migrate groups and projects to array |
| 1.8.0 | 2023-10-12 | 31339 | Add undeclared fields to streams schemas, validate date/date-time format in stream schemas |
| 1.7.1 | 2023-10-10 | 31210 | Added expired access_token handling, while checking the connection |
| 1.7.0 | 2023-08-08 | 29203 | Add Deployments stream |
| 1.6.0 | 2023-06-30 | 27869 | Add shared_runners_setting field to groups |
| 1.5.1 | 2023-06-24 | 27679 | Fix formatting |
| 1.5.0 | 2023-06-15 | 27392 | Make API URL an optional parameter in spec. |
| 1.4.2 | 2023-06-15 | 27346 | Partially revert changes made in version 1.0.4, disallow http calls in cloud. |
| 1.4.1 | 2023-06-13 | 27351 | Fix OAuth token expiry date. |
| 1.4.0 | 2023-06-12 | 27234 | Skip stream slices on 403/404 errors, do not fail syncs. |
| 1.3.1 | 2023-06-08 | 27147 | Improve connectivity check for connections with no projects/groups |
| 1.3.0 | 2023-06-08 | 27150 | Update stream schemas |
| 1.2.1 | 2023-06-02 | 26947 | New field name added to Pipelines and PipelinesExtended stream schema |
| 1.2.0 | 2023-05-17 | 22293 | Preserve data in records with flattened keys |
| 1.1.1 | 2023-05-23 | 26422 | Fix error 404 Repository Not Found when syncing project with Repository feature disabled |
| 1.1.0 | 2023-05-10 | 25948 | Introduce two new fields in the Projects stream schema |
| 1.0.4 | 2023-04-20 | 21373 | Accept api_url with or without scheme |
| 1.0.3 | 2023-02-14 | 22992 | Specified date formatting in specification |
| 1.0.2 | 2023-01-27 | 22001 | Set AvailabilityStrategy for streams explicitly to None |
| 1.0.1 | 2023-01-23 | 21713 | Fix missing data issue |
| 1.0.0 | 2022-12-05 | 7506 | Add OAuth2.0 authentication option |
| 0.1.12 | 2022-12-15 | 20542 | Revert HttpAvailability changes, run on cdk 0.15.0 |
| 0.1.11 | 2022-12-14 | 20479 | Use HttpAvailabilityStrategy + add unit tests |
| 0.1.10 | 2022-12-12 | 20384 | Fetch groups along with their subgroups |
| 0.1.9 | 2022-12-11 | 20348 | Fix 403 error when syncing EpicIssues stream |
| 0.1.8 | 2022-12-02 | 20023 | Fix duplicated records issue for Projects stream |
| 0.1.7 | 2022-12-01 | 19986 | Fix GroupMilestones stream schema |
| 0.1.6 | 2022-06-23 | 13252 | Add GroupIssueBoards stream |
| 0.1.5 | 2022-05-02 | 11907 | Fix null projects param and container_expiration_policy |
| 0.1.4 | 2022-03-23 | 11140 | Ingest All Accessible Groups if not Specified in Config |
| 0.1.3 | 2021-12-21 | 8991 | Update connector fields title/description |
| 0.1.2 | 2021-10-18 | 7108 | Allow all domains to be used as api_url |
| 0.1.1 | 2021-10-12 | 6932 | Fix pattern field in spec file, remove unused fields from config files, use cache from CDK |
| 0.1.0 | 2021-07-06 | 4174 | Initial Release |