Postgres
Airbyte’s certified Postgres connector includes these features:
- Replicate data from tables, views and materialized views. Other objects, such as indexes and permissions, aren’t replicated to the destination.
- Keep your data fresh using multiple methods, including Change Data Capture (CDC) and replication with the xmin system column.
- Use all available sync modes, giving you flexibility in how data reaches your destination.
- Replicate tables reliably at any size using checkpointing and chunked database reads.
The sections below cover quick start, advanced setup and reference information, including data type mapping and changelogs. See here to troubleshoot Postgres connector issues.
Please note the required minimum platform version is v0.58.0 for this connector.
Quick start
Complete these minimum steps to configure a Postgres connector:
- Create a dedicated, read-only Postgres user with permission to replicate data.
- Create a Postgres source in the Airbyte UI using the
xminsystem column. - (Airbyte Cloud Only) Allow inbound traffic from Airbyte IPs.
After completing these steps, you can select Postgres as a source for data replication.
Step 1: Create a read-only Postgres user
These steps create a dedicated, read-only user for data replication. Alternatively, you can use an existing Postgres database user.
Run these commands to create a user:
CREATE USER <user_name> PASSWORD 'your_password_here';
Now give this user read-only access to the relevant schemas and tables. Repeat this command for each schema you plan to replicate:
GRANT USAGE ON SCHEMA <schema_name> TO <user_name>;
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO <user_name>;
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO <user_name>;
Step 2: Create a Postgres source
In your Airbyte Cloud or Airbyte Open Source account, select Sources from the left navigation bar. Search for Postgres, then create a Postgres source.
Enter the required information:
- Enter your Postgres database’s hostname, port number and name. On Airbyte Cloud, the host must be reachable from the internet. A database on
localhost,127.0.0.1or a private network is reachable only through an SSH tunnel. This also provides the encryption that Airbyte Cloud requires. In Host, enter only the hostname or IP address, not a URL orhost:port.
Step 3: (Airbyte Cloud Only) Allow inbound traffic
If you use Airbyte Cloud, configure your database to allow inbound traffic from Airbyte IPs. The Airbyte Security docs lists every IP you need to allow.
Airbyte Cloud never connects to a database without encryption. Use TLS/SSL with require, verify-ca or verify-full, or connect through an SSH tunnel. The tunnel encrypts the connection between Airbyte Cloud and your network. On Airbyte Cloud, the unencrypted modes disable, allow and prefer require an SSH tunnel. Airbyte Cloud can’t directly reach a database on localhost or a private network. Connect through an SSH tunnel. Before connecting, Airbyte Cloud checks the Host field. localhost, 127.0.0.1 or 0.0.0.0 with No Tunnel fails the connection test with Airbyte Cloud cannot reach a database on your own machine. Use a hostname reachable from the internet, or configure an SSH tunnel.. A URL, credentials or host:port in the field fails with Enter only the hostname in Host; put the port in the Port field..
Now select Set up source in the Airbyte UI. Airbyte tests the connection to your database. When the test succeeds, your Airbyte Postgres source is configured.
If the test fails with Airbyte could not reach the database at the configured host and port, Airbyte couldn’t open a TCP connection to your database. Check the host and port. Confirm that Postgres accepts remote connections through listen_addresses in postgresql.conf and a matching pg_hba.conf entry. Confirm that your firewall or security group allows the Airbyte IP addresses listed above.
Advanced CDC configuration
Airbyte uses logical replication from the Postgres write-ahead log (WAL) to capture deletes incrementally through a replication plugin:
- See here to learn how Airbyte implements CDC.
- See here for Postgres CDC requirements and limitations.
Configure your Postgres source with CDC when:
- You need a record of deletions.
- Your database is very large (500 GB or more).
- Your table has a primary key but no suitable cursor field for incremental syncing (
updated_at).
After completing the quick start, follow these additional steps to configure your Postgres source with CDC:
- Give the read-only user additional
REPLICATIONpermissions. - Enable logical replication on your Postgres database.
- Create a replication slot on your Postgres database.
- Create publication and replication identities for each Postgres table.
- Enable CDC replication in the Airbyte UI.
Step 1: Prepopulate the source configuration
Follow the quick start steps to confirm that Airbyte can connect to your Postgres database before configuring CDC settings.
CDC requires PostgreSQL 14 or later. Debezium 3.4 is the CDC engine in connector version 3.8.10 and later. It no longer supports PostgreSQL 13 or earlier.
For CDC, you can connect to primary/master databases or replicas. Using a replica requires Postgres 16.1 or later and connector version 3.6.21 or later. You must also enable additional configurations on the database instance. See the Postgres official documentation for help.
Step 2: Add read-only user permissions
To configure CDC for the Postgres source connector, grant the user created in step 1 of the quick start these REPLICATION permissions:
ALTER USER <user_name> REPLICATION;
Step 3: Enable logical replication
Enable logical replication by following the steps for your deployment environment.
Bare metal, VMs and Docker
For bare metal, VMs (EC2/GCE/etc) or Docker, configure these parameters in your Postgres database’s postgresql.conf file.
| Parameter | Description | Set value to |
|---|---|---|
| wal_level | Type of coding used within the Postgres write-ahead log | logical |
| max_wal_senders | The maximum number of processes used for handling WAL changes | min: 1 |
| max_replication_slots | The maximum number of replication slots that are allowed to stream WAL changes | 1 (if Airbyte is the only service reading subscribing to WAL changes. More than 1 if other services are also reading from the WAL) |
AWS Postgres RDS or Aurora
- Open the Configuration tab for your DB cluster.
- Find your cluster parameter group. Edit its parameters or create a copy to edit. If you create a copy, change your cluster’s parameter group before restarting.
- On the parameter group page, search for
rds.logical_replication. Select the row, then click Edit parameters. Set the value to1. - Wait for the maintenance window to restart the instance automatically, or restart it manually.
:::note
AWS Aurora implements a CDC caching layer that’s incompatible with Airbyte’s CDC implementation. To use Airbyte with AWS Aurora, disable the CDC caching layer. Set the rds.logical_wal_cache parameter to 0 in the AWS Aurora parameter group.
:::
Azure Database for Postgres
In the Azure Portal, use your PostgreSQL instance’s replication menu to change the Postgres DB replication mode to logical. Alternatively, run this command with the Azure CLI:
az postgres server configuration set --resource-group group --server-name server --name azure.replication_support --value logical
az postgres server restart --resource-group group --name server
Step 4: Create a replication slot on your Postgres database
Airbyte requires the pgoutput plugin for this step. Run the following command as the user with the new REPLICATION role. It creates a replication slot named airbyte_slot using pgoutput:
SELECT pg_create_logical_replication_slot('airbyte_slot', 'pgoutput');
The command output includes the replication slot name. Enter this name on the Airbyte source setup page.
Step 5: Create publications and identities
- Add a replication identity, which distinguishes between rows, for each table you want to replicate:
ALTER TABLE tbl1 REPLICA IDENTITY DEFAULT;
In rare cases, consider replica identity type full if your tables support TOAST or contain very large field values: ALTER TABLE tbl1 REPLICA IDENTITY FULL;. Ensure TOAST-able tables use non-TOAST-able primary keys, such as integers or varchars. There will only be a modest increase in resource utilization, in addition to increased WAL storage size.
- Create the Postgres publication. Include every table you want to replicate in the publication:
CREATE PUBLICATION airbyte_publication FOR TABLE <tbl1, tbl2, tbl3>;
You can customise the publication name. See the Postgres docs if you need to add or remove publication tables later.
:::note The Airbyte UI currently lets you select any tables for CDC. A selected table won’t replicate if it isn’t part of the publication. A published table without a replication identity gets one automatically during the first run. This requires the Airbyte user to have the necessary permissions. :::
Step 6: Enable CDC replication
In your Postgres source, change the update method to Read Changes using Change Data Capture (CDC). Enter the replication slot and publication you created.
:::note
If max_slot_wal_keep_size is exceeded, PostgreSQL can invalidate the replication slot. The slot then has wal_status = lost and a null restart_lsn. The connector fails the sync and can’t recover automatically. Drop and recreate the replication slot, then reset the connection for a full re-sync.
:::
Postgres replication methods
The Postgres source offers three methods for replicating updates: CDC, xmin and standard with a user-defined cursor. CDC and xmin are the most reliable methods for updating your data.
CDC
Airbyte uses logical replication from the Postgres write-ahead log (WAL) to capture deletes incrementally through a replication plugin. See Change Data Capture (CDC) to learn how Airbyte implements CDC. Configure your Postgres source with CDC when:
- You need a record of deletions.
- Your database is very large (500 GB or more).
- Your table has a primary key but no suitable cursor field for incremental syncing (
updated_at).
If you need a destination snapshot but CDC limitations prevent its use, consider the xmin replication method.
Xmin
Xmin is the new cursor-less replication method for Postgres. Cursorless syncs replicate new or updated rows without requiring you to choose a cursor field. The xmin system column, available in every Postgres database, tracks inserts and updates to your source data.
Xmin is a good fit when:
- You don’t have a well-defined cursor candidate for Standard incremental mode.
- You want to replace a previously configured full-refresh sync.
- Your database doesn’t have heavy writes that could cause transaction ID wraparound.
- You aren’t replicating non-materialized views. Xmin replication doesn’t support non-materialized views.
Connect with SSL or SSH tunnelling
SSL modes
These SSL connection modes are available:
disabledisables encrypted communication between Airbyte and the source.allowenables encrypted communication only when the source requires it.preferallows unencrypted communication only when the source doesn’t support encryption.requirealways requires encryption. The connection fails if the source doesn’t support encryption.verify-caalways requires encryption and verifies that the source has a valid SSL certificate.verify-fullalways requires encryption and verifies the source’s identity.
SSH tunnelling
Airbyte Cloud requires encrypted communication, which an SSH tunnel provides. SSH encrypts the connection between Airbyte Cloud and your bastion host. On Airbyte Cloud, the unencrypted modes disable, allow and prefer require SSH Key Authentication or Password Authentication. With No Tunnel, the connection fails. An SSH tunnel also lets Airbyte Cloud reach databases that aren’t exposed to the internet. This includes databases on localhost or a private network. Airbyte connects to a bastion host exposed to the Airbyte IPs. The bastion then forwards the connection to the database.
For SSH Tunnel Method, select:
No Tunnelto connect directly to the database.SSH Key Authenticationto establish the SSH tunnel using an RSA private key as your secret.Password Authenticationto establish the SSH tunnel using a password as your secret.
Connect through an SSH tunnel
You can connect to a Postgres instance through an SSH tunnel.
An SSH tunnel configures Airbyte to connect through an intermediate server with direct database access. This server is also called a bastion or jump server. Airbyte connects to the bastion, which then connects directly to the database server.
To connect to a Postgres instance through an SSH tunnel:
- While setting up the Postgres source connector, select an option from the SSH tunnel dropdown:
- SSH Key Authentication uses a private key as your secret for establishing the SSH tunnel.
- Password Authentication uses a password as your secret for establishing the SSH Tunnel.
- For SSH Tunnel Jump Server Host, enter the hostname or IP address of the intermediate bastion server that Airbyte connects to.
- For SSH Connection Port, enter the bastion server’s port. The default SSH port is 22.
- For SSH Login Username, enter the username for connecting to the bastion server. Note: Use the operating system username, not the Postgres username.
- Configure authentication:
- If you selected SSH Key Authentication, set SSH Private Key to the private Key used for the SSH connection.
- If you selected Password Authentication, enter the operating system user’s password for the bastion server. Note: Use the operating system password, not the Postgres password.
Generate a private key for SSH tunnelling
The connector supports any SSH-compatible key format, including RSA and Ed25519. For example, run this command to generate an RSA key:
ssh-keygen -t rsa -m PEM -f myuser_rsa
The command creates a PEM-formatted private key. The public key remains in the standard format used by the bastion server’s authorized_keys file. Add the public key to the bastion host for the user you want Airbyte to use. Copy and paste the private key into the Airbyte connector configuration screen. This lets the connector sign in to the bastion server.
Configure Entra authentication
You can configure the Airbyte source to authenticate as a Microsoft Entra service principal. Airbyte can then authenticate to an Azure Postgres server using short-lived identity tokens. See the Microsoft documentation on this topic for details about configuring the server and other Entra resources.
To configure the Airbyte Postgres source with Entra authentication:
- Set
Usernameto the Entra ID, as described in Microsoft’s documentation. - Set the password to your Entra service principal’s client secret.
- Enable the Entra service principal authentication toggle.
- Provide the service principal’s Entra tenant ID and Entra client (or app) ID.
Limitations and troubleshooting
See in our Postgres troubleshooting guide for connector limitations and help troubleshooting your Postgres connector.
Data type mapping
Postgres data types map to the following data types during synchronisation, according to the Postgres documentation. See here for test value examples. If your data type isn’t listed or you find a problem, add a new test.
| Postgres Type | Resulting Type | Notes |
|---|---|---|
bigint | number | |
bigserial, serial8 | number | |
bit | string | Fixed-length bit string (e.g. “0100”). |
bit varying, varbit | string | Variable-length bit string (e.g. “0100”). |
boolean, bool | boolean | |
box | string | |
bytea | string | Variable length binary string with hex output format prefixed with “\x” (e.g. “\x6b707a”). |
character, char | string | |
character varying, varchar | string | |
cidr | string | |
circle | string | |
date | string | Parsed as ISO8601 date time at midnight. CDC mode doesn’t support era indicators. Issue: #14590 |
double precision, float, float8 | number | Infinity, -Infinity, and NaN are not supported and converted to null. Issue: #8902. |
hstore | string | |
inet | string | |
integer, int, int4 | number | |
interval | string | |
json | string | |
jsonb | string | |
line | string | |
lseg | string | |
macaddr | string | |
macaddr8 | string | |
money | number | |
numeric, decimal | number | Infinity, -Infinity, and NaN are not supported and converted to null. Issue: #8902. |
path | string | |
pg_lsn | string | |
point | string | |
polygon | string | |
real, float4 | number | |
smallint, int2 | number | |
smallserial, serial2 | number | |
serial, serial4 | number | |
text | string | |
time | string | Parsed as a time string without a time-zone in the ISO-8601 calendar system. |
timetz | string | Parsed as a time string with time-zone in the ISO-8601 calendar system. |
timestamp | string | Parsed as a date-time string without a time-zone in the ISO-8601 calendar system. |
timestamptz | string | Parsed as a date-time string with time-zone in the ISO-8601 calendar system. |
tsquery | string | |
tsvector | string | |
uuid | string | |
xml | string | |
enum | string | |
tsrange | string | |
array | array | E.g. ”[“10001”,“10002”,“10003”,“10004”]“. |
| composite type | string |
IP allow list
If you use Airbyte Cloud and restrict access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.
Changelog
Expand to review
| Version | Date | Pull Request | Subject |
|---|---|---|---|
| 3.8.11 | 2026-10-21 | 89637 | Upgrade to Bulk CDK 1.2.3: fix critical CVEs in Apache MINA (mina-core 2.0.31), MINA SSHD 2.20.0 and BouncyCastle 1.86; Kafka 4.1.2, Jetty 12.0.40, plexus-utils 3.6.2. |
| 3.8.10 | 2026-10-21 | 89623 | Upgrade to Bulk CDK 1.2.1, which brings Debezium 3.4.3 (CDC requires PostgreSQL 14 or later; the JDBC driver’s keep-alive no longer advances the replication slot while the captured tables are idle, so configure the Debezium heartbeat query on low-activity databases), CDC records written while Debezium shuts down are counted in the final checkpoint, Kotlin 2.4 / K2 runtime, Micronaut 4.10.31, Jackson 2.21.7. |
| 3.8.9 | 2026-10-14 | 88485 | Cold-start the CDC initial snapshot when the saved stream state is an empty object instead of skipping the stream. |
| 3.8.8 | 2026-10-12 | 88498 | Classify unreachable-host connection errors (connection refused, unknown host, connect timeout) as configuration errors that point to the host and port, the Postgres remote-connection settings and the Airbyte Cloud IP allowlist. Explain the Discovered zero tables. connection-test failure: the message now names the schema USAGE and table SELECT grants to add and asks to check the database name, the Schemas setting and the table filters. On Airbyte Cloud, reject localhost without an SSH tunnel and a URL or host:port pasted into Host with an explicit message before the connection is attempted. |
| 3.8.7 | 2026-10-09 | 88431 | Upgrade to Bulk CDK 1.1.14. |
| 3.8.6 | 2026-09-16 | 86369 | Fix NULL array columns taking the previous record’s value on the socket/protobuf CDC path. |
| 3.8.5 | 2026-07-29 | 82773 | Fix ClassCastException when an interval column has a default value during CDC. |
| 3.8.4 | 2026-07-28 | 79110 | Fix PostGIS geometry/geography columns returning NULL on CDC path. Values now always use hex-WEKB format, for both CDC and snapshots. |
| 3.8.3 | 2026-07-27 | 82782 | Remove the non-functional invalid_cdc_cursor_position_behavior option; invalidated replication slots always fail the sync. |
| 3.8.2 | 2026-07-24 | 82728 | Fix sync failures on money columns over the socket/protobuf output path. |
| 3.8.1 | 2026-06-04 | 79120 | Allow connecting without a password for passwordless auth methods. |
| 3.8.0 | 2026-06-02 | 75637 | Initial release of rewritten connector on the bulk CDK |
| 3.7.2 | 2026-03-04 | 74294 | Fix CDC bug where a replication slot can be advanced too far, losing needed WAL segments. Remove CVEs. |
| 3.7.1 | 2026-01-27 | 72396 | This version causes issues discovering streams for some users. Do not use. |
| 3.7.0 | 2025-08-12 | 57511 | Add configurations for Azure authentication to Azure Postgres servers. |
| 3.6.35 | 2025-06-18 | 61527 | Add error handling for connection issues and adopt the latest CDK version. |
| 3.6.34 | 2025-05-29 | 60876 | Cache CDC initial state once constructed. |
| 3.6.33 | 2025-05-13 | 60214 | Migrate to new Gradle flow. |
| 3.6.32 | 2025-05-08 | 59722 | Consolidate gradle set up. |
| 3.6.31 | 2025-04-24 | 58132 | Fix vulnerabilities in dependencies. |
| 3.6.30 | 2025-03-06 | 55234 | Update base image version for certified DB source connectors |
| 3.6.29 | 2025-02-13 | 53649 | Fix issue that column default value did not get converted |
| 3.6.28 | 2024-12-23 | 50870 | Use airbyte/java-connector-base:2.0.0 |
| 3.6.27 | 2024-12-23 | 50410 | Use a non root base image. |
| 3.6.26 | 2024-12-20 | 48495 | Increase MAX_FIRST_RECORD_WAIT_TIME and use Debezium 3.0.1 |
| 3.6.25 | 2024-12-17 | 49838 | Use a base image: airbyte/java-connector-base:1.0.0 |
| 3.6.24 | 2024-12-16 | 49469 | Simplify CTID_TABLE_BLOCK_SIZE query for Postgres integration |
| 3.6.23 | 2024-11-13 | 48482 | Convert large integer typed using NUMERIC(X, 0) into a BigInteger. l |
| 3.6.22 | 2024-10-02 | 46900 | Fixed a bug where source docs won’t render on Airbyte 1.1 |
| 3.6.21 | 2024-10-02 | 46322 | Support CDC against a read-replica (continuation) |
| 3.6.20 | 2024-10-01 | 46299 | Make postgres source compile and use the latest CDK |
| 3.6.19 | 2024-09-17 | 45639 | Adopt latest CDK to use the latest apache sshd mina to handle tcpkeepalive requests. |
| 3.6.18 | 2024-08-28 | 44878 | Enable tcpKeepAlive for jdbc connection. |
| 3.6.17 | 2024-08-27 | 44841 | Adopt latest CDK. |
| 3.6.16 | 2024-08-15 | 44119 | Fix incorrect final state on initial read in CDC mode. |
| 3.6.15 | 2024-08-12 | 43945 | Add missing replication slot config error. |
| 3.6.14 | 2024-08-08 | 43418 | Adopt latest CDK. |
| 3.6.13 | 2024-07-30 | 42869 | Adopt latest CDK. |
| 3.6.12 | 2024-07-30 | 42550 | Correctly report stream states. |
| 3.6.11 | 2024-07-29 | 42852 | Bump CDK version to latest to use new bug fixes on error translation. |
| 3.6.10 | 2024-07-23 | 42417 | Handle null error message in ConnectorExceptionHandler. |
| 3.6.9 | 2024-07-23 | 42421 | Remove final transient error emitter iterators. |
| 3.6.8 | 2024-07-22 | 41622 | Bump CDK version to latest. |
| 3.6.7 | 2024-07-22 | 42411 | Hide the “initial load timeout in hours” field by default in UI |
| 3.6.6 | 2024-07-22 | 41622 | Fix bug in CDC syncing |
| 3.6.5 | 2024-07-22 | 42024 | Fix a bug on resuming from a failed attempt. |
| 3.6.4 | 2024-07-17 | 42087 | Translate more errors for Postgres source. |
| 3.6.3 | 2024-07-19 | 42122 | Improve wass error message + logging. |
| 3.6.2 | 2024-07-18 | 42108 | Disable incremental sync for view streams in xmin replication mode |
| 3.6.1 | 2024-07-05 | 40716 | Fix typo in connector specification |
| 3.6.0 | 2024-07-17 | 40208 | Start using the new error Postgres source error handler that comes with a new error translation layer. |
| 3.5.2 | 2024-07-17 | 42068 | Add analytics for WASS case occurrence. |
| 3.5.1 | 2024-07-17 | 42055 | Add debezium heartbeat timeout back to shutdown debezium. |
| 3.5.0 | 2024-07-17 | 41651 | Implement WASS algo - large initial snapshots shouldn’t block CDC. |
| 3.4.26 | 2024-07-15 | 41654 | Allow null value for array typed columns in CDC. |
| 3.4.25 | 2024-07-12 | 41651 | Throw transient error if tables of interest and undergoing full vacuum. |
| 3.4.24 | 2024-07-05 | 41067 | Fix Postgres sending duplicated streams |
| 3.4.23 | 2024-07-01 | 40757 | Rollback 3.4.22. |
| 3.4.22 | 2024-07-05 | 40719 | Fix Postgres sending duplicated streams (rolled back in 3.4.23) |
| 3.4.21 | 2024-07-01 | 40516 | Remove dbz hearbeat. |
| 3.4.20 | 2024-06-23 | 40559 | Remove strict check for stream states of unknown types |
| 3.4.19 | 2024-06-23 | 40223 | Revert the changes introduced in version 3.4.15. |
| 3.4.18 | 2024-06-14 | 39349 | Full refresh stream sending internal count metadata. |
| 3.4.17 | 2024-06-13 | 39460 | Bump postgres JDBC driver version |
| 3.4.16 | 2024-05-29 | 39474 | Adopt latest CDK. |
| 3.4.15 | 2024-05-29 | 38773 | Connect with adaptiveFetch=true. |
| 3.4.14 | 2024-06-08 | 39353 | Upgrade Debezium to 2.6.2 |
| 3.4.13 | 2024-06-04 | 38875 | read() throws config exception upon detecting transaction ID wraparound. |
| 3.4.12 | 2024-06-04 | 38836 | check() throws config error upon detecting transaction ID wraparound. |
| 3.4.11 | 2024-06-04 | 38848 | Improve UI message and doc on xmin |
| 3.4.10 | 2024-05-29 | 38584 | Set is_resumable flag in discover. |
| 3.4.9 | 2024-05-29 | 38775 | Publish CDK |
| 3.4.9 | 2024-05-28 | 38716 | Publish CDK |
| 3.4.8 | 2024-05-28 | 38716 | Stream status for postgres |
| 3.4.7 | 2024-05-20 | 38365 | Rollback a previously version (3.4.6) |
| 3.4.6 | 2024-05-20 | 38289 | Connect with adaptiveFetch=true (rolled back in 3.4.7) |
| 3.4.5 | 2024-05-16 | 38303 | Streams not in the CDC publication still have a cursor and PK. |
| 3.4.4 | 2024-05-15 | 38208 | disable counts in full refresh stream in state message. |
| 3.4.3 | 2024-05-13 | 38104 | Handle transient error messages. |
| 3.4.2 | 2024-05-10 | 38171 | Bug fix on final state setup. |
| 3.4.1 | 2024-05-10 | 38130 | Bug fix on old PG where ctid column not found when stream is a view. |
| 3.4.0 | 2024-04-29 | 37112 | resumeable full refresh. |
| 3.3.33 | 2024-05-07 | 38030 | Mark PG hot standby error as transient. |
| 3.3.32 | 2024-04-30 | 37758 | Correct previous release to disable debezium retries |
| 3.3.31 | 2024-04-30 | 37754 | Add CDC logs |
| 3.3.30 | 2024-04-30 | 37726 | Remove debezium retries |
| 3.3.29 | 2024-04-23 | 37509 | remove excessive logs |
| 3.3.28 | 2024-04-23 | 37509 | Better error messages on switching between sync modes. |
| 3.3.27 | 2024-04-22 | 37445 | Remove legacy bad values handling code. |
| 3.3.26 | 2024-04-10 | 36982 | Populate airbyte_meta.changes for xmin path |
| 3.3.25 | 2024-04-10 | 36981 | Track latest CDK |
| 3.3.24 | 2024-04-10 | 36865 | Track latest CDK |
| 3.3.23 | 2024-04-02 | 36759 | Track latest CDK |
| 3.3.22 | 2024-04-01 | 36739 | Fix useLocalCdk flag. |
| 3.3.21 | 2024-03-25 | 36584 | Adopt Kotlin CDK. |
| 3.3.20 | 2024-03-25 | 36432 | Failure to serialize values from Postgres DB shouldn’t fail sync. |
| 3.3.19 | 2024-03-12 | 36333 | Use newest CDK - deprecate dbz iterator |
| 3.3.18 | 2024-03-12 | 35599 | Use newest CDK |
| 3.3.17 | 2024-03-12 | 35939 | Use lsn_commit value instead of lsn_proc for CDC checkpointing logic. |
| 3.3.16 | 2024-03-11 | 35904 | Adopt Java CDK 0.23.1- debezium retries. |
| 3.3.15 | 2024-02-29 | 34724 | Add record count in state message. |
| 3.3.14 | 2024-03-06 | 35842 | Add logging to understand cases with a large number of records with the same LSN. |
| 3.3.13 | 2024-02-27 | 35675 | Fix invalid cdc error message. |
| 3.3.12 | 2024-02-22 | 35569 | Fix logging bug. |
| 3.3.11 | 2024-02-20 | 35304 | Add config to throw an error on invalid CDC position and enable it by default. |
| 3.3.10 | 2024-02-13 | 34751 | Emit analytics message for invalid CDC cursor. |
| 3.3.9 | 2024-02-13 | 35224 | Adopt CDK 0.20.4 |
| 3.3.8 | 2024-02-08 | 34751 | Adopt CDK 0.19.0 |
| 3.3.7 | 2024-02-08 | 34781 | Add a setting in the setup page to advance the LSN. |
| 3.3.6 | 2024-02-07 | 34892 | Adopt CDK v0.16.6 |
| 3.3.5 | 2024-02-07 | 34948 | Adopt CDK v0.16.5 |
| 3.3.4 | 2024-01-31 | 34723 | Adopt CDK v0.16.3 |
| 3.3.3 | 2024-01-26 | 34573 | Adopt CDK v0.16.0 |
| 3.3.2 | 2024-01-24 | 34465 | Check xmin only if user selects xmin sync mode. |
| 3.3.1 | 2024-01-10 | 34119 | Adopt java CDK version 0.11.5. |
| 3.3.0 | 2023-12-19 | 33437 | Remove LEGACY state flag |
| 3.2.27 | 2023-12-18 | 33605 | Advance Postgres LSN for PG 14 & below. |
| 3.2.26 | 2023-12-11 | 32961 | Support for better debugging tools. |
| 3.2.25 | 2023-11-29 | 32961 | Bump debezium wait time default to 20 min. |
| 3.2.24 | 2023-11-28 | 32686 | Better logging to understand dbz closing reason attribution. |
| 3.2.23 | 2023-11-28 | 32891 | Fix CDK dependency in build. |
| 3.2.22 | 2023-11-22 | 32656 | Adopt java CDK version 0.5.0. |
| 3.2.21 | 2023-11-07 | 31856 | handle date/timestamp infinity values properly |
| 3.2.20 | 2023-11-06 | 32193 | Adopt java CDK version 0.4.1. |
| 3.2.19 | 2023-11-03 | 32050 | Adopt java CDK version 0.4.0. |
| 3.2.18 | 2023-11-01 | 29038 | Fix typo (s/Airbtye/Airbyte/) |
| 3.2.17 | 2023-11-01 | 32068 | Bump Debezium 2.2.0Final -> 2.4.0Final |
| 3.2.16 | 2023-10-31 | 31976 | Speed up tests involving Debezium |
| 3.2.15 | 2023-10-30 | 31960 | Adopt java CDK version 0.2.0. |
| 3.2.14 | 2023-10-24 | 31792 | fail sync when debezeum fails to shutdown cleanly |
| 3.2.14 | 2023-10-24 | 31792 | Fix error message link on issue with standby |
| 3.2.13 | 2023-10-16 | 31029 | Enforces encrypted traffic settings when env var DEPLOYMENT_MODE = CLOUD. |
| 3.1.13 | 2023-10-13 | 31309 | Addressed decimals being incorrectly deserialized into scientific notation. |
| 3.1.12 | 2023-10-12 | 31328 | Improvements to initial load of tables in older versions of postgres. |
| 3.1.11 | 2023-10-11 | 31322 | Correct previous release |
| 3.1.10 | 2023-09-29 | 30806 | Cap log line length to 32KB to prevent loss of records. |
| 3.1.9 | 2023-09-25 | 30534 | Fix JSONB[] column type handling bug. |
| 3.1.8 | 2023-09-20 | 30125 | Improve initial load performance for older versions of PostgreSQL. |
| 3.1.7 | 2023-09-05 | 29672 | Handle VACUUM happening during initial sync |
| 3.1.6 | 2023-08-24 | 29821 | Set replication_method display_type to radio, update titles and descriptions, and make CDC the default choice |
| 3.1.5 | 2023-08-22 | 29534 | Support “options” JDBC URL parameter |
| 3.1.4 | 2023-08-21 | 28687 | Under the hood: Add dependency on Java CDK v0.0.2. |
| 3.1.3 | 2023-08-03 | 28708 | Enable checkpointing snapshots in CDC connections |
| 3.1.2 | 2023-08-01 | 28954 | Fix an issue that prevented use of tables with names containing uppercase letters |
| 3.1.1 | 2023-07-31 | 28892 | Fix an issue that prevented use of cursor columns with names containing uppercase letters |
| 3.1.0 | 2023-07-25 | 28339 | Checkpointing initial load for incremental syncs: enabled for xmin and cursor based only. |
| 3.0.2 | 2023-07-18 | 28336 | Add full-refresh mode back to Xmin syncs. |
| 3.0.1 | 2023-07-14 | 28345 | Increment patch to trigger a rebuild |
| 3.0.0 | 2023-07-12 | 27442 | Set _ab_cdc_lsn as the source defined cursor for CDC mode to prepare for Destination v2 normalization |
| 2.1.1 | 2023-07-06 | 26723 | Add new xmin replication method. |
| 2.1.0 | 2023-06-26 | 27737 | License Update: Elv2 |
| 2.0.34 | 2023-06-20 | 27212 | Fix silent exception swallowing in StreamingJdbcDatabase |
| 2.0.33 | 2023-06-01 | 26873 | Add prepareThreshold=0 to JDBC url to mitigate PGBouncer prepared statement [X] already exists. |
| 2.0.32 | 2023-05-31 | 26810 | Remove incremental sync estimate from Postgres to increase performance. |
| 2.0.31 | 2023-05-25 | 26633 | Collect and log information related to full vacuum operation in db |
| 2.0.30 | 2023-05-25 | 26473 | CDC : Limit queue size |
| 2.0.29 | 2023-05-18 | 25898 | Translate Numeric values without decimal, e.g: NUMERIC(38,0), as BigInt instead of Double |
| 2.0.28 | 2023-04-27 | 25401 | CDC : Upgrade Debezium to version 2.2.0 |
| 2.0.27 | 2023-04-26 | 24971 | Emit stream status updates |
| 2.0.26 | 2023-04-26 | 25560 | Revert some logging changes |
| 2.0.25 | 2023-04-24 | 25459 | Better logging formatting |
| 2.0.24 | 2023-04-19 | 25345 | Logging : Log database indexes per stream |
| 2.0.23 | 2023-04-19 | 24582 | CDC : Enable frequent state emission during incremental syncs + refactor for performance improvement |
| 2.0.22 | 2023-04-17 | 25220 | Logging changes : Log additional metadata & clean up noisy logs |
| 2.0.21 | 2023-04-12 | 25131 | Make Client Certificate and Client Key always show |
| 2.0.20 | 2023-04-11 | 24859 | Removed SSL toggle and rely on SSL mode dropdown to enable/disable SSL |
| 2.0.19 | 2023-04-11 | 24656 | CDC minor refactor |
| 2.0.18 | 2023-04-06 | 24820 | Fix data loss bug during an initial failed non-CDC incremental sync |
| 2.0.17 | 2023-04-05 | 24622 | Allow streams not in CDC publication to be synced in Full-refresh mode |
| 2.0.16 | 2023-04-05 | 24895 | Fix spec for cloud |
| 2.0.15 | 2023-04-04 | 24833 | Fix Debezium retry policy configuration |
| 2.0.14 | 2023-04-03 | 24609 | Disallow the “disable” SSL Modes |
| 2.0.13 | 2023-03-28 | 24166 | Fix InterruptedException bug during Debezium shutdown |
| 2.0.12 | 2023-03-27 | 24373 | Add CDC checkpoint state messages |
| 2.0.11 | 2023-03-23 | 24446 | Set default SSL Mode to require in strict-encrypt |
| 2.0.10 | 2023-03-23 | 24417 | Add field groups and titles to improve display of connector setup form |
| 2.0.9 | 2023-03-22 | 20760 | Removed redundant date-time datatypes formatting |
| 2.0.8 | 2023-03-22 | 24255 | Add field groups and titles to improve display of connector setup form |
| 2.0.7 | 2023-03-21 | 24207 | Fix incorrect schema change warning in CDC mode |
| 2.0.6 | 2023-03-21 | 24271 | Fix NPE in CDC mode |
| 2.0.5 | 2023-03-21 | 21533 | Add integration with datadog |
| 2.0.4 | 2023-03-21 | 24275 | Fix error with CDC checkpointing |
| 2.0.3 | 2023-03-14 | 24000 | Removed check method call on read. |
| 2.0.2 | 2023-03-13 | 21727 | Add state checkpointing for CDC sync. |
| 2.0.0 | 2023-03-06 | 23112 | Upgrade Debezium version to 2.1.2 |
| 1.0.51 | 2023-03-02 | 23642 | Revert : Support JSONB datatype for Standard sync mode |
| 1.0.50 | 2023-02-27 | 21695 | Support JSONB datatype for Standard sync mode |
| 1.0.49 | 2023-02-24 | 23383 | Fixed bug with non readable double-quoted values within a database name or column name |
| 1.0.48 | 2023-02-23 | 22623 | Increase max fetch size of JDBC streaming mode |
| 1.0.47 | 2023-02-22 | 23138 | Fix previous versions which doesn’t verify privileges correctly, preventing CDC syncs to run. |
| 1.0.46 | 2023-02-21 | 23105 | Include log levels and location information (class, method and line number) with source connector logs published to Airbyte Platform. |
| 1.0.45 | 2023-02-09 | 22371 | Ensures that user has required privileges for CDC syncs. |
| 2023-02-15 | 23028 | ||
| 1.0.44 | 2023-02-06 | 22221 | Exclude new set of system tables when using pg_stat_statements extension. |
| 1.0.43 | 2023-02-06 | 21634 | Improve Standard sync performance by caching objects. |
| 1.0.42 | 2023-01-23 | 21523 | Check for null in cursor values before replacing. |
| 1.0.41 | 2023-01-25 | 20939 | Adjust batch selection memory limits databases. |
| 1.0.40 | 2023-01-24 | 21825 | Put back the original change that will cause an incremental sync to error if table contains a NULL value in cursor column. |
| 1.0.39 | 2023-01-20 | 21683 | Speed up esmtimates for trace messages in non-CDC mode. |
| 1.0.38 | 2023-01-17 | 20346 | Consolidate date/time values mapping for JDBC sources |
| 1.0.37 | 2023-01-17 | 20783 | Emit estimate trace messages for non-CDC mode. |
| 1.0.36 | 2023-01-11 | 21003 | Handle null values for array data types in CDC mode gracefully. |
| 1.0.35 | 2023-01-04 | 20469 | Introduce feature to make LSN commit behaviour configurable. |
| 1.0.34 | 2022-12-13 | 20378 | Improve descriptions |
| 1.0.33 | 2022-12-12 | 18959 | CDC : Don’t timeout if snapshot is not complete. |
| 1.0.32 | 2022-12-12 | 20192 | Only throw a warning if cursor column contains null values. |
| 1.0.31 | 2022-12-02 | 19889 | Check before each sync and stop if an incremental sync cursor column contains a null value. |
| 2022-12-02 | 19985 | Reenable incorrectly-disabled wal2json CDC plugin | |
| 1.0.30 | 2022-11-29 | 19024 | Skip tables from schema where user do not have Usage permission during discovery. |
| 1.0.29 | 2022-11-29 | 19623 | Mark PSQLException related to using replica that is configured as a hot standby server as config error. |
| 1.0.28 | 2022-11-28 | 19514 | Adjust batch selection memory limits databases. |
| 1.0.27 | 2022-11-28 | 16990 | Handle arrays data types |
| 1.0.26 | 2022-11-18 | 19551 | Fixes bug with ssl modes |
| 1.0.25 | 2022-11-16 | 19004 | Use Debezium heartbeats to improve CDC replication of large databases. |
| 1.0.24 | 2022-11-07 | 19291 | Default timeout is reduced from 1 min to 10sec |
| 1.0.23 | 2022-11-07 | 19025 | Stop enforce SSL if ssl mode is disabled |
| 1.0.22 | 2022-10-31 | 18538 | Encode database name |
| 1.0.21 | 2022-10-25 | 18256 | Disable allow and prefer ssl modes in CDC mode |
| 1.0.20 | 2022-10-25 | 18383 | Better SSH error handling + messages |
| 1.0.19 | 2022-10-21 | 18263 | Fixes bug introduced in 15833 and adds better error messaging for SSH tunnel in Destinations |
| 1.0.18 | 2022-10-19 | 18087 | Better error messaging for configuration errors (SSH configs, choosing an invalid cursor) |
| 1.0.17 | 2022-10-17 | 18041 | Fixes bug introduced 2022-09-12 with SshTunnel, handles iterator exception properly |
| 1.0.16 | 2022-10-13 | 16238 | Update incremental query to avoid data missing when new data is inserted at the same time as a sync starts under non-CDC incremental mode |
| 1.0.15 | 2022-10-11 | 17782 | Handle 24:00:00 value for Time column |
| 1.0.14 | 2022-10-03 | 17515 | Fix an issue preventing connection using client certificate |
| 1.0.13 | 2022-10-01 | 17459 | Upgrade debezium version to 1.9.6 from 1.9.2 |
| 1.0.12 | 2022-09-27 | 17299 | Improve error handling for strict-encrypt postgres source |
| 1.0.11 | 2022-09-26 | 17131 | Allow nullable columns to be used as cursor |
| 1.0.10 | 2022-09-14 | 15668 | Wrap logs in AirbyteLogMessage |
| 1.0.9 | 2022-09-13 | 16657 | Improve CDC record queueing performance |
| 1.0.8 | 2022-09-08 | 16202 | Adds error messaging factory to UI |
| 1.0.7 | 2022-08-30 | 16114 | Prevent traffic going on an unsecured channel in strict-encryption version of source postgres |
| 1.0.6 | 2022-08-30 | 16138 | Remove unnecessary logging |
| 1.0.5 | 2022-08-25 | 15993 | Add support for connection over SSL in CDC mode |
| 1.0.4 | 2022-08-23 | 15877 | Fix temporal data type bug which was causing failure in CDC mode |
| 1.0.3 | 2022-08-18 | 14356 | DB Sources: only show a table can sync incrementally if at least one column can be used as a cursor field |
| 1.0.2 | 2022-08-11 | 15538 | Allow additional properties in db stream state |
| 1.0.1 | 2022-08-10 | 15496 | Fix state emission in incremental sync |
| 2022-08-10 | 15481 | Fix data handling from WAL logs in CDC mode | |
| 1.0.0 | 2022-08-05 | 15380 | Change connector label to generally_available (requires upgrading your Airbyte platform to v0.40.0-alpha) |
| 0.4.44 | 2022-08-05 | 15342 | Adjust titles and descriptions in spec.json |
| 0.4.43 | 2022-08-03 | 15226 | Make connectionTimeoutMs configurable through JDBC url parameters |
| 0.4.42 | 2022-08-03 | 15273 | Fix a bug in 0.4.36 and correctly parse the CDC initial record waiting time |
| 0.4.41 | 2022-08-03 | 15077 | Sync data from beginning if the LSN is no longer valid in CDC |
| 2022-08-03 | 14903 | Emit state messages more frequently (⛔ this version has a bug; use 1.0.1 instead | |
| 0.4.40 | 2022-08-03 | 15187 | Add support for BCE dates/timestamps |
| 2022-08-03 | 14534 | Align regular and CDC integration tests and data mappers | |
| 0.4.39 | 2022-08-02 | 14801 | Fix multiple log bindings |
| 0.4.38 | 2022-07-26 | 14362 | Integral columns are now discovered as int64 fields. |
| 0.4.37 | 2022-07-22 | 14714 | Clarified error message when invalid cursor column selected |
| 0.4.36 | 2022-07-21 | 14451 | Make initial CDC waiting time configurable (⛔ this version has a bug and will not work; use 0.4.42 instead) |
| 0.4.35 | 2022-07-14 | 14574 | Removed additionalProperties:false from JDBC source connectors |
| 0.4.34 | 2022-07-17 | 13840 | Added the ability to connect using different SSL modes and SSL certificates. |
| 0.4.33 | 2022-07-14 | 14586 | Validate source JDBC url parameters |
| 0.4.32 | 2022-07-07 | 14694 | Force to produce LEGACY state if the use stream capable feature flag is set to false |
| 0.4.31 | 2022-07-07 | 14447 | Under CDC mode, retrieve only those tables included in the publications |
| 0.4.30 | 2022-06-30 | 14251 | Use more simple and comprehensive query to get selectable tables |
| 0.4.29 | 2022-06-29 | 14265 | Upgrade postgresql JDBC version to 42.3.5 |
| 0.4.28 | 2022-06-23 | 14077 | Use the new state management |
| 0.4.26 | 2022-06-17 | 13864 | Updated stacktrace format for any trace message errors |
| 0.4.25 | 2022-06-15 | 13823 | Publish adaptive postgres source that enforces ssl on cloud + Debezium version upgrade to 1.9.2 from 1.4.2 |
| 0.4.24 | 2022-06-14 | 13549 | Fixed truncated precision if the value of microseconds or seconds is 0 |
| 0.4.23 | 2022-06-13 | 13745 | Fixed handling datetime cursors when upgrading from older versions of the connector |
| 0.4.22 | 2022-06-09 | 13655 | Fixed bug with unsupported date-time datatypes during incremental sync |
| 0.4.21 | 2022-06-06 | 13435 | Adjust JDBC fetch size based on max memory and max row size |
| 0.4.20 | 2022-06-02 | 13367 | Added convertion hstore to json format |
| 0.4.19 | 2022-05-25 | 13166 | Added timezone awareness and handle BC dates |
| 0.4.18 | 2022-05-25 | 13083 | Add support for tsquey type |
| 0.4.17 | 2022-05-19 | 13016 | CDC modify schema to allow null values |
| 0.4.16 | 2022-05-14 | 12840 | Added custom JDBC parameters field |
| 0.4.15 | 2022-05-13 | 12834 | Fix the bug that the connector returns empty catalog for Azure Postgres database |
| 0.4.14 | 2022-05-08 | 12689 | Add table retrieval according to role-based SELECT privilege |
| 0.4.13 | 2022-05-05 | 10230 | Explicitly set null value for field in json |
| 0.4.12 | 2022-04-29 | 12480 | Query tables with adaptive fetch size to optimize JDBC memory consumption |
| 0.4.11 | 2022-04-11 | 11729 | Bump mina-sshd from 2.7.0 to 2.8.0 |
| 0.4.10 | 2022-04-08 | 11798 | Fixed roles for fetching materialized view processing |
| 0.4.8 | 2022-02-21 | 10242 | Fixed cursor for old connectors that use non-microsecond format. Now connectors work with both formats |
| 0.4.7 | 2022-02-18 | 10242 | Updated timestamp transformation with microseconds |
| 0.4.6 | 2022-02-14 | 10256 | (unpublished) Add -XX:+ExitOnOutOfMemoryError JVM option |
| 0.4.5 | 2022-02-08 | 10173 | Improved discovering tables in case if user does not have permissions to any table |
| 0.4.4 | 2022-01-26 | 9807 | Update connector fields title/description |
| 0.4.3 | 2022-01-24 | 9554 | Allow handling of java sql date in CDC |
| 0.4.2 | 2022-01-13 | 9360 | Added schema selection |
| 0.4.1 | 2022-01-05 | 9116 | Added materialized views processing |
| 0.4.0 | 2021-12-13 | 8726 | Support all Postgres types |
| 0.3.17 | 2021-12-01 | 8371 | Fixed incorrect handling “\n” in ssh key |
| 0.3.16 | 2021-11-28 | 7995 | Fixed money type with amount > 1000 |
| 0.3.15 | 2021-11-26 | 8266 | Fixed the case, when Views are not listed during schema discovery |
| 0.3.14 | 2021-11-17 | 8010 | Added checking of privileges before table internal discovery |
| 0.3.13 | 2021-10-26 | 7339 | Support or improve support for Interval, Money, Date, various geometric data types, inventory_items, and others |
| 0.3.12 | 2021-09-30 | 6585 | Improved SSH Tunnel key generation steps |
| 0.3.11 | 2021-09-02 | 5742 | Add SSH Tunnel support |
| 0.3.9 | 2021-08-17 | 5304 | Fix CDC OOM issue |
| 0.3.8 | 2021-08-13 | 4699 | Added json config validator |
| 0.3.4 | 2021-06-09 | 3973 | Add AIRBYTE_ENTRYPOINT for Kubernetes support |
| 0.3.3 | 2021-06-08 | 3960 | Add method field in specification parameters |
| 0.3.2 | 2021-05-26 | 3179 | Remove isCDC logging |
| 0.3.1 | 2021-04-21 | 2878 | Set defined cursor for CDC |
| 0.3.0 | 2021-04-21 | 2990 | Support namespaces |
| 0.2.7 | 2021-04-16 | 2923 | SSL spec as optional |
| 0.2.6 | 2021-04-16 | 2757 | Support SSL connection |
| 0.2.5 | 2021-04-12 | 2859 | CDC bugfix |
| 0.2.4 | 2021-04-09 | 2548 | Support CDC |
| 0.2.3 | 2021-03-28 | 2600 | Add NCHAR and NVCHAR support to DB and cursor type casting |
| 0.2.2 | 2021-03-26 | 2460 | Destination supports destination sync mode |
| 0.2.1 | 2021-03-18 | 2488 | Sources support primary keys |
| 0.2.0 | 2021-03-09 | 2238 | Protocol allows future/unknown properties |
| 0.1.13 | 2021-02-02 | 1887 | Migrate AbstractJdbcSource to use iterators |
| 0.1.12 | 2021-01-25 | 1746 | Fix NPE in State Decorator |
| 0.1.11 | 2021-01-25 | 1765 | Add field titles to specification |
| 0.1.10 | 2021-01-19 | 1724 | Fix JdbcSource handling of tables with same names in different schemas |
| 0.1.9 | 2021-01-14 | 1655 | Fix JdbcSource OOM |
| 0.1.8 | 2021-01-13 | 1588 | Handle invalid numeric values in JDBC source |
| 0.1.7 | 2021-01-08 | 1307 | Migrate Postgres and MySql to use new JdbcSource |
| 0.1.6 | 2020-12-09 | 1172 | Support incremental sync |
| 0.1.5 | 2020-11-30 | 1038 | Change JDBC sources to discover more than standard schemas |
| 0.1.4 | 2020-11-30 | 1046 | Add connectors using an index YAML file |