Advanced Analytica Advanced Analytica: IBOM
← All connectors
Postgres logo

Source connector

Postgres

Set up Postgres as a source and move its data into your chosen destination.

Version 3.8.11 certified generally available

Postgres

Airbyte’s certified Postgres connector includes these features:

  • Replicate data from tables, views and materialized views. Other objects, such as indexes and permissions, aren’t replicated to the destination.
  • Keep your data fresh using multiple methods, including Change Data Capture (CDC) and replication with the xmin system column.
  • Use all available sync modes, giving you flexibility in how data reaches your destination.
  • Replicate tables reliably at any size using checkpointing and chunked database reads.

The sections below cover quick start, advanced setup and reference information, including data type mapping and changelogs. See here to troubleshoot Postgres connector issues.

Please note the required minimum platform version is v0.58.0 for this connector.

Quick start

Complete these minimum steps to configure a Postgres connector:

  1. Create a dedicated, read-only Postgres user with permission to replicate data.
  2. Create a Postgres source in the Airbyte UI using the xmin system column.
  3. (Airbyte Cloud Only) Allow inbound traffic from Airbyte IPs.

After completing these steps, you can select Postgres as a source for data replication.

Step 1: Create a read-only Postgres user

These steps create a dedicated, read-only user for data replication. Alternatively, you can use an existing Postgres database user.

Run these commands to create a user:

CREATE USER <user_name> PASSWORD 'your_password_here';

Now give this user read-only access to the relevant schemas and tables. Repeat this command for each schema you plan to replicate:

GRANT USAGE ON SCHEMA <schema_name> TO <user_name>;
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO <user_name>;
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO <user_name>;

Step 2: Create a Postgres source

In your Airbyte Cloud or Airbyte Open Source account, select Sources from the left navigation bar. Search for Postgres, then create a Postgres source.

Enter the required information:

  1. Enter your Postgres database’s hostname, port number and name. On Airbyte Cloud, the host must be reachable from the internet. A database on localhost, 127.0.0.1 or a private network is reachable only through an SSH tunnel. This also provides the encryption that Airbyte Cloud requires. In Host, enter only the hostname or IP address, not a URL or host:port.
2. Optionally, list the schemas containing tables you want to sync. Schema names are case-sensitive. If you don't provide them, all available schemas are discovered. 3. Enter the username you created in [Step 1](#step-1-create-a-dedicated-read-only-postgres-user). If your Postgres user uses a password, enter it in **Password**. The password is optional. Some PostgreSQL authentication methods don't require one, including `trust`, `peer` and certificate-based authentication. 4. Select an SSL mode. You'll usually choose `require` or `verify-ca`, which always require encryption. `verify-ca` also requires certificates from your Postgres database. Airbyte Cloud never connects without encryption. The unencrypted modes `disable`, `allow` and `prefer` require an SSH tunnel on Airbyte Cloud. See [Connecting with SSL or SSH Tunneling](#connecting-with-ssl-or-ssh-tunneling). 5. Select `Standard (xmin)` from the available replication methods. This method uses the [xmin system column](#xmin) to replicate data reliably from your database. 1. If your database is particularly large (> 500 GB), you'll benefit from [configuring your Postgres source using logical replication (CDC)](#cdc).

Step 3: (Airbyte Cloud Only) Allow inbound traffic

If you use Airbyte Cloud, configure your database to allow inbound traffic from Airbyte IPs. The Airbyte Security docs lists every IP you need to allow.

Airbyte Cloud never connects to a database without encryption. Use TLS/SSL with require, verify-ca or verify-full, or connect through an SSH tunnel. The tunnel encrypts the connection between Airbyte Cloud and your network. On Airbyte Cloud, the unencrypted modes disable, allow and prefer require an SSH tunnel. Airbyte Cloud can’t directly reach a database on localhost or a private network. Connect through an SSH tunnel. Before connecting, Airbyte Cloud checks the Host field. localhost, 127.0.0.1 or 0.0.0.0 with No Tunnel fails the connection test with Airbyte Cloud cannot reach a database on your own machine. Use a hostname reachable from the internet, or configure an SSH tunnel.. A URL, credentials or host:port in the field fails with Enter only the hostname in Host; put the port in the Port field..

Now select Set up source in the Airbyte UI. Airbyte tests the connection to your database. When the test succeeds, your Airbyte Postgres source is configured.

If the test fails with Airbyte could not reach the database at the configured host and port, Airbyte couldn’t open a TCP connection to your database. Check the host and port. Confirm that Postgres accepts remote connections through listen_addresses in postgresql.conf and a matching pg_hba.conf entry. Confirm that your firewall or security group allows the Airbyte IP addresses listed above.

Advanced CDC configuration

Airbyte uses logical replication from the Postgres write-ahead log (WAL) to capture deletes incrementally through a replication plugin:

  • See here to learn how Airbyte implements CDC.
  • See here for Postgres CDC requirements and limitations.

Configure your Postgres source with CDC when:

  • You need a record of deletions.
  • Your database is very large (500 GB or more).
  • Your table has a primary key but no suitable cursor field for incremental syncing (updated_at).

After completing the quick start, follow these additional steps to configure your Postgres source with CDC:

  1. Give the read-only user additional REPLICATION permissions.
  2. Enable logical replication on your Postgres database.
  3. Create a replication slot on your Postgres database.
  4. Create publication and replication identities for each Postgres table.
  5. Enable CDC replication in the Airbyte UI.

Step 1: Prepopulate the source configuration

Follow the quick start steps to confirm that Airbyte can connect to your Postgres database before configuring CDC settings.

CDC requires PostgreSQL 14 or later. Debezium 3.4 is the CDC engine in connector version 3.8.10 and later. It no longer supports PostgreSQL 13 or earlier.

For CDC, you can connect to primary/master databases or replicas. Using a replica requires Postgres 16.1 or later and connector version 3.6.21 or later. You must also enable additional configurations on the database instance. See the Postgres official documentation for help.

Step 2: Add read-only user permissions

To configure CDC for the Postgres source connector, grant the user created in step 1 of the quick start these REPLICATION permissions:

ALTER USER <user_name> REPLICATION;

Step 3: Enable logical replication

Enable logical replication by following the steps for your deployment environment.

Bare metal, VMs and Docker

For bare metal, VMs (EC2/GCE/etc) or Docker, configure these parameters in your Postgres database’s postgresql.conf file.

ParameterDescriptionSet value to
wal_levelType of coding used within the Postgres write-ahead loglogical
max_wal_sendersThe maximum number of processes used for handling WAL changesmin: 1
max_replication_slotsThe maximum number of replication slots that are allowed to stream WAL changes1 (if Airbyte is the only service reading subscribing to WAL changes. More than 1 if other services are also reading from the WAL)

AWS Postgres RDS or Aurora

  1. Open the Configuration tab for your DB cluster.
  2. Find your cluster parameter group. Edit its parameters or create a copy to edit. If you create a copy, change your cluster’s parameter group before restarting.
  3. On the parameter group page, search for rds.logical_replication. Select the row, then click Edit parameters. Set the value to 1.
  4. Wait for the maintenance window to restart the instance automatically, or restart it manually.

:::note AWS Aurora implements a CDC caching layer that’s incompatible with Airbyte’s CDC implementation. To use Airbyte with AWS Aurora, disable the CDC caching layer. Set the rds.logical_wal_cache parameter to 0 in the AWS Aurora parameter group. :::

Azure Database for Postgres

In the Azure Portal, use your PostgreSQL instance’s replication menu to change the Postgres DB replication mode to logical. Alternatively, run this command with the Azure CLI:

az postgres server configuration set --resource-group group --server-name server --name azure.replication_support --value logical
az postgres server restart --resource-group group --name server

Step 4: Create a replication slot on your Postgres database

Airbyte requires a replication slot configured only for its use. Configure only one source to use this replication slot.

Airbyte requires the pgoutput plugin for this step. Run the following command as the user with the new REPLICATION role. It creates a replication slot named airbyte_slot using pgoutput:

SELECT pg_create_logical_replication_slot('airbyte_slot', 'pgoutput');

The command output includes the replication slot name. Enter this name on the Airbyte source setup page.

Step 5: Create publications and identities

Follow these steps for each table you want to replicate with CDC:
  1. Add a replication identity, which distinguishes between rows, for each table you want to replicate:
ALTER TABLE tbl1 REPLICA IDENTITY DEFAULT;

In rare cases, consider replica identity type full if your tables support TOAST or contain very large field values: ALTER TABLE tbl1 REPLICA IDENTITY FULL;. Ensure TOAST-able tables use non-TOAST-able primary keys, such as integers or varchars. There will only be a modest increase in resource utilization, in addition to increased WAL storage size.

  1. Create the Postgres publication. Include every table you want to replicate in the publication:
CREATE PUBLICATION airbyte_publication FOR TABLE <tbl1, tbl2, tbl3>;

You can customise the publication name. See the Postgres docs if you need to add or remove publication tables later.

:::note The Airbyte UI currently lets you select any tables for CDC. A selected table won’t replicate if it isn’t part of the publication. A published table without a replication identity gets one automatically during the first run. This requires the Airbyte user to have the necessary permissions. :::

Step 6: Enable CDC replication

In your Postgres source, change the update method to Read Changes using Change Data Capture (CDC). Enter the replication slot and publication you created.

:::note If max_slot_wal_keep_size is exceeded, PostgreSQL can invalidate the replication slot. The slot then has wal_status = lost and a null restart_lsn. The connector fails the sync and can’t recover automatically. Drop and recreate the replication slot, then reset the connection for a full re-sync. :::

Postgres replication methods

The Postgres source offers three methods for replicating updates: CDC, xmin and standard with a user-defined cursor. CDC and xmin are the most reliable methods for updating your data.

CDC

Airbyte uses logical replication from the Postgres write-ahead log (WAL) to capture deletes incrementally through a replication plugin. See Change Data Capture (CDC) to learn how Airbyte implements CDC. Configure your Postgres source with CDC when:

  • You need a record of deletions.
  • Your database is very large (500 GB or more).
  • Your table has a primary key but no suitable cursor field for incremental syncing (updated_at).

If you need a destination snapshot but CDC limitations prevent its use, consider the xmin replication method.

Xmin

Xmin is the new cursor-less replication method for Postgres. Cursorless syncs replicate new or updated rows without requiring you to choose a cursor field. The xmin system column, available in every Postgres database, tracks inserts and updates to your source data.

Xmin is a good fit when:

  • You don’t have a well-defined cursor candidate for Standard incremental mode.
  • You want to replace a previously configured full-refresh sync.
  • Your database doesn’t have heavy writes that could cause transaction ID wraparound.
  • You aren’t replicating non-materialized views. Xmin replication doesn’t support non-materialized views.

Connect with SSL or SSH tunnelling

SSL modes

Airbyte uses SSL (`require`) by default. Airbyte Cloud never connects to a database without encryption. Use TLS/SSL with `require`, `verify-ca` or `verify-full`, or connect through an [SSH tunnel](#ssh-tunneling). The tunnel encrypts the connection between Airbyte Cloud and your network. On Airbyte Cloud, the unencrypted modes `disable`, `allow` and `prefer` require an SSH tunnel. With `No Tunnel`, the connection test fails with `Connection from Airbyte Cloud requires SSL encryption or an SSH tunnel`. You'll usually choose `require` or `verify-ca`, which always require encryption. `verify-ca` also requires certificates from your Postgres database.

These SSL connection modes are available:

  • disable disables encrypted communication between Airbyte and the source.
  • allow enables encrypted communication only when the source requires it.
  • prefer allows unencrypted communication only when the source doesn’t support encryption.
  • require always requires encryption. The connection fails if the source doesn’t support encryption.
  • verify-ca always requires encryption and verifies that the source has a valid SSL certificate.
  • verify-full always requires encryption and verifies the source’s identity.

SSH tunnelling

Airbyte Cloud requires encrypted communication, which an SSH tunnel provides. SSH encrypts the connection between Airbyte Cloud and your bastion host. On Airbyte Cloud, the unencrypted modes disable, allow and prefer require SSH Key Authentication or Password Authentication. With No Tunnel, the connection fails. An SSH tunnel also lets Airbyte Cloud reach databases that aren’t exposed to the internet. This includes databases on localhost or a private network. Airbyte connects to a bastion host exposed to the Airbyte IPs. The bastion then forwards the connection to the database.

For SSH Tunnel Method, select:

  • No Tunnel to connect directly to the database.
  • SSH Key Authentication to establish the SSH tunnel using an RSA private key as your secret.
  • Password Authentication to establish the SSH tunnel using a password as your secret.

Connect through an SSH tunnel

You can connect to a Postgres instance through an SSH tunnel.

An SSH tunnel configures Airbyte to connect through an intermediate server with direct database access. This server is also called a bastion or jump server. Airbyte connects to the bastion, which then connects directly to the database server.

To connect to a Postgres instance through an SSH tunnel:

  1. While setting up the Postgres source connector, select an option from the SSH tunnel dropdown:
    • SSH Key Authentication uses a private key as your secret for establishing the SSH tunnel.
    • Password Authentication uses a password as your secret for establishing the SSH Tunnel.
  2. For SSH Tunnel Jump Server Host, enter the hostname or IP address of the intermediate bastion server that Airbyte connects to.
  3. For SSH Connection Port, enter the bastion server’s port. The default SSH port is 22.
  4. For SSH Login Username, enter the username for connecting to the bastion server. Note: Use the operating system username, not the Postgres username.
  5. Configure authentication:
    • If you selected SSH Key Authentication, set SSH Private Key to the private Key used for the SSH connection.
    • If you selected Password Authentication, enter the operating system user’s password for the bastion server. Note: Use the operating system password, not the Postgres password.

Generate a private key for SSH tunnelling

The connector supports any SSH-compatible key format, including RSA and Ed25519. For example, run this command to generate an RSA key:

ssh-keygen -t rsa -m PEM -f myuser_rsa

The command creates a PEM-formatted private key. The public key remains in the standard format used by the bastion server’s authorized_keys file. Add the public key to the bastion host for the user you want Airbyte to use. Copy and paste the private key into the Airbyte connector configuration screen. This lets the connector sign in to the bastion server.

Configure Entra authentication

You can configure the Airbyte source to authenticate as a Microsoft Entra service principal. Airbyte can then authenticate to an Azure Postgres server using short-lived identity tokens. See the Microsoft documentation on this topic for details about configuring the server and other Entra resources.

To configure the Airbyte Postgres source with Entra authentication:

  1. Set Username to the Entra ID, as described in Microsoft’s documentation.
  2. Set the password to your Entra service principal’s client secret.
  3. Enable the Entra service principal authentication toggle.
  4. Provide the service principal’s Entra tenant ID and Entra client (or app) ID.

Limitations and troubleshooting

See in our Postgres troubleshooting guide for connector limitations and help troubleshooting your Postgres connector.

Data type mapping

Postgres data types map to the following data types during synchronisation, according to the Postgres documentation. See here for test value examples. If your data type isn’t listed or you find a problem, add a new test.

Postgres TypeResulting TypeNotes
bigintnumber
bigserial, serial8number
bitstringFixed-length bit string (e.g. “0100”).
bit varying, varbitstringVariable-length bit string (e.g. “0100”).
boolean, boolboolean
boxstring
byteastringVariable length binary string with hex output format prefixed with “\x” (e.g. “\x6b707a”).
character, charstring
character varying, varcharstring
cidrstring
circlestring
datestringParsed as ISO8601 date time at midnight. CDC mode doesn’t support era indicators. Issue: #14590
double precision, float, float8numberInfinity, -Infinity, and NaN are not supported and converted to null. Issue: #8902.
hstorestring
inetstring
integer, int, int4number
intervalstring
jsonstring
jsonbstring
linestring
lsegstring
macaddrstring
macaddr8string
moneynumber
numeric, decimalnumberInfinity, -Infinity, and NaN are not supported and converted to null. Issue: #8902.
pathstring
pg_lsnstring
pointstring
polygonstring
real, float4number
smallint, int2number
smallserial, serial2number
serial, serial4number
textstring
timestringParsed as a time string without a time-zone in the ISO-8601 calendar system.
timetzstringParsed as a time string with time-zone in the ISO-8601 calendar system.
timestampstringParsed as a date-time string without a time-zone in the ISO-8601 calendar system.
timestamptzstringParsed as a date-time string with time-zone in the ISO-8601 calendar system.
tsquerystring
tsvectorstring
uuidstring
xmlstring
enumstring
tsrangestring
arrayarrayE.g. ”[“10001”,“10002”,“10003”,“10004”]“.
composite typestring

IP allow list

If you use Airbyte Cloud and restrict access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review
VersionDatePull RequestSubject
3.8.112026-10-2189637Upgrade to Bulk CDK 1.2.3: fix critical CVEs in Apache MINA (mina-core 2.0.31), MINA SSHD 2.20.0 and BouncyCastle 1.86; Kafka 4.1.2, Jetty 12.0.40, plexus-utils 3.6.2.
3.8.102026-10-2189623Upgrade to Bulk CDK 1.2.1, which brings Debezium 3.4.3 (CDC requires PostgreSQL 14 or later; the JDBC driver’s keep-alive no longer advances the replication slot while the captured tables are idle, so configure the Debezium heartbeat query on low-activity databases), CDC records written while Debezium shuts down are counted in the final checkpoint, Kotlin 2.4 / K2 runtime, Micronaut 4.10.31, Jackson 2.21.7.
3.8.92026-10-1488485Cold-start the CDC initial snapshot when the saved stream state is an empty object instead of skipping the stream.
3.8.82026-10-1288498Classify unreachable-host connection errors (connection refused, unknown host, connect timeout) as configuration errors that point to the host and port, the Postgres remote-connection settings and the Airbyte Cloud IP allowlist. Explain the Discovered zero tables. connection-test failure: the message now names the schema USAGE and table SELECT grants to add and asks to check the database name, the Schemas setting and the table filters. On Airbyte Cloud, reject localhost without an SSH tunnel and a URL or host:port pasted into Host with an explicit message before the connection is attempted.
3.8.72026-10-0988431Upgrade to Bulk CDK 1.1.14.
3.8.62026-09-1686369Fix NULL array columns taking the previous record’s value on the socket/protobuf CDC path.
3.8.52026-07-2982773Fix ClassCastException when an interval column has a default value during CDC.
3.8.42026-07-2879110Fix PostGIS geometry/geography columns returning NULL on CDC path. Values now always use hex-WEKB format, for both CDC and snapshots.
3.8.32026-07-2782782Remove the non-functional invalid_cdc_cursor_position_behavior option; invalidated replication slots always fail the sync.
3.8.22026-07-2482728Fix sync failures on money columns over the socket/protobuf output path.
3.8.12026-06-0479120Allow connecting without a password for passwordless auth methods.
3.8.02026-06-0275637Initial release of rewritten connector on the bulk CDK
3.7.22026-03-0474294Fix CDC bug where a replication slot can be advanced too far, losing needed WAL segments. Remove CVEs.
3.7.12026-01-2772396This version causes issues discovering streams for some users. Do not use.
3.7.02025-08-1257511Add configurations for Azure authentication to Azure Postgres servers.
3.6.352025-06-1861527Add error handling for connection issues and adopt the latest CDK version.
3.6.342025-05-2960876Cache CDC initial state once constructed.
3.6.332025-05-1360214Migrate to new Gradle flow.
3.6.322025-05-0859722Consolidate gradle set up.
3.6.312025-04-2458132Fix vulnerabilities in dependencies.
3.6.302025-03-0655234Update base image version for certified DB source connectors
3.6.292025-02-1353649Fix issue that column default value did not get converted
3.6.282024-12-2350870Use airbyte/java-connector-base:2.0.0
3.6.272024-12-2350410Use a non root base image.
3.6.262024-12-2048495Increase MAX_FIRST_RECORD_WAIT_TIME and use Debezium 3.0.1
3.6.252024-12-1749838Use a base image: airbyte/java-connector-base:1.0.0
3.6.242024-12-1649469Simplify CTID_TABLE_BLOCK_SIZE query for Postgres integration
3.6.232024-11-1348482Convert large integer typed using NUMERIC(X, 0) into a BigInteger. l
3.6.222024-10-0246900Fixed a bug where source docs won’t render on Airbyte 1.1
3.6.212024-10-0246322Support CDC against a read-replica (continuation)
3.6.202024-10-0146299Make postgres source compile and use the latest CDK
3.6.192024-09-1745639Adopt latest CDK to use the latest apache sshd mina to handle tcpkeepalive requests.
3.6.182024-08-2844878Enable tcpKeepAlive for jdbc connection.
3.6.172024-08-2744841Adopt latest CDK.
3.6.162024-08-1544119Fix incorrect final state on initial read in CDC mode.
3.6.152024-08-1243945Add missing replication slot config error.
3.6.142024-08-0843418Adopt latest CDK.
3.6.132024-07-3042869Adopt latest CDK.
3.6.122024-07-3042550Correctly report stream states.
3.6.112024-07-2942852Bump CDK version to latest to use new bug fixes on error translation.
3.6.102024-07-2342417Handle null error message in ConnectorExceptionHandler.
3.6.92024-07-2342421Remove final transient error emitter iterators.
3.6.82024-07-2241622Bump CDK version to latest.
3.6.72024-07-2242411Hide the “initial load timeout in hours” field by default in UI
3.6.62024-07-2241622Fix bug in CDC syncing
3.6.52024-07-2242024Fix a bug on resuming from a failed attempt.
3.6.42024-07-1742087Translate more errors for Postgres source.
3.6.32024-07-1942122Improve wass error message + logging.
3.6.22024-07-1842108Disable incremental sync for view streams in xmin replication mode
3.6.12024-07-0540716Fix typo in connector specification
3.6.02024-07-1740208Start using the new error Postgres source error handler that comes with a new error translation layer.
3.5.22024-07-1742068Add analytics for WASS case occurrence.
3.5.12024-07-1742055Add debezium heartbeat timeout back to shutdown debezium.
3.5.02024-07-1741651Implement WASS algo - large initial snapshots shouldn’t block CDC.
3.4.262024-07-1541654Allow null value for array typed columns in CDC.
3.4.252024-07-1241651Throw transient error if tables of interest and undergoing full vacuum.
3.4.242024-07-0541067Fix Postgres sending duplicated streams
3.4.232024-07-0140757Rollback 3.4.22.
3.4.222024-07-0540719Fix Postgres sending duplicated streams (rolled back in 3.4.23)
3.4.212024-07-0140516Remove dbz hearbeat.
3.4.202024-06-2340559Remove strict check for stream states of unknown types
3.4.192024-06-2340223Revert the changes introduced in version 3.4.15.
3.4.182024-06-1439349Full refresh stream sending internal count metadata.
3.4.172024-06-1339460Bump postgres JDBC driver version
3.4.162024-05-2939474Adopt latest CDK.
3.4.152024-05-2938773Connect with adaptiveFetch=true.
3.4.142024-06-0839353Upgrade Debezium to 2.6.2
3.4.132024-06-0438875read() throws config exception upon detecting transaction ID wraparound.
3.4.122024-06-0438836check() throws config error upon detecting transaction ID wraparound.
3.4.112024-06-0438848Improve UI message and doc on xmin
3.4.102024-05-2938584Set is_resumable flag in discover.
3.4.92024-05-2938775Publish CDK
3.4.92024-05-2838716Publish CDK
3.4.82024-05-2838716Stream status for postgres
3.4.72024-05-2038365Rollback a previously version (3.4.6)
3.4.62024-05-2038289Connect with adaptiveFetch=true (rolled back in 3.4.7)
3.4.52024-05-1638303Streams not in the CDC publication still have a cursor and PK.
3.4.42024-05-1538208disable counts in full refresh stream in state message.
3.4.32024-05-1338104Handle transient error messages.
3.4.22024-05-1038171Bug fix on final state setup.
3.4.12024-05-1038130Bug fix on old PG where ctid column not found when stream is a view.
3.4.02024-04-2937112resumeable full refresh.
3.3.332024-05-0738030Mark PG hot standby error as transient.
3.3.322024-04-3037758Correct previous release to disable debezium retries
3.3.312024-04-3037754Add CDC logs
3.3.302024-04-3037726Remove debezium retries
3.3.292024-04-2337509remove excessive logs
3.3.282024-04-2337509Better error messages on switching between sync modes.
3.3.272024-04-2237445Remove legacy bad values handling code.
3.3.262024-04-1036982Populate airbyte_meta.changes for xmin path
3.3.252024-04-1036981Track latest CDK
3.3.242024-04-1036865Track latest CDK
3.3.232024-04-0236759Track latest CDK
3.3.222024-04-0136739Fix useLocalCdk flag.
3.3.212024-03-2536584Adopt Kotlin CDK.
3.3.202024-03-2536432Failure to serialize values from Postgres DB shouldn’t fail sync.
3.3.192024-03-1236333Use newest CDK - deprecate dbz iterator
3.3.182024-03-1235599Use newest CDK
3.3.172024-03-1235939Use lsn_commit value instead of lsn_proc for CDC checkpointing logic.
3.3.162024-03-1135904Adopt Java CDK 0.23.1- debezium retries.
3.3.152024-02-2934724Add record count in state message.
3.3.142024-03-0635842Add logging to understand cases with a large number of records with the same LSN.
3.3.132024-02-2735675Fix invalid cdc error message.
3.3.122024-02-2235569Fix logging bug.
3.3.112024-02-2035304Add config to throw an error on invalid CDC position and enable it by default.
3.3.102024-02-1334751Emit analytics message for invalid CDC cursor.
3.3.92024-02-1335224Adopt CDK 0.20.4
3.3.82024-02-0834751Adopt CDK 0.19.0
3.3.72024-02-0834781Add a setting in the setup page to advance the LSN.
3.3.62024-02-0734892Adopt CDK v0.16.6
3.3.52024-02-0734948Adopt CDK v0.16.5
3.3.42024-01-3134723Adopt CDK v0.16.3
3.3.32024-01-2634573Adopt CDK v0.16.0
3.3.22024-01-2434465Check xmin only if user selects xmin sync mode.
3.3.12024-01-1034119Adopt java CDK version 0.11.5.
3.3.02023-12-1933437Remove LEGACY state flag
3.2.272023-12-1833605Advance Postgres LSN for PG 14 & below.
3.2.262023-12-1132961Support for better debugging tools.
3.2.252023-11-2932961Bump debezium wait time default to 20 min.
3.2.242023-11-2832686Better logging to understand dbz closing reason attribution.
3.2.232023-11-2832891Fix CDK dependency in build.
3.2.222023-11-2232656Adopt java CDK version 0.5.0.
3.2.212023-11-0731856handle date/timestamp infinity values properly
3.2.202023-11-0632193Adopt java CDK version 0.4.1.
3.2.192023-11-0332050Adopt java CDK version 0.4.0.
3.2.182023-11-0129038Fix typo (s/Airbtye/Airbyte/)
3.2.172023-11-0132068Bump Debezium 2.2.0Final -> 2.4.0Final
3.2.162023-10-3131976Speed up tests involving Debezium
3.2.152023-10-3031960Adopt java CDK version 0.2.0.
3.2.142023-10-2431792fail sync when debezeum fails to shutdown cleanly
3.2.142023-10-2431792Fix error message link on issue with standby
3.2.132023-10-1631029Enforces encrypted traffic settings when env var DEPLOYMENT_MODE = CLOUD.
3.1.132023-10-1331309Addressed decimals being incorrectly deserialized into scientific notation.
3.1.122023-10-1231328Improvements to initial load of tables in older versions of postgres.
3.1.112023-10-1131322Correct previous release
3.1.102023-09-2930806Cap log line length to 32KB to prevent loss of records.
3.1.92023-09-2530534Fix JSONB[] column type handling bug.
3.1.82023-09-2030125Improve initial load performance for older versions of PostgreSQL.
3.1.72023-09-0529672Handle VACUUM happening during initial sync
3.1.62023-08-2429821Set replication_method display_type to radio, update titles and descriptions, and make CDC the default choice
3.1.52023-08-2229534Support “options” JDBC URL parameter
3.1.42023-08-2128687Under the hood: Add dependency on Java CDK v0.0.2.
3.1.32023-08-0328708Enable checkpointing snapshots in CDC connections
3.1.22023-08-0128954Fix an issue that prevented use of tables with names containing uppercase letters
3.1.12023-07-3128892Fix an issue that prevented use of cursor columns with names containing uppercase letters
3.1.02023-07-2528339Checkpointing initial load for incremental syncs: enabled for xmin and cursor based only.
3.0.22023-07-1828336Add full-refresh mode back to Xmin syncs.
3.0.12023-07-1428345Increment patch to trigger a rebuild
3.0.02023-07-1227442Set _ab_cdc_lsn as the source defined cursor for CDC mode to prepare for Destination v2 normalization
2.1.12023-07-0626723Add new xmin replication method.
2.1.02023-06-2627737License Update: Elv2
2.0.342023-06-2027212Fix silent exception swallowing in StreamingJdbcDatabase
2.0.332023-06-0126873Add prepareThreshold=0 to JDBC url to mitigate PGBouncer prepared statement [X] already exists.
2.0.322023-05-3126810Remove incremental sync estimate from Postgres to increase performance.
2.0.312023-05-2526633Collect and log information related to full vacuum operation in db
2.0.302023-05-2526473CDC : Limit queue size
2.0.292023-05-1825898Translate Numeric values without decimal, e.g: NUMERIC(38,0), as BigInt instead of Double
2.0.282023-04-2725401CDC : Upgrade Debezium to version 2.2.0
2.0.272023-04-2624971Emit stream status updates
2.0.262023-04-2625560Revert some logging changes
2.0.252023-04-2425459Better logging formatting
2.0.242023-04-1925345Logging : Log database indexes per stream
2.0.232023-04-1924582CDC : Enable frequent state emission during incremental syncs + refactor for performance improvement
2.0.222023-04-1725220Logging changes : Log additional metadata & clean up noisy logs
2.0.212023-04-1225131Make Client Certificate and Client Key always show
2.0.202023-04-1124859Removed SSL toggle and rely on SSL mode dropdown to enable/disable SSL
2.0.192023-04-1124656CDC minor refactor
2.0.182023-04-0624820Fix data loss bug during an initial failed non-CDC incremental sync
2.0.172023-04-0524622Allow streams not in CDC publication to be synced in Full-refresh mode
2.0.162023-04-0524895Fix spec for cloud
2.0.152023-04-0424833Fix Debezium retry policy configuration
2.0.142023-04-0324609Disallow the “disable” SSL Modes
2.0.132023-03-2824166Fix InterruptedException bug during Debezium shutdown
2.0.122023-03-2724373Add CDC checkpoint state messages
2.0.112023-03-2324446Set default SSL Mode to require in strict-encrypt
2.0.102023-03-2324417Add field groups and titles to improve display of connector setup form
2.0.92023-03-2220760Removed redundant date-time datatypes formatting
2.0.82023-03-2224255Add field groups and titles to improve display of connector setup form
2.0.72023-03-2124207Fix incorrect schema change warning in CDC mode
2.0.62023-03-2124271Fix NPE in CDC mode
2.0.52023-03-2121533Add integration with datadog
2.0.42023-03-2124275Fix error with CDC checkpointing
2.0.32023-03-1424000Removed check method call on read.
2.0.22023-03-1321727Add state checkpointing for CDC sync.
2.0.02023-03-0623112Upgrade Debezium version to 2.1.2
1.0.512023-03-0223642Revert : Support JSONB datatype for Standard sync mode
1.0.502023-02-2721695Support JSONB datatype for Standard sync mode
1.0.492023-02-2423383Fixed bug with non readable double-quoted values within a database name or column name
1.0.482023-02-2322623Increase max fetch size of JDBC streaming mode
1.0.472023-02-2223138Fix previous versions which doesn’t verify privileges correctly, preventing CDC syncs to run.
1.0.462023-02-2123105Include log levels and location information (class, method and line number) with source connector logs published to Airbyte Platform.
1.0.452023-02-0922371Ensures that user has required privileges for CDC syncs.
2023-02-1523028
1.0.442023-02-0622221Exclude new set of system tables when using pg_stat_statements extension.
1.0.432023-02-0621634Improve Standard sync performance by caching objects.
1.0.422023-01-2321523Check for null in cursor values before replacing.
1.0.412023-01-2520939Adjust batch selection memory limits databases.
1.0.402023-01-2421825Put back the original change that will cause an incremental sync to error if table contains a NULL value in cursor column.
1.0.392023-01-2021683Speed up esmtimates for trace messages in non-CDC mode.
1.0.382023-01-1720346Consolidate date/time values mapping for JDBC sources
1.0.372023-01-1720783Emit estimate trace messages for non-CDC mode.
1.0.362023-01-1121003Handle null values for array data types in CDC mode gracefully.
1.0.352023-01-0420469Introduce feature to make LSN commit behaviour configurable.
1.0.342022-12-1320378Improve descriptions
1.0.332022-12-1218959CDC : Don’t timeout if snapshot is not complete.
1.0.322022-12-1220192Only throw a warning if cursor column contains null values.
1.0.312022-12-0219889Check before each sync and stop if an incremental sync cursor column contains a null value.
2022-12-0219985Reenable incorrectly-disabled wal2json CDC plugin
1.0.302022-11-2919024Skip tables from schema where user do not have Usage permission during discovery.
1.0.292022-11-2919623Mark PSQLException related to using replica that is configured as a hot standby server as config error.
1.0.282022-11-2819514Adjust batch selection memory limits databases.
1.0.272022-11-2816990Handle arrays data types
1.0.262022-11-1819551Fixes bug with ssl modes
1.0.252022-11-1619004Use Debezium heartbeats to improve CDC replication of large databases.
1.0.242022-11-0719291Default timeout is reduced from 1 min to 10sec
1.0.232022-11-0719025Stop enforce SSL if ssl mode is disabled
1.0.222022-10-3118538Encode database name
1.0.212022-10-2518256Disable allow and prefer ssl modes in CDC mode
1.0.202022-10-2518383Better SSH error handling + messages
1.0.192022-10-2118263Fixes bug introduced in 15833 and adds better error messaging for SSH tunnel in Destinations
1.0.182022-10-1918087Better error messaging for configuration errors (SSH configs, choosing an invalid cursor)
1.0.172022-10-1718041Fixes bug introduced 2022-09-12 with SshTunnel, handles iterator exception properly
1.0.162022-10-1316238Update incremental query to avoid data missing when new data is inserted at the same time as a sync starts under non-CDC incremental mode
1.0.152022-10-1117782Handle 24:00:00 value for Time column
1.0.142022-10-0317515Fix an issue preventing connection using client certificate
1.0.132022-10-0117459Upgrade debezium version to 1.9.6 from 1.9.2
1.0.122022-09-2717299Improve error handling for strict-encrypt postgres source
1.0.112022-09-2617131Allow nullable columns to be used as cursor
1.0.102022-09-1415668Wrap logs in AirbyteLogMessage
1.0.92022-09-1316657Improve CDC record queueing performance
1.0.82022-09-0816202Adds error messaging factory to UI
1.0.72022-08-3016114Prevent traffic going on an unsecured channel in strict-encryption version of source postgres
1.0.62022-08-3016138Remove unnecessary logging
1.0.52022-08-2515993Add support for connection over SSL in CDC mode
1.0.42022-08-2315877Fix temporal data type bug which was causing failure in CDC mode
1.0.32022-08-1814356DB Sources: only show a table can sync incrementally if at least one column can be used as a cursor field
1.0.22022-08-1115538Allow additional properties in db stream state
1.0.12022-08-1015496Fix state emission in incremental sync
2022-08-1015481Fix data handling from WAL logs in CDC mode
1.0.02022-08-0515380Change connector label to generally_available (requires upgrading your Airbyte platform to v0.40.0-alpha)
0.4.442022-08-0515342Adjust titles and descriptions in spec.json
0.4.432022-08-0315226Make connectionTimeoutMs configurable through JDBC url parameters
0.4.422022-08-0315273Fix a bug in 0.4.36 and correctly parse the CDC initial record waiting time
0.4.412022-08-0315077Sync data from beginning if the LSN is no longer valid in CDC
2022-08-0314903Emit state messages more frequently (⛔ this version has a bug; use 1.0.1 instead
0.4.402022-08-0315187Add support for BCE dates/timestamps
2022-08-0314534Align regular and CDC integration tests and data mappers
0.4.392022-08-0214801Fix multiple log bindings
0.4.382022-07-2614362Integral columns are now discovered as int64 fields.
0.4.372022-07-2214714Clarified error message when invalid cursor column selected
0.4.362022-07-2114451Make initial CDC waiting time configurable (⛔ this version has a bug and will not work; use 0.4.42 instead)
0.4.352022-07-1414574Removed additionalProperties:false from JDBC source connectors
0.4.342022-07-1713840Added the ability to connect using different SSL modes and SSL certificates.
0.4.332022-07-1414586Validate source JDBC url parameters
0.4.322022-07-0714694Force to produce LEGACY state if the use stream capable feature flag is set to false
0.4.312022-07-0714447Under CDC mode, retrieve only those tables included in the publications
0.4.302022-06-3014251Use more simple and comprehensive query to get selectable tables
0.4.292022-06-2914265Upgrade postgresql JDBC version to 42.3.5
0.4.282022-06-2314077Use the new state management
0.4.262022-06-1713864Updated stacktrace format for any trace message errors
0.4.252022-06-1513823Publish adaptive postgres source that enforces ssl on cloud + Debezium version upgrade to 1.9.2 from 1.4.2
0.4.242022-06-1413549Fixed truncated precision if the value of microseconds or seconds is 0
0.4.232022-06-1313745Fixed handling datetime cursors when upgrading from older versions of the connector
0.4.222022-06-0913655Fixed bug with unsupported date-time datatypes during incremental sync
0.4.212022-06-0613435Adjust JDBC fetch size based on max memory and max row size
0.4.202022-06-0213367Added convertion hstore to json format
0.4.192022-05-2513166Added timezone awareness and handle BC dates
0.4.182022-05-2513083Add support for tsquey type
0.4.172022-05-1913016CDC modify schema to allow null values
0.4.162022-05-1412840Added custom JDBC parameters field
0.4.152022-05-1312834Fix the bug that the connector returns empty catalog for Azure Postgres database
0.4.142022-05-0812689Add table retrieval according to role-based SELECT privilege
0.4.132022-05-0510230Explicitly set null value for field in json
0.4.122022-04-2912480Query tables with adaptive fetch size to optimize JDBC memory consumption
0.4.112022-04-1111729Bump mina-sshd from 2.7.0 to 2.8.0
0.4.102022-04-0811798Fixed roles for fetching materialized view processing
0.4.82022-02-2110242Fixed cursor for old connectors that use non-microsecond format. Now connectors work with both formats
0.4.72022-02-1810242Updated timestamp transformation with microseconds
0.4.62022-02-1410256(unpublished) Add -XX:+ExitOnOutOfMemoryError JVM option
0.4.52022-02-0810173Improved discovering tables in case if user does not have permissions to any table
0.4.42022-01-269807Update connector fields title/description
0.4.32022-01-249554Allow handling of java sql date in CDC
0.4.22022-01-139360Added schema selection
0.4.12022-01-059116Added materialized views processing
0.4.02021-12-138726Support all Postgres types
0.3.172021-12-018371Fixed incorrect handling “\n” in ssh key
0.3.162021-11-287995Fixed money type with amount > 1000
0.3.152021-11-268266Fixed the case, when Views are not listed during schema discovery
0.3.142021-11-178010Added checking of privileges before table internal discovery
0.3.132021-10-267339Support or improve support for Interval, Money, Date, various geometric data types, inventory_items, and others
0.3.122021-09-306585Improved SSH Tunnel key generation steps
0.3.112021-09-025742Add SSH Tunnel support
0.3.92021-08-175304Fix CDC OOM issue
0.3.82021-08-134699Added json config validator
0.3.42021-06-093973Add AIRBYTE_ENTRYPOINT for Kubernetes support
0.3.32021-06-083960Add method field in specification parameters
0.3.22021-05-263179Remove isCDC logging
0.3.12021-04-212878Set defined cursor for CDC
0.3.02021-04-212990Support namespaces
0.2.72021-04-162923SSL spec as optional
0.2.62021-04-162757Support SSL connection
0.2.52021-04-122859CDC bugfix
0.2.42021-04-092548Support CDC
0.2.32021-03-282600Add NCHAR and NVCHAR support to DB and cursor type casting
0.2.22021-03-262460Destination supports destination sync mode
0.2.12021-03-182488Sources support primary keys
0.2.02021-03-092238Protocol allows future/unknown properties
0.1.132021-02-021887Migrate AbstractJdbcSource to use iterators
0.1.122021-01-251746Fix NPE in State Decorator
0.1.112021-01-251765Add field titles to specification
0.1.102021-01-191724Fix JdbcSource handling of tables with same names in different schemas
0.1.92021-01-141655Fix JdbcSource OOM
0.1.82021-01-131588Handle invalid numeric values in JDBC source
0.1.72021-01-081307Migrate Postgres and MySql to use new JdbcSource
0.1.62020-12-091172Support incremental sync
0.1.52020-11-301038Change JDBC sources to discover more than standard schemas
0.1.42020-11-301046Add connectors using an index YAML file
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.