Advanced Analytica Advanced Analytica: IBOM
← All connectors
E2E Testing logo

Source connector

E2E Testing

Set up E2E Testing as a source and move its data into your chosen destination.

Version 2.2.2 community alpha

import {SchemaGenerator} from ’./e2e-test.js’

End-to-end testing source

Overview

Use this mock source to test the Airbyte pipeline and generate arbitrary data streams.

Modes

Continuous feed

This is the only mode available starting from 2.0.0.

Specify a single-stream or multi-stream catalog with any schema. The schema must comply with Json schema draft-07.

Use the single-stream catalog config when one stream is enough for your test. It’s provided for convenience.

A multi-stream catalog config with one stream is equivalent to a single-stream catalog config.

Use this configuration:

Mock Catalog TypeParametersTypeRequiredDefaultNotes
Single-streamstream namestringyesName of the stream in the catalog.
stream schemajsonyesJson schema of the stream in the catalog. It must be a valid Json schema.
stream duplicationintegerno1Duplicate the stream N times to quickly create a multi-stream catalog.
Multi-streamstreams and schemasjsonyesA Json object specifying multiple data streams and their schemas. Each key in this object is one stream name. Each value is the schema for that stream.
Bothmax recordsintegeryes100The number of record messages to emit from this connector. Min 1. Max 100 billion.
random seedintegernocurrent time millisThe seed is used in random Json object generation. Min 0. Max 1 million.
message intervalintegerno0The time interval between messages in millisecond. Min 0 ms. Max 60000 ms (1 minute).

Example stream schemas

To simulate a wide table in performance tests, use the example 500 column stream.

Alternatively, use the form below to generate a schema with any width. Then copy and paste it into your configuration.

Legacy infinite feed

Airbyte integration tests used this legacy mode. It was removed in 2.0.0.

It has a simple catalog containing one data stream with this schema:

{
  "type": "object",
  "properties": {
    "column1": { "type": "string" }
  }
}

The value of column1 increases from 1.

This mode can generate an infinite number of records, which can be dangerous. It’s therefore excluded from this connector’s Cloud variant.

You shouldn’t usually use this mode.

You can configure two parameters:

ParametersTypeRequiredDefaultNotes
max recordsintegernonullNumber of message records to emit. When it is left empty, the connector will generate infinite number of messages.
message intervalintegernonullTime interval between messages in millisecond.

Exception after N

Airbyte integration tests used this legacy mode. It was removed in 2.0.0.

It throws an IllegalStateException after a specified number of messages. Its only parameter sets how many messages to emit before the exception.

This mode is also excluded from this connector’s Cloud variant.

IP allow list

If you use Airbyte Cloud and your organisation restricts access by IP, add the Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review

The OSS and Cloud variants have the same version number. The Cloud variant was initially released at version 1.0.0.

VersionDatePull requestSubject
2.2.32025-07-1062917Convert to new gradle build flow
2.2.32024-05-0959761Update to new gradle flow.
2.2.22024-04-2537581bump jsonschemafriend to 0.12.4
2.2.12024-02-1335231Adopt JDK 0.20.4.
2.2.02023-12-1833485Remove LEGACY state
2.1.52023-10-0431092Bump jsonschemafriend dependency version to fix bug
2.1.42023-03-0123656Add speed benchmark mode to e2e test
2.1.32022-08-2515591Declare supported sync modes in catalogs
2.1.12022-06-1713864Updated stacktrace format for any trace message errors
2.1.02021-02-12#10298Support stream duplication to quickly create a multi-stream catalog.
2.0.02021-02-01#9954Remove legacy modes. Use more efficient Json generator.
1.0.12021-01-29#9745Integrate with Sentry.
1.0.02021-01-23#9720Add new continuous feed mode that supports arbitrary catalog specification. Initial release to cloud.
0.1.22022-10-18#18100Set supported sync mode on streams
0.1.12021-12-16#8217Fix sleep time in infinite feed mode.
0.1.02021-07-23#3290 #4939Initial release.
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.