Advanced Analytica Advanced Analytica: IBOM
← All connectors
End-to-End Testing (Mock API) logo

Source connector

End-to-End Testing (Mock API)

Set up End-to-End Testing (Mock API) as a source and move its data into your chosen destination.

Version 2.2.3 community alpha

import {SchemaGenerator} from ’./e2e-test.js’

End-to-End Testing Source

Overview

Use this mock source to test the Airbyte pipeline and generate arbitrary data streams.

Mode

Continuous Feed

This is the only mode available starting from 2.0.0.

Specify a single-stream or multi-stream catalog with any schema. Your schema should comply with Json schema draft-07.

Use the single-stream catalog config when one stream is enough for your test. A multi-stream catalog config with one stream is equivalent to a single-stream catalog config.

Configure it as follows:

Mock Catalog TypeParametersTypeRequiredDefaultNotes
Single-streamstream namestringyesName of the stream in the catalog.
stream schemajsonyesJson schema of the stream in the catalog. It must be a valid Json schema.
stream duplicationintegerno1Duplicate the stream N times to quickly create a multi-stream catalog.
Multi-streamstreams and schemasjsonyesA Json object specifying multiple data streams and their schemas. Each key in this object is one stream name. Each value is the schema for that stream.
Bothmax recordsintegeryes100The number of record messages to emit from this connector. Min 1. Max 100 billion.
random seedintegernocurrent time millisThe seed is used in random Json object generation. Min 0. Max 1 million.
message intervalintegerno0The time interval between messages in millisecond. Min 0 ms. Max 60000 ms (1 minute).

Example Stream Schemas

To simulate a wide table in performance testing, use the example 500 column stream. Alternatively, use the form below to set any width. Then copy and paste the resulting schema into your configuration.

Legacy Infinite Feed

Airbyte integration tests used this legacy mode. It has been removed since 2.0.0. Its simple catalog contains one data stream with this schema:

{
  "type": "object",
  "properties": {
    "column1": { "type": "string" }
  }
}

The value of column1 increases from 1.

This mode can generate an infinite number of records, which can be dangerous. The Cloud variant therefore excludes it. You usually shouldn’t use this mode.

You can configure two parameters:

ParametersTypeRequiredDefaultNotes
max recordsintegernonullNumber of message records to emit. When it is left empty, the connector will generate infinite number of messages.
message intervalintegernonullTime interval between messages in millisecond.

Exception after N

Airbyte integration tests used this legacy mode. It has been removed since 2.0.0. It throws an IllegalStateException after a specified number of messages. Its only parameter sets how many messages it emits before the exception.

The Cloud variant also excludes this mode.

IP allow list

If you use Airbyte Cloud and restrict access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review

The OSS and Cloud variants have the same version number. The Cloud variant was initially released at version 1.0.0.

VersionDatePull requestSubject
2.2.32025-07-1062917Convert to new gradle build flow
2.2.32024-05-0959761Update to new gradle flow.
2.2.22024-04-2537581bump jsonschemafriend to 0.12.4
2.2.12024-02-1335231Adopt JDK 0.20.4.
2.2.02023-12-1833485Remove LEGACY state
2.1.52023-10-0431092Bump jsonschemafriend dependency version to fix bug
2.1.42023-03-0123656Add speed benchmark mode to e2e test
2.1.32022-08-2515591Declare supported sync modes in catalogs
2.1.12022-06-1713864Updated stacktrace format for any trace message errors
2.1.02021-02-12#10298Support stream duplication to quickly create a multi-stream catalog.
2.0.02021-02-01#9954Remove legacy modes. Use more efficient Json generator.
1.0.12021-01-29#9745Integrate with Sentry.
1.0.02021-01-23#9720Add new continuous feed mode that supports arbitrary catalog specification. Initial release to cloud.
0.1.22022-10-18#18100Set supported sync mode on streams
0.1.12021-12-16#8217Fix sleep time in infinite feed mode.
0.1.02021-07-23#3290 #4939Initial release.
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.