Advanced Analytica Advanced Analytica: IBOM
← All connectors
Reddit logo

Source connector

Reddit

Set up Reddit as a source and move its data into your chosen destination.

Version 0.0.71 community alpha

Reddit

Overview

The Reddit source supports both Full Refresh and Incremental syncs.

A Full Refresh sync copies all configured table rows and columns into a new destination table each time. An Incremental sync copies only resources that have changed in Reddit. The first run uses a Full Refresh sync.

Setup

Step 1: Create a Reddit app

  1. Visit Reddit’s app preferences page and select create another app.
  2. Choose your app type:
    • Choose script to authenticate with your Reddit username and password. This is the simplest option, but doesn’t support accounts with two-factor authentication.
    • Choose web app to authenticate with an OAuth 2.0 refresh token.
  3. Set a redirect URI and save the app. You can use any URL you control, such as http://localhost:8080.
  4. Copy the Client ID shown below the app name. Then copy the Client Secret labelled secret.

Step 2: Choose authentication

Reddit access tokens expire after one hour. Choose either of the first two options so Airbyte refreshes the token before every sync.

Option A: Reddit username and password

Select Reddit username and password (script app) and enter the Client ID and Client Secret from Step 1. Then enter the Reddit username and password for the account that owns the app. Before each sync, Airbyte requests a fresh access token using the OAuth 2.0 password grant.

Option B: OAuth 2.0 refresh token

Select OAuth 2.0 (refresh token) and enter the Client ID, Client Secret and a refresh token.

To get a refresh token:

  1. Complete the Reddit authorization code flow with duration=permanent and your required scopes: identity, read, privatemessages and mysubreddits.
  2. Exchange the returned code at https://www.reddit.com/api/v1/access_token.
  3. Use HTTP Basic auth, with the Client ID as the username and the Client Secret as the password.
  4. Use the body grant_type=authorization_code&code=<code>&redirect_uri=<redirect_uri>.
  5. Copy the refresh_token from the response.

Option C: Access token

Select Access token (expires after one hour) and paste a manually obtained access token. For example, you can use Postman:

  • Request - POST https://www.reddit.com/api/v1/access_token
  • Authorization - Basic Auth - username: <CLIENT_ID>, password: <CLIENT_SECRET>
  • Body - x-www-form-urlencoded - grant_type: password, username: YOUR_REDDIT_USERNAME, password: YOUR_REDDIT_PASSWORD

The response’s access_token is the API Key. It expires after one hour, and Airbyte can’t refresh it. This option therefore supports one-off syncs only. Existing connections using the legacy top-level api_key field migrate to this option automatically.

Records and rate limits

  • The Reddit API has rate limiting of 100 queries per minute (QPM) per OAuth client ID. The connector handles this with exponential backoff and a maximum of 3 retries.
  • Access tokens expire after one hour. Airbyte automatically gets a new token when you use username/password or refresh token authentication. With the access token option, you must generate a new token manually.
  • Reddit requires a descriptive User-Agent. The connector sends airbyte:source-reddit:v1 (+https://docs.airbyte.com/integrations/sources/reddit) with every request.
  • Each Reddit API stream call can fetch up to 1000 records, with subsequent pagination.

Configuration

InputTypeDescriptionDefault Value
credentialsobjectAuthentication. One of: OAuth 2.0 refresh token (client_id, client_secret, refresh_token), Reddit username and password (client_id, client_secret, username, password), or a short-lived access token (api_key).
querystringQuery. Specifies the query for searching in reddits and subredditsairbyte
include_over_18booleanInclude over 18 flag. Includes mature contentfalse
exactbooleanExact. Specifies exact keyword and reduces distractions
limitnumberLimit. Max records per page limit1000
subredditsarraySubreddits. Subreddits for exploration (funny, r/funny and /r/funny are all accepted)[r/funny, r/AskReddit]
start_datestringStart date.

Streams

Stream NamePrimary KeyPaginationSupports Full SyncSupports Incremental
selfnameNo pagination✅❌
searchDefaultPaginator✅❌
subreddit_searchDefaultPaginator✅❌
message_inboxDefaultPaginator✅❌
subreddit_popularDefaultPaginator✅❌
subreddit_exploreDefaultPaginator✅✅

IP allow list

If you use Airbyte Cloud and restrict access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

Changelog

Expand to review
VersionDatePull RequestSubject
0.0.712026-11-1091685Update dependencies
0.0.702026-11-0390052Update dependencies
0.0.692026-10-2089370Update dependencies
0.0.682026-10-1388766Update dependencies
0.0.672026-10-0788257Add refreshable OAuth 2.0 (refresh token / password grant) authentication, send a descriptive User-Agent, fix subreddit_explore to iterate over the configured subreddits and read its created cursor, and harden pagination against non-object data responses
0.0.662026-10-0688007Update dependencies
0.0.652026-09-2987320Update dependencies
0.0.642026-09-2286793Update dependencies
0.0.632026-09-1586199Update dependencies
0.0.622026-09-0885627Update dependencies
0.0.612026-08-1884754Update dependencies
0.0.602026-08-1184090Update dependencies
0.0.592026-08-0483607Update dependencies
0.0.582026-07-2883067Update dependencies
0.0.572026-07-2182583Update dependencies
0.0.562026-07-1481975Update dependencies
0.0.552026-06-3081234Update dependencies
0.0.542026-06-2380628Update dependencies
0.0.532026-06-1680049Update dependencies
0.0.522026-06-0979512Update dependencies
0.0.512026-06-0278919Update dependencies
0.0.502026-04-2877395Update dependencies
0.0.492026-04-2176717Update dependencies
0.0.482026-03-3175878Update dependencies
0.0.472026-03-1774911Update dependencies
0.0.462026-02-2473532Update dependencies
0.0.452026-02-0372651Update dependencies
0.0.442026-01-2072024Update dependencies
0.0.432026-01-1471479Update dependencies
0.0.422025-12-1870617Update dependencies
0.0.412025-11-2570060Update dependencies
0.0.402025-11-1869624Update dependencies
0.0.392025-10-2968912Update dependencies
0.0.382025-10-2168335Update dependencies
0.0.372025-10-1467872Update dependencies
0.0.362025-10-0867541Update dependencies
0.0.352025-09-3066443Update dependencies
0.0.342025-09-0965716Update dependencies
0.0.332025-08-2465446Update dependencies
0.0.322025-08-1064831Update dependencies
0.0.312025-08-0264457Update dependencies
0.0.302025-07-1963628Update dependencies
0.0.292025-07-1263054Update dependencies
0.0.282025-07-0562697Update dependencies
0.0.272025-06-2862217Update dependencies
0.0.262025-06-2161787Update dependencies
0.0.252025-06-1460557Update dependencies
0.0.242025-05-1060162Update dependencies
0.0.232025-05-0359473Update dependencies
0.0.222025-04-2759106Update dependencies
0.0.212025-04-1958458Update dependencies
0.0.202025-04-1257923Update dependencies
0.0.192025-04-0557298Update dependencies
0.0.182025-03-2956770Update dependencies
0.0.172025-03-2256163Update dependencies
0.0.162025-03-0855061Update dependencies
0.0.152025-02-2354586Update dependencies
0.0.142025-02-1553957Update dependencies
0.0.132025-02-0853455Update dependencies
0.0.122025-02-0153004Update dependencies
0.0.112025-01-2552494Update dependencies
0.0.102025-01-1851854Update dependencies
0.0.92025-01-1151376Update dependencies
0.0.82024-12-2850683Update dependencies
0.0.72024-12-2150232Update dependencies
0.0.62024-12-1449697Update dependencies
0.0.52024-12-1249368Update dependencies
0.0.42024-12-1149104Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
0.0.32024-10-2947827Update dependencies
0.0.22024-10-2847542Update dependencies
0.0.12024-08-2344579Initial release by btkcodedev via Connector Builder
Advanced Analytica

Most businesses are sitting on information they can’t use at speed. The people who hold it are busy, and when they leave, the knowledge goes with them. The documents are static, and the processes around them weren’t designed for AI.

Advanced Analytica works with these businesses to turn that information into structured data assets. Once they're integrated, AI systems can govern it, and the business can use it. We bring in the strategy, the specification and the AI skills to rework how the business operates.

MOVE FAST. STAY SAFE.