S3 ingestion overview

Doris uses Streaming Job + S3 TVF to ingest S3 files into an existing Doris table in batches. Choose ordered, notification, or one-time ingestion based on how files arrive and whether the job should continue running.

Choose an ingestion mode

Guide	Mode	When to use it	Completion
Ordered ingestion	ORDERED_LIST (default)	Keys increase lexicographically, such as timestamps or zero-padded counters	Runs continuously and waits for new files
Notification ingestion	NOTIFICATION	UUID keys or files arriving in any order, discovered through AWS S3 events and a dedicated SQS queue	Runs continuously and waits for messages
One-time ingestion	ONE_TIME	A prepared static file set or isolated backfill	Enters FINISHED after the final batch commits

Notification ingestion does not backfill historical objects, and repeated notifications can cause duplicate ingestion. All modes require source objects to remain readable and immutable during ingestion. Each guide above includes setup, SQL examples, parameters, and limitations.

Related documents

• Amazon S3 ingestion: S3 Load and TVF + INSERT.
• Continuous Load Overview: job and task inspection, pause, resume, and deletion.
• CREATE STREAMING JOB: job creation syntax.
• ALTER JOB: job modification syntax.
• S3 TVF: connection, credentials, and file format options.