S3

This document describes the parameters required for accessing AWS S3. These parameters apply to:

• Catalog properties
• Table Valued Function properties
• Broker Load properties
• Export properties
• Outfile properties

Parameter Overview

Property Name	Former Name	Description	Default Value	Required
s3.endpoint	S3 endpoint	S3 endpoint		Yes
s3.region	S3 region	S3 region		No
s3.access_key	S3 access key	S3 access key		Yes
s3.secret_key	S3 secret key	S3 secret key		Yes
s3.use_path_style	use_path_style	Whether to use path-style access to S3. Used when accessing certain S3-compatible object storage that doesn't support host-style	false	No
s3.connection.maximum		Maximum S3 connections	50	No
s3.connection.request.timeout		S3 request timeout in milliseconds	3000	No
s3.connection.timeout		S3 connection timeout in milliseconds	1000	No

Authentication Configuration

When accessing AWS S3, you need to provide AWS Access Key and AWS Secret Key, which are the following parameters:

• s3.access_key
• s3.secret_key

When accessing AWS S3, Doris also supports the Assumed Role method. Please refer to the following documentation for configuration: AWS integration.

Configuration Example

"s3.access_key" = "ak"
"s3.secret_key" = "sk"
"s3.endpoint" = "s3.us-east-1.amazonaws.com"
"s3.region" = "us-east-1"

Directory Bucket

Directory Bucket is a high-performance storage option for Amazon S3 Express One Zone storage class. When using Directory Bucket, the main difference is in the configuration format of s3.endpoint, while other configuration parameters remain unchanged.

Endpoint Configuration Differences

• General S3 Bucket

  "s3.endpoint" = "s3.us-east-1.amazonaws.com"

• Directory Bucket

  "s3.endpoint" = "s3express-{zone-id}.{region}.amazonaws.com"

Directory Bucket Configuration Example

"s3.access_key" = "ak"
"s3.secret_key" = "sk"
"s3.endpoint" = "s3express-usw2-az1.us-west-2.amazonaws.com"
"s3.region" = "us-west-2"

Where s3express-usw2-az1.us-west-2.amazonaws.com is the dedicated endpoint format for Directory Bucket. For more endpoint addresses, please refer to the AWS official documentation.