Last updated on
Apache Ozone
Starting from version 4.0.4, Doris supports accessing Apache Ozone through the S3 Gateway.
This document describes the parameters required to access Apache Ozone. These parameters apply to:
- Catalog properties
- Table Valued Function properties
- Broker Load properties
- Export properties
- Outfile properties
To use Ozone as a dedicated storage type, configure "fs.ozone.support" = "true" explicitly.
Parameter Overview
| Property Name | Legacy Name | Description | Default Value | Required |
|---|---|---|---|---|
| ozone.endpoint | s3.endpoint | Ozone S3 Gateway endpoint, for example http://ozone-s3g:9878 | None | Yes |
| ozone.region | s3.region | Region of Ozone S3 Gateway | us-east-1 | No |
| ozone.access_key | s3.access_key, s3.access-key-id | Access key used for authentication | None | No* |
| ozone.secret_key | s3.secret_key, s3.secret-access-key | Secret key used for authentication | None | No* |
| ozone.session_token | s3.session_token, s3.session-token | Session token | None | No |
| ozone.connection.maximum | s3.connection.maximum | Maximum number of connections | 100 | No |
| ozone.connection.request.timeout | s3.connection.request.timeout | Request timeout in milliseconds | 10000 | No |
| ozone.connection.timeout | s3.connection.timeout | Connection timeout in milliseconds | 10000 | No |
| ozone.use_path_style | use_path_style, s3.path-style-access | Whether to use path-style access | true | No |
| ozone.force_parsing_by_standard_uri | force_parsing_by_standard_uri | Whether to force standard URI parsing | false | No |
| fs.ozone.support | Whether to enable Ozone as storage type | false | Yes |
Notes:
ozone.access_keyandozone.secret_keymust be configured together.fs.s3a.*keys are not parsed directly by Ozone properties. Useozone.*or compatibles3.*keys.- Ozone supports
s3://,s3a://, ands3n://URI schemas. - Setting
ozone.endpoint(ors3.endpoint) alone does not enable Ozone. You must setfs.ozone.support=true.
Example Configuration
Using ozone.* keys:
"fs.ozone.support" = "true",
"ozone.endpoint" = "http://ozone-s3g:9878",
"ozone.access_key" = "hadoop",
"ozone.secret_key" = "hadoop",
"ozone.region" = "us-east-1",
"ozone.use_path_style" = "true"
Using compatible s3.* aliases:
"fs.ozone.support" = "true",
"s3.endpoint" = "http://ozone-s3g:9878",
"s3.access_key" = "hadoop",
"s3.secret_key" = "hadoop",
"s3.region" = "us-east-1",
"s3.path-style-access" = "true"