CREATE-STORAGE-VAULT
CREATE-STORAGE-VAULT
Description
This command is used to create a storage vault. The subject of this document describes the syntax for creating Doris self-maintained storage vault.
CREATE STORAGE VAULT [IF NOT EXISTS] vault
[properties]
properties
param | is required | desc |
---|---|---|
type | required | Only two types of vaults are allowed: S3 and HDFS . |
S3 Vault
param | is required | desc |
---|---|---|
s3.endpoint | required | The endpoint used for object storage. Notice, please don't provide the endpoint with any http:// or https:// . And for Azure Blob Storage, the endpoint should be like ${ak}.blob.core.windows.net/ . |
s3.region | required | The region of your bucket.(Not required when you'r using GCP or AZURE). |
s3.root.path | required | The path where the data would be stored. |
s3.bucket | required | The bucket of your object storage account. (StorageAccount if you're using Azure). |
s3.access_key | required | The access key of your object storage account. (AccountName if you're using Azure). |
s3.secret_key | required | The secret key of your object storage account. (AccountKey if you're using Azure). |
provider | required | The cloud vendor which provides the object storage service. The supported values include COS , OSS , S3 , OBS , BOS , AZURE , GCP |
HDFS Vault
param | is required | desc |
---|---|---|
fs.defaultFS | required | Hadoop configuration property that specifies the default file system to use. |
path_prefix | optional | The path prefix to where the data would be stored. It would be the root_path of your Hadoop user if you don't provide any prefix. |
hadoop.username | optional | Hadoop configuration property that specifies the user accessing the file system. It would be the user starting Hadoop process if you don't provide any user. |
hadoop.security.authentication | optional | The authentication way used for hadoop. If you'd like to use kerberos you can provide with kerboros . |
hadoop.kerberos.principal | optional | The path to your kerberos principal. |
hadoop.kerberos.keytab | optional | The path to your kerberos keytab. |
Example
create a HDFS storage vault.
CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault
PROPERTIES (
"type"="hdfs",
"fs.defaultFS"="hdfs://127.0.0.1:8020"
);create a S3 storage vault using azure.
CREATE STORAGE VAULT IF NOT EXISTS s3_vault
PROPERTIES (
"type"="S3",
"s3.endpoint"="ak.blob.core.windows.net/",
"s3.access_key" = "ak",
"s3.secret_key" = "sk",
"s3.root.path" = "ssb_sf1_p2_s3",
"s3.bucket" = "doris-build-1308700295",
"provider" = "AZURE"
);create a S3 storage vault using OSS.
CREATE STORAGE VAULT IF NOT EXISTS s3_vault
PROPERTIES (
"type"="S3",
"s3.endpoint"="oss.aliyuncs.com",
"s3.access_key" = "ak",
"s3.secret_key" = "sk",
"s3.region" = "cn-hangzhou",
"s3.root.path" = "ssb_sf1_p2_s3",
"s3.bucket" = "doris-build-1308700295",
"provider" = "OSS"
);create a S3 storage vault using COS.
CREATE STORAGE VAULT IF NOT EXISTS s3_vault
PROPERTIES (
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "ak",
"s3.secret_key" = "sk",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "ssb_sf1_p2_s3",
"s3.bucket" = "doris-build-1308700295",
"provider" = "COS"
);create a S3 storage vault using OBS.
CREATE STORAGE VAULT IF NOT EXISTS s3_vault
PROPERTIES (
"type"="S3",
"s3.endpoint"="obs.cn-north-4.myhuaweicloud.com",
"s3.access_key" = "ak",
"s3.secret_key" = "sk",
"s3.region" = "cn-north-4",
"s3.root.path" = "ssb_sf1_p2_s3",
"s3.bucket" = "doris-build-1308700295",
"provider" = "OBS"
);create a S3 storage vault using AWS.
CREATE STORAGE VAULT IF NOT EXISTS s3_vault
PROPERTIES (
"type"="S3",
"s3.endpoint"="s3.us-east-1.amazonaws.com",
"s3.access_key" = "ak",
"s3.secret_key" = "sk",
"s3.region" = "us-east-1",
"s3.root.path" = "ssb_sf1_p2_s3",
"s3.bucket" = "doris-build-1308700295",
"provider" = "S3"
);create a S3 storage vault using MinIO.
CREATE STORAGE VAULT IF NOT EXISTS s3_vault
PROPERTIES (
"type"="S3",
"s3.endpoint"="127.0.0.1:9000",
"s3.access_key" = "ak",
"s3.secret_key" = "sk",
"s3.region" = "us-east-1",
"s3.root.path" = "ssb_sf1_p2_s3",
"s3.bucket" = "doris-build-1308700295",
"provider" = "S3"
);
Keywords
CREATE, STORAGE VAULT