Skip to main content
Skip to main content

CREATE-STORAGE-VAULT

CREATE-STORAGE-VAULT

Description

This command is used to create a storage vault. The subject of this document describes the syntax for creating Doris self-maintained storage vault.

CREATE STORAGE VAULT [IF NOT EXISTS] vault
[properties]

properties

paramis requireddesc
typerequiredOnly two types of vaults are allowed: S3 and HDFS.
S3 Vault
paramis requireddesc
s3.endpointrequiredThe endpoint used for object storage.
Notice, please don't provide the endpoint with any http:// or https://. And for Azure Blob Storage, the endpoint should be like ${ak}.blob.core.windows.net/.
s3.regionrequiredThe region of your bucket.(Not required when you'r using GCP or AZURE).
s3.root.pathrequiredThe path where the data would be stored.
s3.bucketrequiredThe bucket of your object storage account. (StorageAccount if you're using Azure).
s3.access_keyrequiredThe access key of your object storage account. (AccountName if you're using Azure).
s3.secret_keyrequiredThe secret key of your object storage account. (AccountKey if you're using Azure).
providerrequiredThe cloud vendor which provides the object storage service. The supported values include COS, OSS, S3, OBS, BOS, AZURE, GCP
HDFS Vault
paramis requireddesc
fs.defaultFSrequiredHadoop configuration property that specifies the default file system to use.
path_prefixoptionalThe path prefix to where the data would be stored. It would be the root_path of your Hadoop user if you don't provide any prefix.
hadoop.usernameoptionalHadoop configuration property that specifies the user accessing the file system. It would be the user starting Hadoop process if you don't provide any user.
hadoop.security.authenticationoptionalThe authentication way used for hadoop. If you'd like to use kerberos you can provide with kerboros.
hadoop.kerberos.principaloptionalThe path to your kerberos principal.
hadoop.kerberos.keytaboptionalThe path to your kerberos keytab.

Example

  1. create a HDFS storage vault.

    CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault
    PROPERTIES (
    "type"="hdfs",
    "fs.defaultFS"="hdfs://127.0.0.1:8020"
    );
  2. create a S3 storage vault using azure.

    CREATE STORAGE VAULT IF NOT EXISTS s3_vault
    PROPERTIES (
    "type"="S3",
    "s3.endpoint"="ak.blob.core.windows.net/",
    "s3.access_key" = "ak",
    "s3.secret_key" = "sk",
    "s3.root.path" = "ssb_sf1_p2_s3",
    "s3.bucket" = "doris-build-1308700295",
    "provider" = "AZURE"
    );
  3. create a S3 storage vault using OSS.

    CREATE STORAGE VAULT IF NOT EXISTS s3_vault
    PROPERTIES (
    "type"="S3",
    "s3.endpoint"="oss.aliyuncs.com",
    "s3.access_key" = "ak",
    "s3.secret_key" = "sk",
    "s3.region" = "cn-hangzhou",
    "s3.root.path" = "ssb_sf1_p2_s3",
    "s3.bucket" = "doris-build-1308700295",
    "provider" = "OSS"
    );
  4. create a S3 storage vault using COS.

    CREATE STORAGE VAULT IF NOT EXISTS s3_vault
    PROPERTIES (
    "type"="S3",
    "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
    "s3.access_key" = "ak",
    "s3.secret_key" = "sk",
    "s3.region" = "ap-guangzhou",
    "s3.root.path" = "ssb_sf1_p2_s3",
    "s3.bucket" = "doris-build-1308700295",
    "provider" = "COS"
    );
  5. create a S3 storage vault using OBS.

    CREATE STORAGE VAULT IF NOT EXISTS s3_vault
    PROPERTIES (
    "type"="S3",
    "s3.endpoint"="obs.cn-north-4.myhuaweicloud.com",
    "s3.access_key" = "ak",
    "s3.secret_key" = "sk",
    "s3.region" = "cn-north-4",
    "s3.root.path" = "ssb_sf1_p2_s3",
    "s3.bucket" = "doris-build-1308700295",
    "provider" = "OBS"
    );
  6. create a S3 storage vault using AWS.

    CREATE STORAGE VAULT IF NOT EXISTS s3_vault
    PROPERTIES (
    "type"="S3",
    "s3.endpoint"="s3.us-east-1.amazonaws.com",
    "s3.access_key" = "ak",
    "s3.secret_key" = "sk",
    "s3.region" = "us-east-1",
    "s3.root.path" = "ssb_sf1_p2_s3",
    "s3.bucket" = "doris-build-1308700295",
    "provider" = "S3"
    );
  7. create a S3 storage vault using MinIO.

     CREATE STORAGE VAULT IF NOT EXISTS s3_vault
    PROPERTIES (
    "type"="S3",
    "s3.endpoint"="127.0.0.1:9000",
    "s3.access_key" = "ak",
    "s3.secret_key" = "sk",
    "s3.region" = "us-east-1",
    "s3.root.path" = "ssb_sf1_p2_s3",
    "s3.bucket" = "doris-build-1308700295",
    "provider" = "S3"
    );

Keywords

CREATE, STORAGE VAULT