HUDI_META

Description

hudi_meta table-valued-function(tvf), using for read hudi metadata, operation history, timeline of table, instant state etc.

Syntax

HUDI_META(
    "table" = "<table>", 
    "query_type" = "<query_type>"
  );

Required Parameters

Each parameter in the hudi_meta table function (tvf) is a "key"="value" pair.

Field	Description
<table>	The full table name, which must be specified in the format of database_name.table_name for the hudi table that you want to view.
<query_type>	The type of metadata you want to view. Currently, only timeline is supported.

Examples

• Read and access the hudi tabular metadata for timeline.

  select * from hudi_meta("table" = "ctl.db.tbl", "query_type" = "timeline");

• Can be used with desc function :

  desc function hudi_meta("table" = "ctl.db.tbl", "query_type" = "timeline");

• Inspect the hudi table timeline

  select * from hudi_meta("table" = "hudi_ctl.test_db.test_tbl", "query_type" = "timeline");

  +-------------------+--------+--------------------------+-----------+-----------------------+
  | timestamp         | action | file_name                | state     | state_transition_time |
  +-------------------+--------+--------------------------+-----------+-----------------------+
  | 20240724195843565 | commit | 20240724195843565.commit | COMPLETED | 20240724195844269     |
  | 20240724195845718 | commit | 20240724195845718.commit | COMPLETED | 20240724195846653     |
  | 20240724195848377 | commit | 20240724195848377.commit | COMPLETED | 20240724195849337     |
  | 20240724195850799 | commit | 20240724195850799.commit | COMPLETED | 20240724195851676     |
  +-------------------+--------+--------------------------+-----------+-----------------------+

• Filtered by timestamp

  select * from hudi_meta("table" = "hudi_ctl.test_db.test_tbl", "query_type" = "timeline") where timestamp = 20240724195843565;

  +-------------------+--------+--------------------------+-----------+-----------------------+
  | timestamp         | action | file_name                | state     | state_transition_time |
  +-------------------+--------+--------------------------+-----------+-----------------------+
  | 20240724195843565 | commit | 20240724195843565.commit | COMPLETED | 20240724195844269     |
  +-------------------+--------+--------------------------+-----------+-----------------------+