Blog/Release Notes

Apache Doris announced the official release of version 2.0.2

Apache DorisOctober 13, 2023

Thanks to our community users and developers, 489 improvements and bug fixes have been made in Doris 2.0.2.

Behavior Changes

• Remove json -> operator convert to json_extract #24679

  Remove json '->' operator since it is conflicted with lambda function syntax. It's a syntax sugar for function json_extract and can be replaced with the former.

• Start the script to set metadata_failure_recovery #24308

  Move metadata_failure_recovery from fe.conf to start_fe.sh argument to prevent being used unexpectedly.

• Change ordinary type null value is \N,complex type null value is null #24207

• Optimize priority_ network matching logic for be #23795

• Fix cancel load failed because Job could not be cancelled… #17730

  Allow cancel a retrying load job.

Improvements

Easier to use

• Support custom lib dir to save custom libs #23887

  Add a custom_lib dir to allow users place custom lib files and custom_lib will not be replaced.

• Optimize priority_ network matching logic #23784

  Optimize priority_network logic to avoid error when this config is wrong or not configured.

• Row policy support role #23022

  Support role based auth for row policy.

New optimizer Nereids statistics collection improvement

• Disable file cache while running analysis tasks. #23663
• Show column stats even when error occurred. #23703
• Support basic jdbc external table stats collection. #23965
• Skip unknown col stats check on __internal_scheam and information_schema #24625

Better support for JDBC, HDFS, Hive, MySQL, MaxCompute, Multi-Catalog

• Support hadoop viewfs. #24168

• Avoid calling checksum when replaying creating jdbc catalog and fix ranger issue #22369

• Optimize the JDBC Catalog connection error message #23868

  Improve property check and error message for JDBC catalog

• Fix mc decimal type parse, fix wrong obj location #24242

  Fix some issues for MaxCompute catalog

• Support sql cache for hms catalog #23391

  SQL cache for Hive catalog

• Merge hms partition events. #22869

  Improve performance for Hive metadata sync

• Add metadata_name_ids for quickly get catlogs,db,table and add profiling table in order to Compatible with mysql #22702

Performance for inverted index query

• Add bkd index query cache to improve perf #23952
• Improve performance for count on index other than match #24678
• Improve match performance without index #24751
• Optimize multiple terms conjunction query #23871 Improve performance of MATCH_ALL
• Optimize unnecessary conversions #24389 Improve performance of MATCH

Improve Array functions

• [Fix old optimizer with some array literal functions #23630
• Improve array union support multi params #24327
• Improve explode func with array nested complex type #24455

Important Bug fixes

• The parameter positions of timestamp diff function to sql are reversed #23601
• Fix old optimizer with some array literal functions #23630
• Fix query cache returns wrong result after deleting partitions. #23555
• Fix potential data loss when clone task's dst tablet is cooldown replica #17644
• Fix array map batch append data with right next_array_item_rowid #23779
• Fix or to in rule #23940
• Fix 'char' function's toSql implementation is wrong #23860
• Record wrong best plan properties #23973
• Make TVF's distribution spec always be RANDOM #24020
• External scan use STORAGE_ANY instead of ANY as distibution #24039
• Runtimefilter target is not SlotReference #23958
• mv in select materialized_view should disable show table #24104
• Fail over to remote file reader if local cache failed #24097
• Fix revoke role operation cause fe down #23852
• Handle status code correctly and add a new error code ENTRY_NOT_FOUND #24139
• Fix leaky abstraction and shield the status code END_OF_FILE from upper layers #24165
• Fix bug that Read garbled files caused be crash. #24164
• Fix be core when user sepcified empty column_separator using hdfs tvf #24369
• Fix need to restart BE after replacing the jar package in java-udf #24372
• Need to call 'set_version' in nested functions #24381
• windown_funnel compatibility issue with multi backends #24385
• correlated anti join shouldn't be translated to null aware anti join #24290
• Change ordinary type null value is \N,complex type null value is null #24207
• Fix analyze failed when there are thousands of partitions. #24521
• Do not use enum as the data type for JavaUdfDataType. #24460
• Fix multi window projection issue temporarily #24568
• Make metadata compatible with 2.0.3 #24610
• Select outfile column order is wrong #24595
• Incorrect result of semi/anti mark join #24616
• Fix broker read issue #24635
• Skip unknown col stats check on __internal_scheam and information_schema #24625
• Fixed bug when parsing multi-character delimiters. #24572
• Fix timezone parse when there is no tzfile #24578
• We need to issue an error when starting FE without setting the Java home environment #23943
• Enable_unique_key_partial_update should be forwarded to master #24697
• Fix paimon file catalog meta issue and replication num analysis issue #24681
• Add more log for ingest_binlog && Fix ingest_binlog not rewrite rowset_meta tablet_uid #24617
• Do not abort when a disk is broken #24692
• colocate join could not work well on full outer join #24700
• Optimize unnecessary conversions #24389
• Optimize the reading efficiency of nullable (string) columns. #24698
• Fix segment cache core when output rowset is nullptr #24778
• Fix duplicate key in schema change #24782
• Make metadata compatible for future version after 2.0.2 #24800
• Fix map/array deserialize string with quote pair #24808
• Failed on arm platform, with clang compiler and pch on, close #24633 #24636
• Table column order is changed if add a column and do truncate #24981
• Make parser mode coarse grained by default #24949

See the complete list of improvements and bug fixes on github .

Big Thanks

Thanks all who contribute to this release:

@adonis0147 @airborne12 @amorynan @AshinGau @BePPPower @BiteTheDDDDt @bobhan1 @ByteYue @caiconghui @CalvinKirs @cambyzju @ChengDaqi2023 @ChinaYiGuan @CodeCooker17 @csun5285 @dataroaring @deadlinefen @DongLiang-0 @Doris-Extras @dutyu @eldenmoon @englefly @freemandealer @Gabriel39 @gnehil @GoGoWen @gohalo @HappenLee @hello-stephen @HHoflittlefish777 @hubgeter @hust-hhb @ixzc @JackDrogon @jacktengg @jackwener @Jibing-Li @JNSimba @kaijchen @kaka11chen @Kikyou1997 @Lchangliang @LemonLiTree @liaoxin01 @LiBinfeng-01 @liugddx @luwei16 @mongo360 @morningman @morrySnow @mrhhsg @Mryange @mymeiyi @neuyilan @pingchunzhang @platoneko @qidaye @realize096 @RYH61 @shuke987 @sohardforaname @starocean999 @SWJTU-ZhangLei @TangSiyang2001 @Tech-Circle-48 @w41ter @wangbo @wsjz @wuwenchi @wyx123654 @xiaokang @XieJiann @xinyiZzz @XuJianxu @xutaoustc @xy720 @xyfsjq @xzj7019 @yiguolei @yujun777 @Yukang-Lian @Yulei-Yang @zclllyybb @zddr @zhangguoqiang666 @zhangstar333 @ZhangYu0123 @zhannngchen @zxealous @zy-kkk @zzzxl1993 @zzzzzzzs