Skip to main content

Databend Releases

This page provides information about recent features, enhancements, and bug fixes for Databend.

Jun 16, 2024 (v1.2.530)

What's Changed

Exciting New Features ✨

  • feat(query): implement ST_XMAX by @kkk25641463 in #15406
  • feat(executor): record query profiling for standalone error query by @zhang2014 in #15425
  • feat(query): add external_server_request_batch_rows setting by @sundy-li in #15440
  • feat(function): Implement map_size function by @forsaken628 in #15444
  • feat(query): add created_time column to processes table by @TCeason in #15448
  • feat(function): Implement map_cat function by @shamb0 in #15348
  • feat: support adjust query's priority by @dqhl76 in #15352
  • feat(executor): record errors info into profiling log by @zhang2014 in #15472
  • feat: optimize vacuum drop table, execute in parallel by @lichuang in #15478
  • feat: http handler add endpoint /v1/login/. by @youngsofun in #15483
  • feat(functions): add map_contains_key scalar function by @hanxuanliang in #15465
  • feat(query): build image limit python 3.12.2 by @sundy-li in #15501
  • feat(query): ensure git is installed in pyenv by @sundy-li in #15505
  • feat(query): pyenv 3.12.2 by @sundy-li in #15512
  • feat(query): support unset session setting by @TCeason in #15510
  • feat: add version metrics for meta & query by @everpcpc in #15522
  • feat: support explain and profile multi table insert statemennt by @SkyFan2002 in #15518
  • feat(query): add query query_hash and query_parameterized_hash to query log by @TCeason in #15531
  • feat: add meta key_num metrics by @lichuang in #15536
  • feat(query): bump arrow-udf version to enhance types supported by JS UDFs by @maxjustus in #15529
  • feat: resolve conflict when multi table insert commit by @SkyFan2002 in #15503
  • feat(query): udf script support load from stage location by @sundy-li in #15542
  • feat(metasrv): add metrics for snapshot key numbers by @everpcpc in #15545
  • feat: select support consume stream by @zhyass in #15541
  • feat(query): show users support display roles by @TCeason in #15544
  • feat: remove XML input format. by @youngsofun in #15540
  • feat(query): create table support add inverted index by @b41sh in #15547
  • feat: fuzzy table data disk cache key reload by @dantengsky in #15566
  • feat(query): variant type as map value support bloom filter by @b41sh in #15562
  • feat(metasrv): add metrics for db_size by @everpcpc in #15595
  • feat: support millisecond level task scheduling by @ZhiHanZ in #15599
  • feat(query): show grants support where/limit option by @TCeason in #15607
  • feat: otlp logging support http endpoint by @everpcpc in #15636
  • feat(planner): support ExpressionScan and CacheScan by @Dousir9 in #15561
  • feat(query): implement ST_YMAX/ST_XMIN/ST_YMIN by @kkk25641463 in #15594
  • feat(query): add python udf script runtime by @sundy-li in #15494
  • feat: compact and recluster add table lock by @zhyass in #15632
  • feat(query): support disable vacuum temporary files after query by @zhang2014 in #15673
  • feat(query): Role/UserInfo add field create_on and udpate_on by @TCeason in #15661
  • feat: unload parquet with version as created_by. by @youngsofun in #15680
  • feat(metrics): add process collector by @flaneur2020 in #15591
  • feat: add meta-service config sled_max_cache_size_mb by @drmingdrmer in #15709
  • feat(query): add async retry transform by @sundy-li in #15647
  • feat: basic support input file format ORC. by @youngsofun in #15720
  • feat: recluster unclustered blocks by @zhyass in #15623
  • feat: add task_history function by @ZhiHanZ in #15731
  • feat(query): show indexes add inverted indexes by @b41sh in #15733
  • feat: Bump to opendal v0.47.0 rc by @Xuanwo in #15745
  • feat: Add http config for all storage services by @Xuanwo in #15757
  • feat(query): add inverted index size to block meta by @b41sh in #15752
  • feat: add meta verifier binary by @lichuang in #15756
  • feat(query): add shuffle before window plan by @sundy-li in #15726
  • feat: parquet support option null_if by @youngsofun in #15776
  • feat: rm CREDENTIALS as alias of CONNECTION. by @youngsofun in #15780
  • feat(function): Implement map_delete function by @shamb0 in #15480
  • feat(query): support try_to_timestmap(string, string), try_to_date(string, string) by @TCeason in #15775
  • feat: add io delay chaos meta test scripts by @lichuang in #15769
  • feat: support recursive cte by @xudong963 in #15675
  • feat(ci): add logging test for query by @everpcpc in #15792
  • feat: support dynamic enable of rust backtrace by @dqhl76 in #15749

Thoughtful Bug Fix 🔧

  • fix(query): fix cse projection by @sundy-li in #15409
  • fix(storage): change tracking with alias update row_version bug in merge into by @zhyass in #15412
  • fix(executor): fix broken merge into source profiling by @zhang2014 in #15415
  • fix(query): ensure all inputs pull all data that less than max partition count in new agg singleton by @Freejww in #15418
  • fix(query): fix array_concat function domain panic by @b41sh in #15424
  • fix: ExistsTable should has_result_set. by @youngsofun in #15431
  • fix: max_files not work for transform on ndjson/csv/tsv. by @youngsofun in #15441
  • fix(functions): improve array_prepend and array_append by @andylokandy in #15437
  • fix(executor): fix queries executor cannot kill problem and reactivate kill test assertion by @dqhl76 in #15443
  • fix(query): fix create table with cluster by get map type panic by @b41sh in #15475
  • fix(query): RoleInfo from_pb should not err by @TCeason in #15496
  • fix: fix group by cube with join panic assert by @lichuang in #15515
  • fix: Enable opendal backtrace output to help debug by @Xuanwo in #15521
  • fix: explain ast of multi table insert stmt by @dantengsky in #15527
  • fix: Prevent premature snapshot commit on unfinished writes by @drmingdrmer in #15534
  • fix(query): fix view flaky test with different dialect by @b41sh in #15533
  • fix(query): fix qualify display by @sundy-li in #15537
  • fix: FlightData must contain valid data header by @Xuanwo in #15559
  • fix: http handler kill not working. by @youngsofun in #15558
  • fix: Avoid duplicate temp file name for install-snapshot requests by @drmingdrmer in #15565
  • fix(query): fix shrink_d256 for negative value by @sundy-li in #15574
  • fix(query): fix bloom fitler using is_null and not function by @b41sh in #15577
  • fix(query): fix filter with nested not expr return wrong result by @b41sh in #15580
  • fix: fix normalize disjunctive filter by @xudong963 in #15587
  • fix: consume stream inside explicit txn by @dantengsky in #15582
  • fix: TxnManager with state AutoCommit should not be saved in http manager. by @youngsofun in #15602
  • fix(query): fix subquery writer in udf by @sundy-li in #15610
  • fix: analyze table error by @zhyass in #15614
  • fix: update derive-vistor to unblock rustc upgrage by @andylokandy in #15625
  • fix: allow http client retry when lost response with next_uri=/final. by @youngsofun in #15624
  • fix: subquery correlated column is derived column by @xudong963 in #15629
  • fix(query): unify optimize plan call in one place by @sundy-li in #15630
  • fix: missing service label for otlp logs by @everpcpc in #15639
  • fix(query): delete len check in mget_database/table_names_by_ids by @TCeason in #15640
  • fix(udf): use Debug to display transport error on UDF server by @flaneur2020 in #15646
  • fix(query): show grants also need to check user roles by @TCeason in #15650
  • fix(query): fix current date/time functions return constant value in a statement. by @b41sh in #15659
  • fix: merge into split by @dantengsky in #15644
  • fix: make sure lock guard is dropped before next recluster by @zhyass in #15662
  • fix(query): db owner role can access all tables under this db by @TCeason in #15634
  • fix(cluster): fix complex plan serde in cluster actions by @zhang2014 in #15672
  • fix: potential missing index creation in merge process by @dantengsky in #15674
  • fix(query): fix result cache in empty result by @sundy-li in #15683
  • fix: display expr considering associativity by @andylokandy in #15685
  • fix: fix create table error but table id uncommttied by @lichuang in #15665
  • fix: failed to select blocks for recluster by @zhyass in #15667
  • fix: add unit test of concurrent commit table meta data by @lichuang in #15703
  • fix(executor): fix lost pipeline on finished callback by @zhang2014 in #15693
  • fix(executor): fix query log lost for http kill by @zhang2014 in #15715
  • fix: fuse_snapshot only show the latest snapshot for external table by @dantengsky in #15721
  • fix: fix union and lazy materialization by @xudong963 in #15727
  • fix(ast): assert ast display and reparse by @andylokandy in #15666
  • fix: Handle Duplicate Tombstones in Meta-Service List Operation by @drmingdrmer in #15736
  • fix: rebuild missing bloom index while pruning by @dantengsky in #15738
  • fix(storage): Fix a potential data loss while concurrent write by @Xuanwo in #15759
  • fix: unload csv file too large. by @youngsofun in #15758
  • fix: off by one mistake in auto compaction by @dantengsky in #15761
  • fix(query): fix decimal cmp by @sundy-li in #15764
  • fix: lazy materialization does not support set operation by @xudong963 in #15754
  • fix: auto cast fail error in copy should show the related columns #15768 by @youngsofun in #15770
  • fix: http handler handle panic when starting query. by @youngsofun in #15785
  • fix(ci): meta chaos dockerfile path by @everpcpc in #15787
  • fix: add settings enable_streaming_load, default 0. by @youngsofun in #15794
  • fix(query): create ownership obejct only check current_role privilege by @TCeason in #15751

Code Refactor 🎉

  • refactor: list stage use stream to avoid OOM. by @youngsofun in #15405
  • refactor: Minimize lock duration during raft-log export by @drmingdrmer in #15417
  • refactor: move storage/fuse/io/files.rs to storage/common/io/files.rs by @BohuTANG in #15428
  • refactor: define CatalogIdIdent with TIdent by @drmingdrmer in #15429
  • refactor: define CatalogIdToNameIdent with TIdent by @drmingdrmer in #15432
  • refactor: set enable_experimental_rbac_check default 1 by @TCeason in #15436
  • refactor(query): enable little vacuum after query finish by @zhang2014 in #15434
  • refactor: remove MergeIntoSource operator by @xudong963 in #15435
  • refactor: ScalarVisitor refactor replace_predicate_column by @TCeason in #15439
  • refactor: use scalarvisitor replace predicate recursive by @TCeason in #15449
  • refactor(query): remove useless settings by @zhang2014 in #15461
  • refactor(query): add query id for query log by @zhang2014 in #15466
  • refactor: Bump OpenDAL 0.46, arrow 51, tonic 0.11, reqwest 0.12, hyper 1, http 1 by @Xuanwo in #15442
  • refactor: SessionContext doesn't require Arc by @drmingdrmer in #15491
  • refactor: remove project operator by @xudong963 in #15479
  • refactor(query): revert dev_setup by @sundy-li in #15516
  • refactor: remove databend-common-meta-app from databend-common-ast by @andylokandy in #15509
  • refactor: remove databend-common-base from databend-common-ast by @andylokandy in #15519
  • refactor(query): allow udf server in insert source by @sundy-li in #15523
  • refactor: remove redundant lifetime on fmt::Formatter by @andylokandy in #15528
  • refactor: SessionContext::current_tenant should not be changed at runtime by @drmingdrmer in #15504
  • refactor: migrate TSV input format to new framework. by @youngsofun in #15506
  • refactor(cluster): refactor flight service actions by @zhang2014 in #15419
  • refactor: Improve logging and snapshot management by @drmingdrmer in #15571
  • refactor: Embed key number into snapshot id by @drmingdrmer in #15588
  • refactor: Remove install_snapshot v0 meta-service API by @drmingdrmer in #15590
  • refactor: Enhance Meta-Service Cluster Management by @drmingdrmer in #15601
  • refactor: Decouple Snapshot Write and Commit Operations by @drmingdrmer in #15603
  • refactor: polish memory usage when copy into parquet. by @youngsofun in #15596
  • refactor(storage): bloom index filter use fold_with_domain instead fold by @b41sh in #15619
  • refactor: refine cast reloperator error info by @xudong963 in #15648
  • refactor: refactor merge into optimizer and fix unexpected distribution plan by @xudong963 in #15507
  • refactor(query): no need to deserialize all userinfo when create user by @TCeason in #15671
  • refactor: enable mem-profiling for meta-service by @drmingdrmer in #15705
  • refactor: delete raft log one by one. by @drmingdrmer in #15723
  • refactor: more explicit error confirmation for http query route error. by @youngsofun in #15621
  • refactor: split None and Owner/Grantobject by @TCeason in #15739
  • refactor(cluster): record query id for cluster mode log by @zhang2014 in #15609
  • refactor: GrantEntry not store ownership privilege by @TCeason in #15730
  • refactor: add workspace crates to Cargo.toml by @andylokandy in #15747
  • refactor: remove unsafe impl Send+Sync for GlobalRegistry by @drmingdrmer in #15782
  • refactor: remove meta-service V001 data version by @drmingdrmer in #15778
  • refactor(ast): eliminate clone when displaying expr by @zhang2014 in #15773
  • refactor: clustering_information support specify cluster keys by @zhyass in #15783
  • refactor: add non-async version of mete-client API request_sync by @drmingdrmer in #15786

Build/Testing/CI Infra Changes 🔌

  • ci: Bump version to 2024-02-08 (the same commit with 1.78) by @Xuanwo in #15455
  • ci: tcph/tpcds use copy instead of streaming load for prepare data. by @youngsofun in #15681

Documentation 📔

  • docs: delete repeat summary by @baerwang in #15744

Others 📒

  • chore: Vendor parquet2 instead by @Xuanwo in #15422
  • chore(query): refactor metrics by @sundy-li in #15421
  • chore: remove unused file by @drmingdrmer in #15438
  • chore(storage): allow append when change tracking was enabled by @zhyass in #15427
  • chore: Bump to rust 2024-02-08 (also 1.78) by @Xuanwo in #15456
  • chore: ignore stream column compare in block compact mutator by @zhyass in #15446
  • chore: polish log for copy. by @youngsofun in #15463
  • chore: improve show streams performance by @zhyass in #15460
  • chore(ci): update deprecated actions by @everpcpc in #15470
  • chore: change flood info log to debug by @BohuTANG in #15473
  • chore: remove breakline for log by @BohuTANG in #15476
  • chore: remove incorrect usage of Box::leak by @dantengsky in #15481
  • chore: fix copy/spill log message by @BohuTANG in #15482
  • chore: Upgrade opentelemetry and remove metrics crate by @Xuanwo in #15484
  • chore: add KVPbApi::get_pb_stream() but it is not yet ready for production use by @drmingdrmer in #15471
  • chore: log file_name name for short by @BohuTANG in #15487
  • chore: Bump poem to 3.0 by @Xuanwo in #15486
  • chore: fix some flood logs with batch progress by @BohuTANG in #15492
  • chore(meta): tracking query id for meta log by @zhang2014 in #15497
  • chore: Increase default io timeout for behavior changes by @Xuanwo in #15493
  • chore(query): tracking query log for mysql handler by @zhang2014 in #15495
  • chore: polish error message NDJSON. by @youngsofun in #15502
  • chore(query): pyenv to 3.9 by @sundy-li in #15511
  • chore: recluster disable sort spill by @zhyass in #15490
  • chore: change load_file_metadata_expire_hours default from 24*7 to 12 hours by @BohuTANG in #15514
  • chore(ci): use python3.12 as build tool base by @everpcpc in #15520
  • chore: adjust elapsed time of various log messages by @dantengsky in #15517
  • chore: Upgrade minitrace related crates to 0.6.5 by @Xuanwo in #15530
  • chore: reduce snapshot building progress logging by @drmingdrmer in #15532
  • chore: convert to arrow column remove ignore inside nullable by @b41sh in #15477
  • chore: leave all un-committed data to purge/vacuum by @dantengsky in #15535
  • chore: mask the config sensitive field for display by @BohuTANG in #15543
  • chore: Remove tokio-console since we don't need it by @Xuanwo in #15546
  • chore: Bump databend client and geozero to latest version by @Xuanwo in #15550
  • chore: Bump rustls to 0.22 by @Xuanwo in #15552
  • chore: remove unused micromarshal patch by @PsiACE in #15551
  • chore(executor): set plain_statistics to false for spill time profiling statistics by @zhang2014 in #15555
  • chore: change the short_sql from 64bytes to 10KB by @BohuTANG in #15554
  • chore: support aborting table navigation by @dantengsky in #15549
  • chore(query): add more heavy actions query type by @zhang2014 in #15556
  • chore(storage): remove abort operation by @zhyass in #15557
  • chore(query): all queryies should generate hash by @TCeason in #15560
  • chore: Remove sentry entirely and bump databend client by @Xuanwo in #15563
  • chore: Reduce the duplicated clap and gix by @Xuanwo in #15564
  • chore(ast): unify all of {escape,unescape,quote,unquote}_{string/at_string/ident} by @andylokandy in #15548
  • chore: short_sql from 10KB to 30KB by @BohuTANG in #15575
  • chore: disk cache load refine and reduce the info flood logs by @BohuTANG in #15576
  • chore: Bump jsonb version 0.4.0 by @b41sh in #15589
  • chore: fix error capture of mulit statement transaction in mysql session by @SkyFan2002 in #15583
  • chore: udpate doc for meta-service compat change by @drmingdrmer in #15593
  • chore(deps): bump rustls from 0.22.2 to 0.22.4 by @dependabot in #15586
  • chore: add more logs for vacuum temp files by @BohuTANG in #15597
  • chore: Refine state machine v002 by @drmingdrmer in #15608
  • chore(deps): bump gix from 0.62.0 to 0.63.0 by @dependabot in #15615
  • chore: more commit logs by @dantengsky in #15617
  • chore: prepare for SMV003: move shared code to separate dir by @drmingdrmer in #15622
  • chore(query): add sort plan before window plan by @sundy-li in #15616
  • chore(query): fix filter reorder check by @Dousir9 in #15613
  • chore: reduce the short_sql to 128bytes by @BohuTANG in #15626
  • chore: remove databend-common-{io, exception} from databend-common-ast by @andylokandy in #15618
  • chore(query): add timeout for open telemetry logger by @zhang2014 in #15627
  • chore: spill out short_sql to common/base/src/string.rs by @BohuTANG in #15633
  • chore: add setting compact_max_block_selection by @zhyass in #15641
  • chore(executor): add some time tracking log by @zhang2014 in #15651
  • chore: minimize #[derive(skip)] by @andylokandy in #15654
  • chore(planner): remove z3 by @Dousir9 in #15663
  • chore: re-enable musl-g++ link for musl build image by @dantengsky in #15679
  • chore(query): fix replace into from distributed select plan by @sundy-li in #15657
  • chore: fix wrong print status by @zhyass in #15684
  • chore(query): add assert on explain parser by @sundy-li in #15658
  • chore(ci): add build for udf feature by @everpcpc in #15682
  • chore(ci): fix release workflow by @everpcpc in #15687
  • chore(ci): fix sqllogic test for release by @everpcpc in #15688
  • chore(ci): fix pack for release by @everpcpc in #15689
  • chore(ci): fix artifact upload/download for gcs by @everpcpc in #15691
  • chore: add unit test of vacuum orphan table meta data by @lichuang in #15690
  • chore(ci): fix checkout and version on release build by @everpcpc in #15692
  • chore(ci): fix distribution for release by @everpcpc in #15695
  • chore(ci): fix notify & publish release by @everpcpc in #15696
  • chore(ci): fix release notify template by @everpcpc in #15697
  • chore(query): add create query in admin tables api by @zhang2014 in #15694
  • chore(ci): fix publish service image by @everpcpc in #15698
  • chore(ci): fix run benchmark after release by @everpcpc in #15699
  • chore(ci): ci-cloud build with python-udf by @everpcpc in #15701
  • chore(ci): only provide debian based service image by @everpcpc in #15704
  • chore(ci): fix download artifacts for docker_all_in_one by @everpcpc in #15708
  • chore: more log for table history traversing by @dantengsky in #15710
  • chore(executor): revert #15693 by @zhang2014 in #15711
  • chore(ci): fix download artifacts for cloud build by @everpcpc in #15712
  • chore(ci): fix dockerfile for cloud service image build by @everpcpc in #15713
  • chore(ci): fix checking release status from jobs by @everpcpc in #15714
  • chore(planner): fix merge predicates with different data types by @Dousir9 in #15725
  • chore: add databend user to docker service images by @everpcpc in #15728
  • chore(query): fix pipeline display by @sundy-li in #15729
  • chore(ci): fix download artifacts for local benchmark by @everpcpc in #15735
  • chore(ci): setup python3.12 for local benchmark by @everpcpc in #15737
  • chore(ci): fix check release status by @everpcpc in #15748
  • chore(ci): fix release notify icon by @everpcpc in #15750
  • chore: output raft log index range for purge logging by @drmingdrmer in #15740
  • chore: add more test case about create table fail by @lichuang in #15765
  • chore: add error info when table name is misleading by @xudong963 in #15771
  • chore: Pin opendal to released version by @Xuanwo in #15766
  • chore: limit the count of set operation by @xudong963 in #15777
  • chore: replace manually incr/decr with couter guard by @drmingdrmer in #15784
  • chore(query): remove useless async keyword by @zhang2014 in #15781
  • chore: add more alias name tests for recursive cte by @xudong963 in #15788
  • chore: remove code about access share table into from meta service by @lichuang in #15793
  • chore: remove meta chaos from release action by @lichuang in #15796
  • chore(query): substr support function domain by @zhyass in #15795
  • chore: update doc about meta compatibility by @drmingdrmer in #15799

New Contributors

  • @forsaken628 made their first contribution in #15444
  • @hanxuanliang made their first contribution in #15465
  • @maxjustus made their first contribution in #15529
  • @baerwang made their first contribution in #15744

Full Changelog: https://github.com/datafuselabs/databend/compare/v1.2.453...v1.2.530

May 6, 2024 (v1.2.452)

What's Changed

Exciting New Features ✨

  • feat(query): implement ST_GEOMPOINTFROMGEOHASH by @kkk25641463 in #15182
  • feat(rbac): drop role will transfer role owns object to account_admin by @TCeason in #15154
  • feat(query): implement ST_MAKEPOLYGON by @kkk25641463 in #15194
  • feat: distinguish early close from kill/cancel with new LogType::Closed. by @youngsofun in #15206
  • feat: add EXECUTE IMMEDIATE statement by @andylokandy in #15144
  • feat(http): get client ip from headers by @flaneur2020 in #15205
  • feat(query): support alter table comment by @TCeason in #15227
  • feat(query): match function support multiple fields with boost by @b41sh in #15196
  • feat: support insert with cte by @xudong963 in #15241
  • feat: add setting script_max_steps by @andylokandy in #15244
  • feat: atomic CTAS by @dantengsky in #15186
  • feat(query): integrate "like" to filter execution by @Dousir9 in #15233
  • feat(query): add user option: disabled by @TCeason in #15249
  • feat(query): Inverted index support set filters and index record options by @b41sh in #15254
  • feat: add sequence meta api by @lichuang in #15247
  • feat(query): add new setting parse_datetime_ignore_remainder. by @TCeason in #15260
  • feat: http handler response add field has_result_set and state Starting. by @youngsofun in #15258
  • feat: support cte in update selection by @xudong963 in #15276
  • feat: impl create dynamic table sql parser and planner by @zhyass in #15250
  • feat: support cte in delete by @xudong963 in #15284
  • feat(query): implement ST_ASGEOJSON by @kkk25641463 in #15214
  • feat(query): support map_keys and map_values function by @b41sh in #15291
  • feat(query): implement TO_GEOMETRY/TRY_TO_GEOMETRY function by @kkk25641463 in #15302
  • feat(query): 14925, support udf wasm by @shamb0 in #15107
  • feat(query): top k syntax support by @TCeason in #15318
  • feat(query): implement st_asewkb/st_aswkb/st_asewkt/st_aswkt/st_geohash by @kkk25641463 in #15325
  • feat(query): support with clause in copy into by @sundy-li in #15343
  • feat(query): inverted index support json type by @b41sh in #15267
  • feat(query): support compute quota feature by @zhang2014 in #15336
  • feat: Introduce enterprise storage quota feature by @Xuanwo in #15338
  • feat(query): inverted index use empty position data when query not contain phrase terms by @b41sh in #15362
  • feat(query): implement ST_DIMENSION function by @kkk25641463 in #15347
  • feat(query): implement ST_ENDPOINT by @kkk25641463 in #15366
  • feat(query): implement ST_POINTN by @kkk25641463 in #15373
  • feat(query): support explain insert with insert source by @TCeason in #15357
  • feat(query): implement ST_SRID by @kkk25641463 in #15377
  • feat(query): implement ST_X/ST_Y/ST_STARTPOINT by @kkk25641463 in #15384
  • feat: Add: stream based snapshot API install_snapshot_v2() to meta-service by @drmingdrmer in #15395

Thoughtful Bug Fix 🔧

  • fix(query): set setting case insensitive by @TCeason in #15199
  • fix(query): show grants panic when role/user has immute db/table privilege by @TCeason in #15188
  • fix: geos-sys compile error #15200 by @drmingdrmer in #15209
  • fix(query): alter modify col comment need modify when type not modify by @TCeason in #15217
  • fix(query): fix negative session_running_acquired_queries by @zhang2014 in #15223
  • fix: fix map get return double nested nullable by @ariesdevil in #15230
  • fix: show locks in current tenant by @zhyass in #15252
  • fix: solve some inconsistencies between code and comments by @YichiZhang0613 in #15229
  • fix: fix delete from subquery and one more condition bug by @lichuang in #15279
  • fix(query): fix udf name with upper case letters by @b41sh in #15300
  • fix: don't constant fold giant expr by @andylokandy in #15311
  • fix(query): fix decimal op loss precision by @sundy-li in #15313
  • fix(query): NDJSON copy into allow cast bool and number to string by @b41sh in #15308
  • fix: incorrect overlap checking (replace into) by @dantengsky in #15315
  • fix(typo): refering -> referring by @andylokandy in #15323
  • fix: insert Plan should not have schema. by @youngsofun in #15330
  • fix(executor): queries exeuctor points calc incorrectly by @dqhl76 in #15349
  • fix(query): show processlit host is null in HTTPQuery by @TCeason in #15363
  • fix: check all partitions for small files instead of the last only. by @youngsofun in #15360
  • fix: structlog should print non-zero-rooted tree by @andylokandy in #15370
  • fix: when listing-kv, limit the right bound if possible by @drmingdrmer in #15372
  • fix: list stage returns at most 15000 files. by @youngsofun in #15385
  • fix(query): spill block need consider scalar by @TCeason in #15387
  • fix: Respond with SnapshotMismatch error instead of internal error by @drmingdrmer in #15391
  • fix: solve some inconsistencies between code and comments by @YichiZhang0613 in #15394

Code Refactor 🎉

  • refactor(query): remove useless code (system.tracing) by @zhang2014 in #15185
  • refactor: explain merge into by @xudong963 in #15195
  • refactor: simplify DatabaseFactory, just static dispatch by @drmingdrmer in #15203
  • refactor(query): move api and metrics to servers mod by @zhang2014 in #15210
  • refactor: add view system tables by @TCeason in #15184
  • refactor: introduce UninitTenant and TIdentRaw by @drmingdrmer in #15207
  • refactor: define DatabaseNameIdent with TIdent by @drmingdrmer in #15235
  • refactor: ScalarVisitor refactor push_down_prewhere by @TCeason in #15215
  • refactor: replace String with Tenant in xxxPlan by @drmingdrmer in #15240
  • refactor: avoid using PREFIX directly. use method to build list-prefix by @drmingdrmer in #15243
  • refactor: remove unused list_all_tables() by @drmingdrmer in #15251
  • refactor: remove kvapi::Key impl for String by @drmingdrmer in #15253
  • refactor: grant_ownership retry logic by @TCeason in #15275
  • refactor: Extract kvapi::KeyCodec trait from kvapi::Key by @drmingdrmer in #15262
  • refactor: Introduce generic parameter N to TIdent by @drmingdrmer in #15271
  • refactor: define ShareConsumer with TIdent by @drmingdrmer in #15280
  • refactor: define VirtualColumnNameIdent with TIdent by @drmingdrmer in #15285
  • refactor: define DbIdListKey with TIdent, and rename it to DatabaseIdHistoryIdent by @drmingdrmer in #15287
  • refactor: define TenantQuotaIdent with TIdent by @drmingdrmer in #15289
  • refactor: Define TableLockKey with TIdent by @drmingdrmer in #15319
  • refactor: TIdent add a flag to indicate if the tenant should be encoded into key by @drmingdrmer in #15321
  • refactor: define BackgroundJobIdIdent with TIdent, with HAS_TENANT disabled by @drmingdrmer in #15327
  • refactor: http handler check has_result_set() before get schema of plan. by @youngsofun in #15329
  • refactor: Define DataMaskIdIdent with TIdent by @drmingdrmer in #15331
  • refactor: add right bound when listing expired keys to avoid unnecessary data copy by @drmingdrmer in #15334
  • refactor: Upgrade Openraft to 0.9.7 by @drmingdrmer in #15339
  • refactor: improve install-snapshot performance by @drmingdrmer in #15345
  • refactor: move snapshot building io to other task. by @drmingdrmer in #15351
  • refactor: only has password policy update user login res by @TCeason in #15356
  • refactor: Define TenantOwnershiptObject with TIdent by @drmingdrmer in #15358
  • refactor: define TenantUserIdent with TIdent by @drmingdrmer in #15361
  • refactor: session_ctx.client_ip refactor to Option<String> by @TCeason in #15368
  • refactor: move DirName to separate file; add peek() to KeyParser by @drmingdrmer in #15365
  • refactor: Define StageFileIdent with TIdent by @drmingdrmer in #15371
  • refactor: remove stage use stream to avoid OOM. by @youngsofun in #15378
  • refactor: sync create inverted index by @b41sh in #15379
  • refactor: dispatch load to 16 threads when installing snapshot by @drmingdrmer in #15381
  • refactor: get_table_by_id() returns an Option<Seqv<TableMeta>> by @drmingdrmer in #15388
  • refactor: remove get_table_name_by_id() by @drmingdrmer in #15389

Others 📒

  • chore(executor): add time limit daemon for queries executor by @dqhl76 in #15174
  • chore(query): max_running_queries from 0 to 8, enable the max running queries by @zhang2014 in #15190
  • chore(query): add session_running_acquired_queries metric by @zhang2014 in #15193
  • chore(executor): add end_of_stream flag for executor stream by @zhang2014 in #15202
  • chore(planner): refine explain read size by @Dousir9 in #15204
  • chore: change parquet_fast_read_bytes setting from 0 to 16MB by @BohuTANG in #15212
  • chore(planner): fix decorrelate exists subquery by @Dousir9 in #15208
  • chore: old op LogicalGet modify to new op name Scan by @TracyZYJ in #15222
  • chore(query): parser error need to log error by @sundy-li in #15219
  • chore: vacuum drop table for external location table by @BohuTANG in #15220
  • chore(query): fix nullable array domain by @sundy-li in #15224
  • chore(query): add query queued duration for query log by @zhang2014 in #15225
  • chore(query): remove useless function map:get by @sundy-li in #15245
  • chore: update sqllogictests crate to 0.19.1 by @finchxx in #15242
  • chore(query): support like push down not by @Dousir9 in #15268
  • chore(executor): add node level error tracking by @dqhl76 in #15239
  • chore(query): refine set sql_dialect error message by @Dousir9 in #15282
  • chore(query): bump arrow-udf to extra lib by @sundy-li in #15283
  • chore(planner): improve delete explain by @Dousir9 in #15290
  • chore: Enhance Flexibility of Dynamic Table Creation by Allowing Options in Any Order by @zhyass in #15297
  • chore: add license to cargo.toml by @BohuTANG in #15299
  • chore(planner): fix can_filter_null by @Dousir9 in #15298
  • chore: improve distributed plan by @xudong963 in #15296
  • chore(query): simplify stage list by @sundy-li in #15303
  • chore: enable distributed optimization for subquery in delete by @xudong963 in #15288
  • chore: correct stream error message by @zhyass in #15309
  • chore: tune auto-compact parameters triggered by insertions by @dantengsky in #15261
  • chore(query): improve delete pruner by @sundy-li in #15316
  • chore(planner): fix merge into statistics by @Dousir9 in #15320