v1.1.1

Release Notes - Gluten - Version 1.1.1

We are pleased to announce that Gluten has been accepted as an Apache Incubating project. Additionally, we are excited to unveil the release of Gluten-1.1.1. This version marks the final release before our transition to Apache.

Highlights (Velox backend only)

• Support Spark 3.2, 3.3, and 3.4(API only)
• Support 30 common Spark Operators
• Support 220 common Spark Functions
• Velox codebase updated to 2024/02/29
• Refactor Data Lake API to support Delta Lake Scan and Iceberg read COW table
• Better S3, GCS support
• More stability in Spill support
• Enhance metric support for spill, shuffle, and additional metrics.
• Enhance fallback case support by expanding coverage for missing cases and updating messages accordingly
• Enhance Shuffle including merge before compressing, push based shuffle, and more
• More Bug Fixing

What's Changed

• [GLUTEN-3855][VL] Fix ORC related failed UT by @chenxu14 in #3805
• [VL] Support IsNull filter pushdown by @rui-mo in #3791
• [VL] Update velox-backend-limitations.md by @FelixYBW in #3639
• [GLUTEN-2169][VL] Enable GlutenEnsureRequirementsSuite in unit tests by @JkSelf in #3860
• [CH] Fix exception of pb MessageToJsonString by @exmy in #3823
• [GLUTTEN-3851][VL] Add remaining filter time metric by @zhli1142015 in #3852
• [VL] Support ignoreNulls for NthValue window function by @PHILO-HE in #3857
• [VL] Enable using static link for QAT by @marin-ma in #3863
• [VL] Fix assertion failures when mixing use of partial aggregation spilling and flushing by @zhztheplayer in #3872
• [GLUTEN-3796][VL][FOLLOW_UP] Correct test name match and move black list to exclude in VeloxTestSettings by @zwangsheng in #3874
• [GLUTEN-3528][VL] Construct unique & non-overlapping partition/sort keys for window operator by @PHILO-HE in #3883
• [GLUTEN-3879][CH] salt 1% of TPCH-1 data to NULL instead of 10% by @binmahone in #3880
• [VL] Doc refresh by @zhouyuan in #3882
• [GLUTEN-3865][CH] Refactor aggregating without keys by @lgbo-ustc in #3866
• [GLUTEN-3722][CH] Improve shuffle writer by @taiyang-li in #3728
• [VL] Map date_format to a Velox function name by @PHILO-HE in #3878
• [VL]Daily Update Velox Version (20231129) by @yma11 in #3877
• [CORE] Add InputIteratorTransformer to decouple ReadRel and iterator index by @ulysses-you in #3854
• [GLUTEN-3732][VL] Use arrow result-returning variants FileWriter::Open API by @yangzhg in #3733
• [CORE] Move validate methods from TransformerApi to ValidatorApi by @exmy in #3881
• [GLUTEN-3824][CH]Bug fix hdfs path contains space by @KevinyhZou in #3825
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231201) by @lwz9103 in #3898
• [VL] Break up spilling operation to two phases: shrink phase and spill phase by @zhztheplayer in #3895
• [GLUTEN-1699][VL] Support loadLibFromJar on RedHat 7/8 by @ychris78 in #3893
• [GLUTEN-3906] [VL] fix: fix package.sh failed for x86 by @lzjqsdd in #3907
• [GLUTEN-3750][CH]Bug fix json parse error by @KevinyhZou in #3751
• [GLUTEN-3902][VL] Add documentation to configure the Velox+GCS connector by @tigrux in #3902
• [DOC] Revise Gluten document by @PHILO-HE in #3892
• [VL]Daily Update Velox Version (20231203) by @yma11 in #3913
• [VL] Minor improvements for CI stale bot by @zhztheplayer in #3888
• [VL] Avoid reapplying code patches for external projects when ENABLE_EP_CACHE=ON by @zhztheplayer in #3916
• [VL] minor change for fallback log by @zhli1142015 in #3919
• [VL] Add sort merge join metrics by @ulysses-you in #3920
• [GLUTEN-3378][CORE] Datasource V2 data lake read support by @liujiayi771 in #3843
• [VL] ENABLE_EP_CACHE=ON still uses cached Velox build although the build arguments were changed by @zhztheplayer in #3926
• [VL] Make bloom_filter_agg fall back when might_contain is not transformable by @zhli1142015 in #3917
• [VL][CI] update docker build script by @zhouyuan in #3904
• [GLUTEN-3917][FOLLOWUP] Add back SparkShimLoader import by @ulysses-you in #3940
• [VL] Fix VeloxTPCHV1BhjSuite and VeloxTPCHV2Suite useV1SourceList by @liujiayi771 in #3930
• [VL] Fix syntax error in stale.yml by @zhztheplayer in #3945
• [GLUTEN-3854][CORE][FOLLOWUP] Add ColumnarInputAdapter back to recover UI graph by @ulysses-you in #3933
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231206) by @lwz9103 in #3938
• [VL] Add output row metric for InputIteratorTransformer by @Yohahaha in #3939
• [GLUTEN-3927][CH] Improve the performance of element_at by @taiyang-li in #3928
• [GLUTEN-3908][CH] Improve shuffle split for clickhouse backend by remove ColumnNullable's memcmp by @KevinyhZou in #3909
• [GLUTEN-3924][CORE] Match hive UDF name in case-insensitive mode during expression transformation by @taiyang-li in #3925
• [GLUTEN-3958] Use getDeclaredConstructor().newInstance() in ScanTransformerFactory by @liujiayi771 in #3961
• [GLUTEN-3944][CH]Fix gluten.jar with delta20 when use spark 3.3 by @lwz9103 in #3947
• [VL] gluten-te: In dockerfiles, use symbolic link for /opt/velox by @zhztheplayer in #3946
• [VL]Daily Update Velox Version (20231206) by @yma11 in #3954
• Revert "[GLUTEN-3908][CH] Improve shuffle split for clickhouse backend by remove ColumnNullable's memcmp " by @baibaichen in #3965
• [GLUTEN-3890][CH] Respect spill_threshold for all buffers in shuffle writer by @taiyang-li in #3891
• [CORE] Fix wrong fallback cost by @ulysses-you in #3967
• [GLUTEN-3922][CH] Fix incorrect shuffle hash id value when executing modulo by @zzcclp in #3923
• [VL] quick fix for static build git conflict by @zhouyuan in #3971
• [GLUTEN-3486][CH] Fix AQE cannot coalesce shuffle partitions by @exmy in #3941
• [GLUTEN-3949][CH] Merge small blocks from upstream phase into a large one by @lgbo-ustc in #3952
• [GLUTEN-3948][CH] Fix exception and diff of trunc function by @exmy in #3968
• [GLUTEN-3979][CORE] Use exists() instead of map().exists() to improve code readability by @dcoliversun in #3980
• [VL]Daily Update Velox Version (20231208) by @yma11 in #3973
• Revert "[VL] Make bloom_filter_agg fall back when might_contain is not transformable (#3917)" by @loneylee in #3977
• [GLUTEN-3580][VL] support read data from abfs with account key by @gaoyangxiaozhu in #3897
• [GLUTEN-3991][CH] Fix the incorrect display name for the mergetree file format by @zzcclp in #3992
• [VL] gluten-te: Enable BuildKit to support --cache-from by @zhztheplayer in #3964
• [GLUTEN-3841][CH] Support spill in 2nd aggregate stage by @lgbo-ustc in #3772
• [VL] Daily Update Velox Version (20231211) by @zhztheplayer in #3999
• [VL] Fix StringToMap test failure by @PHILO-HE in #3995
• [VL] Make bloom_filter_agg fall back when might_contain is not transformable by @zhli1142015 in #3994
• [VL] Following #3996, fix CI error "Runtime factory already registered" by @zhztheplayer in #4001
• [VL] Fix linking simdjson error when building benchmark by @PHILO-HE in #3960
• [GLUTEN-4002][CH] Update InputIteratorTransformer metrics by @zzcclp in #4003
• [VL] add io wait time metric in scan by @yma11 in #3997
• [GLUTEN-3378][CORE] DeltaScanTransformer to support delta table by @YannByron in #3982
• [CORE][GLUTEN-3936] Support collapse project transformer by @liujiayi771 in #3937
• [GLUTEN-3861][CH] Fix parse exception when join postJoinFilter contains singularOrList by @exmy in #3862
• [CH] enable replaceSortAggWithHashAgg by @exmy in #3658
• [VL] Make it easier to replay GHA CI scripts from local environment with docker support: Phase 1 by @zhztheplayer in #3998
• [GLUTEN-2777][CH]Bug fix get_json_object parse json error while path is number by @KevinyhZou in #2780
• [VL] Fix git version conflict issue in build script by @PHILO-HE in #4007
• [GLUTEN-3983][CORE] Shade Guava dependency by @zwangsheng in #3984
• [CORE] Add expression blacklist by @ulysses-you in #3985
• [GLUTEN-3934][CH]Bug fix log function diff by @KevinyhZou in #3935
• [CORE] Improve merge fallback reason by @ulysses-you in #3981
• [VL] Fix aggregation dumpped as json string by @marin-ma in #3969
• [VL] Daily Update Velox Version (20231212) by @zhztheplayer in #4008
• [GLUTEN-3962][VL] Respect parsed attribute name and remove column name validate logic by @Yohahaha in #3963
• [VL] Add plan validation util for debugging validate process by @Yohahaha in #3972
• [GLUTEN-3676][CH] Enable TPCH Deicmal Test by @loneylee in #3677
• [VL] Open all glog when debug mode enabled by @Yohahaha in #4009
• [VL] enable test 'as map of case class - reorder fields by name' by @zhli1142015 in #4028
• [VL] Enable repartition ut 24940, 40407 and 40660 by @zhli1142015 in #4035
• [VL] Add VeloxPartitionedTableTPCHSuite test case by @liujiayi771 in #4010
• [VL] Decouple BUILD_BENCHMARK with BUILD_TEST by @Yohahaha in #4038
• [GLUTEN-4044] [VL] Pass asLowerCase for complex types parsing by @zhli1142015 in #4045
• [GLUTEN-2805][CH] Fix ExpandStep duplicated col name by @exmy in #2806
• [VL] Add config to switch on/off spilling for partial aggregation by @zhztheplayer in #4031
• [GLUTEN-4055][CORE] Wrapper reason with Option instead of Some to avoid reason is null by @zwangsheng in #4056
• [GLUTEN-4036][CH] Lazy intialize the CHStreamReader in shuffle read, avoid OOM by @lgbo-ustc in #4037
• [GLUTEN-4018][VL] set AWS C++ SDK Log Level in gluten to avoid recompile velox by @dcoliversun in #4050
• [UI] Collapse the collapse table arrow of gluten build information by @wForget in #4064
• [VL] Explode support Literal array and map by @liujiayi771 in #4019
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231216) by @lwz9103 in #4068
• [GLUTEN-3799][CORE] Fix records read metric for columnar shuffle by @wForget in #4051
• [GLUTEN-4065][CH] Support forcedHostLocationTag by @baibaichen in #4066
• [CORE] Fix wrong plan if failed to collapse project by @ulysses-you in #4063
• [GLUTEN-4053][TEST] Disable both spark ui and gluten ui for running sql test by @zwangsheng in #4054
• [Delta] fix and refine Delta UTs by @YannByron in #3955
• [VL] Increase async wait time to 30s when stopping tasks by @zhztheplayer in #4025
• [GLUTEN-3743][VL] Support pushed bool filter in scan by @Yohahaha in #4017
• Remove all logo images to avoid license issue by @weiting-chen in #4089
• Remove vectorgraphics2d from xchart by @weiting-chen in #4083
• [VL] Daily Update Velox Version (2023-12-18) by @GlutenPerfBot in #4076
• [GLUTEN-3378][VL] Iceberg support set partition columns in split info by @liujiayi771 in #3987
• [VL] Error "Mutable config cannot return unprotected reference to values" when reading from S3 by @zhztheplayer in #4079
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231219) by @lwz9103 in #4090
• [VL]Use velox Config to extract QueryContextConf by @yma11 in #4047
• [GLUTEN-4080][VL] Add parameter for velox in builddeps-veloxbe.sh by @dcoliversun in #4082
• [GLUTEN-4097][INFRA] Add .DS_Store into gitIgnore by @zwangsheng in #4098
• [GLUTEN-4092][CH] Disable streaming aggregating before bugs fixed by @lgbo-ustc in #4093
• [GLUTEN-3953][CH] Update clickhouse version(to add mmdb contrib in CH) by @taiyang-li in #4062
• [GLUTEN-4073][VL] Warn users about disabling of tests and storage adapters when ENABLE_BENCHMARK=ON by @surnaik in #4074
• [GLUTEN-4100][VL] Should not simply apply collect_set to array_distinct by @zwangsheng in #4101
• [CORE] Fix miss RowToColumnar with columnar table cache in AQE by @ulysses-you in #4104
• [VL] Mark SplitPreloadPerDriver conf as dynamic config by @zhli1142015 in #4102
• [VL] Avoid early return when spark.gluten.sql.columnar.libpath is not blank by @wForget in #4067
• [GLUTEN-4106][VL] Initialize velox global memory manager by @ulysses-you in #4111
• [VL] enable spark ut 36797, 34165 by @zhli1142015 in #4118
• [VL] Daily Update Velox Version (2023-12-20) by @GlutenPerfBot in #4114
• [VL] update aws-sdk-cpp version to 1.11.169 by @yma11 in #4103
• [GLUTEN-4060][MINOR] Remove the override method that is consistent with the parent by @zwangsheng in #4061
• [GLUTEN-4125][VL] add scan preload splits metric by @zhli1142015 in #4126
• [GLUTEN-4116][VL] Support other object store server scheme in native write by @dcoliversun in #4120
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231221) by @lwz9103 in #4140
• [VL][Delta]add more UT for delta lake and enable in CI by @yma11 in #4027
• [MINOR][VL] clean up aws sdk install script by @zhouyuan in #4138
• [GLUTEN-4032][CH] Fix shuffle read core after union by @taiyang-li in #4117
• [VL] Daily Update Velox Version (2023-12-21) by @GlutenPerfBot in #4137
• [VL] Use partial companion functions for distinct aggregation by @rui-mo in #4112
• [VL] CI: split ubuntu 2204 job to speed up by @zhouyuan in #4156
• [GLUTEN-4121][VL] Initialize s3 filesystem with hive configuration in native write by @dcoliversun in #4129
• [GLUTEN-4141][VL] Support Gluten Build in Debian 11 by @surnaik in #4142
• [MINOR] Define gluten ui enabled key in GlutenConfig by @wForget in #4164
• [GLUTEN-4105][VL] Fix parse substrait plan to json plan error when validation by @Yohahaha in #4107
• [VL] Daily Update Velox Version (2023-12-22) by @GlutenPerfBot in #4149
• [GLUTEN-3942][CORE]fix: Change columnar overrides to accept ShuffleExchangeLike by @felipepessoto in #4167
• [GLUTEN-4144][CORE]Use Weak to Weak ReferenceMap instead of WeakHashMap by @srinivasst in #4145
• [VL] Daily Update Velox Version (2023-12-25) by @zhztheplayer in #4175
• [VL] Fix corrupted metrics data in the filter transformer by @liujiayi771 in #4180
• [GLUTEN-4012][CH] Add metrics for GenerateExecTransformer by @exmy in #4014
• [GLUTEN-4000][CORE] Apply Basic Common Subexpression Elimination for Spark Logical Plan by @taiyang-li in #4016
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231226) by @lwz9103 in #4181
• [GLUTEN-4127][CH] Add bytes limit for concatenating small blocks in ShuffleRead by @lgbo-ustc in #4128
• [VL] add data types test for ORC file format by @yma11 in #4139
• [VL] Daily Update Velox Version (2023-12-26) by @zhztheplayer in #4185
• [GLUTEN-4182][VL] Disallow resetting HDFS client credential to fix performance issue / to avoid race condition by @zhztheplayer in #4184
• [CH]Update ch version by @lgbo-ustc in #4197
• [GLUTEN-3559][VL] Fix failing UTs with spark 3.4 by @ayushi-agarwal in #4023
• [VL] Change proxy for CI network failures by @zhztheplayer in #4187
• [CORE] Prefer to use Gluten plan in fallback policy by @ulysses-you in #4077
• [VL] Refine fallback msg in scan type validation by @Yohahaha in #4183
• [GLUTEN-4119][VL] Skip report fallback operator: ReusedExchange, AdaptiveSparkPlan by @zwangsheng in #4158
• [GLUTEN-4190][CH] Crush on flattening a const null column by @lgbo-ustc in #4192
• [CORE] Remove conservativeOffHeapSize config by @ulysses-you in #4200
• [GLUTEN-4151][VL] initialize local&hdfs file write sink with FileSink::create function by @dcoliversun in #4191
• [VL] Daily Update Velox Version (2023-12-27) by @zhztheplayer in #4199
• [GLUTEN-4160][CORE] Fix ClassCastException occurred when fallback broadcast join and broadcast exchange by @fyp711 in #4166
• [GLUTEN-3559][VL] Fix UTs failing in spark 3.4 which have assertions for SMJ node by @ayushi-agarwal in #4109
• [GLUTEN-3547][CORE] [VL] Add native parquet writer in spark 3.4 by @JkSelf in #3690
• [GLUTEN-4115][CH] Identify the final aggregate stage of aggregating without aggregate functions by @lgbo-ustc in #4124
• [VL] Code refactoring of variable names related to TPCH in Velox UTs by @liujiayi771 in #4131
• [GLUTEN-4209][VL] Fix gluten_jar path in Velox.md by @xumingming in #4208
• [VL] Daily Update Velox Version (2023-12-28) by @zhztheplayer in #4212
• [GLUTEN-4202][CH] HashSelectorBuilder: null rows may have different nested values by @lgbo-ustc in #4204
• [GLUTEN-4029][CH]Improve multiIf by @KevinyhZou in #4030
• [VL][DOC] Fix the package name in Velox.md by @HannanKan in #4169
• [GLUTEN-4085][CH]Bug fix unixtimestamp timezone by @KevinyhZou in #4095
• [CH] Use if function if condition size is 1 by @zhanglistar in #4042
• [CORE] Separate backend-specific PosExplodeTransformer by @PHILO-HE in #4148
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231228) by @lwz9103 in #4211
• [GLUTEN-4194][CORE] Clarify fallback reason when columnar filter is disabled by @wForget in #4195
• [VL] Remove linking installed simdjson when building gluten cpp test by @PHILO-HE in #4220
• [GLUTEN-3559][VL] Fixed unit test failing after change of Spark-38674 by @vibhaska in #4143
• [GLUTEN-3559][VL] Enable and fix few failing tests with spark 3.4 by @ayushi-agarwal in #4223
• [VL] Decouple velox benchmarks/tests build from gluten cpp benchmarks/tests build by @PHILO-HE in #4153
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20231229) by @lwz9103 in #4224
• [VL] Daily Update Velox Version (2023-12-29) by @zhztheplayer in #4225
• [VL] Revert "[VL] Minor fix for CI" by @zhztheplayer in #4230
• [VL] Query hangs on destroying memory manager by @zhztheplayer in #4232
• [GLUTEN-3951][CH]Bug fix floor diff by @KevinyhZou in #3956
• [GLUTEN-2723][CH] Improve Broadcast Join performance -- Part 3 by @baibaichen in #4231
• [GLUTEN-3283][VL] Upgrade arrow version to 14.0.1 and add compile arrow java module. by @fyp711 in #4206
• [CORE] Code refactoring by using splitConjunctivePredicates and ExpressionSet by @liujiayi771 in #4233
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240102) by @lwz9103 in #4237
• [VL] Fix shuffle writer invalid write by @marin-ma in #4240
• [CORE] Version bump to 1.2.0-SNAPSHOT by @zhouyuan in #3950
• [VL] Daily Update Velox Version (2024-01-02) by @marin-ma in #4239
• [GLUTEN-4243][CH] Improve sparkFloor function performance with avx2 by @taiyang-li in #4244
• [GLUTEN-3559][VL] Fix unit tests in GlutenCSV* suites by @vibhaska in #4254
• [VL] Followup for native write files by @ulysses-you in #4246
• [VL]Move getExtendedColumnarPostRules method into shim layer by @JkSelf in #4226
• [VL] Daily Update Velox Version (2024-01-03) by @marin-ma in #4257
• [VL] Follow-up: decouple velox test build from gluten cpp test build and use VELOX_BUILD_TEST_UTILS instead by @PHILO-HE in #4247
• [VL] Remove file format check for pushed down filters by @PHILO-HE in #4253
• [GLUTEN-3943][CH] Support complex input arguments for sparkMurmurHash3_32, and switch default hash algorithm to sparkMurmurHash3_32 by @taiyang-li in #4205
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240103) by @lwz9103 in #4256
• [GLUTEN-4159][BUILD] link to target ZLIB before add velox library by @dcoliversun in #4176
• [CORE] Upgrade spark 34 version to 342 by @JkSelf in #4238
• [VL] Disable system curl to build cpr by @boneanxs in #4262
• [GLUTEN-3559][VL] Fix metadata related suite by @ayushi-agarwal in #4265
• Enable units test in Gluten*SQLInsertTestSuite by @vibhaska in #4255
• [VL][DOC] Update GlutenUsage.md to reflect new configures by @boneanxs in #4261
• [VL] do not write EOS in shuffle by default by @zhouyuan in #4269
• [CORE] fix UnsupportedOperationException caused by gluten C2R by @zhli1142015 in #4165
• [CORE][VL] Fix BatchScanExec filter pushdown logic by @liujiayi771 in #4132
• [VL] Add support to write parquet files to GCS by @tigrux in #3978
• [VL]Update individual parameters for Spark and Velox by @weiting-chen in #4268
• [VL] Support spark file commit protocol by @ulysses-you in #4264
• [VL] Daily Update Velox Version (2024-01-04) by @marin-ma in #4274
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240104) by @lwz9103 in #4272
• [GLUTEN-4260][VL] Fix velox build summary by covering new build options by @PHILO-HE in #4277
• [VL] Replace std::cerr/cout with LOG and remove GLUTEN_PRINT_DEBUG macro by @Yohahaha in #3171
• [VL] Support missing types for struct literal by @zhli1142015 in #4267
• [GLUTEN-4249][CH]Improve cast by @KevinyhZou in #4250
• [VL] Remove DeserializationStream from TaskResource when close() is called by @marin-ma in #4276
• [VL] Build & install spark-3.4.2 in docker and correct spark version in pom.xml by @PHILO-HE in #4285
• [GLUTEN-3559][VL] Fix TPCDS Plan Suite by @ayushi-agarwal in #4282
• Revert "[GLUTEN-3559][VL] Fix TPCDS Plan Suite" by @JkSelf in #4295
• [VL] Add orc char type validation by @kerwin-zk in #4259
• [GLUTEN-4178][CH] Reduce memory usage in aggregate operators by @lgbo-ustc in #4179
• [VL] Daily Update Velox Version (2024-01-05) by @marin-ma in #4284
• [VL] Add FlushableHashAggregateExecTransformer to map Velox's partial aggregation which supports flushing and abandoning by @zhztheplayer in #4130
• Revert "[GLUTEN-4249][CH]Improve cast (#4250)" by @baibaichen in #4307
• [VL] Remove native write files sort and project by @ulysses-you in #4290
• [CORE] Minor fix needProjection in SortExecTransformer by @exmy in #4287
• [VL] remove validate for timestamp in sortRel by @zhli1142015 in #4291
• [CORE] Add preProjection and postProjection in WindowExecTransformer by @liujiayi771 in #4207
• [VL] CI: fix boost download by @zhouyuan in #4313
• [VL] Daily Update Velox Version (2024-01-08) by @marin-ma in #4311
• [GLUTEN-4218] fix: Fix Gluten CPP build breaks by @Surbhi-Vijay in #4217
• [VL] Add missing boost-locale dependency in VCPKG, otherwise an building error will be thrown in Docker Ubuntu22.04 image. by @HannanKan in #4322
• [GLUTEN-4302][CH] Fixed bugs about rewriting date comparison by @lgbo-ustc in #4303
• [GLUTEN-4266][VL] Support cross join type with Merge join and Hash join by @Surbhi-Vijay in #4281
• [GLUTEN-4314][CH]Dump pipeline's details upon completion by @lgbo-ustc in #4315
• [VL] Add binary, struct, decimal support for array literal by @zhli1142015 in #4310
• [GLUTEN-3559][VL] Fix TPCDS Plan Suite by @ayushi-agarwal in #4298
• [VL] Support write empty rdd and empty iterator by @ulysses-you in #4318
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240109) by @lwz9103 in #4325
• [VL] Add the metrics for WriteFilesExecTransformer by @JkSelf in #4214
• [VL]Fallback complex type in native write by @JkSelf in #4335
• [VL] Daily Update Velox Version (2024-01-09) by @GlutenPerfBot in #4323
• [VL] Turn on abfs connector in CI by @gaoyangxiaozhu in #4304
• [VL] Modifications to GHA CI scripts by @PZD-CHINA in #4341
• [VL] Improve native write files fallback by @ulysses-you in #4329
• [VL] Remove QueryBenchmark executable by @Yohahaha in #4278
• [VL] Enable FlushableHashAggregateRule by @zhztheplayer in #4312
• [VL] Optimize GHA workflow with nightly built docker image (part-1) by @PZD-CHINA in #4293
• [VL] Add config to velox's file read by @FelixYBW in #3990
• [VL] Daily Update Velox Version (2024-01-10) by @marin-ma in #4345
• [GLUTEN-4327][CH] Avoid generating a lot of small blocks in StreamingAggregatingTransform by @lgbo-ustc in #4328
• [GLUTEN-4336][VL][DOC] Remove unsupported codecs of reader by @marin-ma in #4358
• [VL] Use new docker image in GHA workflow by @PHILO-HE in #4362
• [GLUTEN-3559][VL] Fix unit tests in GlutenParquetV2FilterSuite by @vibhaska in #4363
• [VL] Shuffle writer merge partition payloads before compressing by @marin-ma in #4099
• [GLUTEN-3559][VL] Fix unit tests in GlutenSQLQueryTestSuite suite - Part 1 by @vibhaska in #4357
• [GLUTEN-4364][CH] Remove memory reserving for new hash tables in the new aggregation implementation by @lgbo-ustc in #4365
• [GLUTEN-4306][CH]Fix CI Failure of Cast Function by @KevinyhZou in #4337
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240111) by @lwz9103 in #4371
• [VL] Remove v1write added sort for non-string type by @ulysses-you in #4372
• [VL] A couple of code optimizations to TaskResources by @zhztheplayer in #4292
• [VL] Daily Update Velox Version (2024-01-11) by @marin-ma in #4366
• [GLUTEN-3559][VL] Enable UTs failing due to wrong spark.test.home set earlier by @ayushi-agarwal in #4373
• [GLUTEN-3559][VL] Fix unit tests in GlutenSQLQueryTestSuite suite - Part 2 by @vibhaska in #4380
• [VL] Fix undefined symbol issue in debug mode by @JkSelf in #4375
• [GLUTEN-4376][CH] Fix parse exception when parsing post_join_filter in JoinRelParser by @exmy in #4378
• [GLUTEN-3559][CORE] Fix output partitioning of Hash join by @ayushi-agarwal in #4342
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240112) by @lwz9103 in #4383
• [VL] Daily Update Velox Version (2024-01-12) by @GlutenPerfBot in #4384
• [VL] Add a test for isnan function by @zhli1142015 in #4286
• [VL] Enable native write files by default for Spark3.4 by @ulysses-you in #4280
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240114) by @lwz9103 in #4393
• [CORE] Add simpleString for InputIteratorTransformer by @liujiayi771 in #4388
• [CORE] Add fallback summary for subqueries by @wForget in #4228
• [VL] Enable a spark test: handle am-pm timestamp parsing when hour is missing by @PHILO-HE in #4387
• [Core][VL] Add random parquet data generator and ShuffleWriterFuzzerTest by @marin-ma in #3584
• [VL] Daily Update Velox Version (2024-01-15) by @rui-mo in #4396
• [GLUTEN-1551][VL][FOLLOW-UP] Explicitly set Charset to UTF-8 for leading/trailing whitespace string cast test by @boneanxs in #4374
• [CORE] Update Substrait to 0.24.0 by @ted-jenks in #4361
• [VL] Optimize find duckdb when BUILD_TEST=ON by @Yohahaha in #4351
• [VL] Optimize GHA workflow with nightly built docker image (part-2) by @PZD-CHINA in #4348
• [GLUTEN-4405][CORE] Sort values of In/InSet transformer for deterministic behavior by @zwangsheng in #4406
• Revert "[GLUTEN-3942][CORE]fix: Change columnar overrides to accept ShuffleExchangeLike" by @felipepessoto in #4346
• [VL] Daily Update Velox Version (2024-01-16) by @rui-mo in #4410
• [VL] minor fix for velox cache by @zhli1142015 in #4400
• [VL][CI] increase maven download retry count by @zhouyuan in #4412
• [VL] Fix MicroBenchmark input batch size by @marin-ma in #4403
• [VL][CI] Enable abfs in building velox to install its dependencies in docker image by @PZD-CHINA in #4413
• [VL][CI] Optimization of Docker Build: Integrating Maven Commands into Dockerfile by @PZD-CHINA in #4420
• [GLUTEN-4296][CH] Improve to_date function by @KevinyhZou in #4297
• [GLUTEN-4316][CH] Fix crash on dynamic partition inserting by @exmy in #4317
• [GLUTEN-4122][VL] Add num_threads option to limit build threads for velox by @wForget in #4123
• [VL] Fix incorrect url path caused by decoding on non-original encoded path in PartitionedFile by @yma11 in #3873
• [GLUTEN-4416][VL][CI] Enable AQE for TPC-H test in some CI jobs by @zwangsheng in #4417
• [VL][CI] Ensure latest Gluten is cloned & compiled in docker build by @PZD-CHINA in #4429
• [VL] Change row2Partition to uint32_t to deal with num partitions ... by @marin-ma in #4401
• [GLUTEN-4333][CH] Fix CSE issue in aggregate operator by @taiyang-li in #4349
• [VL] Daily Update Velox Version (2024-01-17) by @rui-mo in #4426
• [GLUTEN-3559][VL] Fix unit tests in GlutenParquetV1FilterSuite and GlutenParquetV2FilterSuite by @vibhaska in #4381
• [VL] add missed validation for in expr and pushdown filter by @zhli1142015 in #4414
• [VL] Use uncompressed size for "dataSize" metrics in shuffle by @marin-ma in #4428
• [VL] Hot fix: remove log by @marin-ma in #4442
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240118) by @lwz9103 in #4435
• [VL] Fix failed tests TestRuntime.GetResultIterator by @boneanxs in #4437
• [VL] Optimize ensureFlattened by @marin-ma in #4415
• [GLUTEN-4170][VL] Decouple partitions from plan to avoid driver stalled by @Yohahaha in #4177
• [VL] Daily Update Velox Version (2024-01-18) by @rui-mo in #4436
• [Core] Fix batch size limit by @marin-ma in #4445
• [CORE] Unify order direction transform by @ulysses-you in #4450
• [VL] Make velox orc scan configurable and keep it on by default. by @SinghAsDev in #4447
• [VL] Gluten-te: Invalidate docker local cache only when branch code is updated by @zhztheplayer in #4444
• [GLUTEN-4421][VL] Disable flushable aggregate when input is already partitioned by grouping keys by @zhztheplayer in #4443
• [CORE] Support native write with noop by @jackylee-ch in #4377
• [VL] support spark nanvl function by @zhli1142015 in #4446
• [GLUTEN-4440][VL] Remove FIXME comment in SparkResourceUtil#getTaskSlots by @kevincmchen in #4439
• [VL][Minor] Fix a format issue of unclosed braces by @marin-ma in #4455
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240120) by @lwz9103 in #4461
• [VL] Daily Update Velox Version (2024-01-19) by @rui-mo in #4463
• [CORE] Make it clear that expand-partitioning only involves in inner-like joins by @zhztheplayer in #4456
• [VL][CI] Use new docker images for static-build-centos7-test. by @PZD-CHINA in #4427
• [VL][CI] Disable run_setup_script in building velox by @PZD-CHINA in #4397
• [MINOR] Code clean for ConverterUtils.getFileFormat by @Yohahaha in #4431
• [VL] Daily Update Velox Version (2024-01-22) by @rui-mo in #4486
• [VL] Support hex function by @Yohahaha in #4469
• [VL] Fix filter push down on short column by @mskapilks in #4221
• [GLUTEN-3582][UT][Part-1] Update the TPCH test Parquet files with a page index. by @baibaichen in #4476
• [CORE] Add rule to rewrite multi-children count by @ulysses-you in #4471
• [CORE] Optimize some code in HashAggregateExecBaseTransformer by @liujiayi771 in #4459
• [CORE] Propagate fallback reason to union and shuffle exchange by @ulysses-you in #4489
• [VL] Code cleanups against whole stage transform: Consolidate C++ iterator stage types by @zhztheplayer in #4478
• [VL] Refactor 'getLiteralValue' logic by @zhli1142015 in #4331
• [CH] Upgrade clickhouse version to the commit 377f89518a6 of the rebase_ch/20240120 by @zzcclp in #4496
• [VL] Daily Update Velox Version (2024-01-23) by @rui-mo in #4488
• [CORE] Fix ut module compilation with hadoop3.x by @Yohahaha in #4490
• [VL] Daily Update Velox Version (2024-01-24) by @rui-mo in #4500
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240124) by @lwz9103 in #4499
• [VL] Clean up some legacy code for expression validation by @PHILO-HE in #4508
• [VL] Correct unit displayed on SQL UI for spill-related metrics by @zhztheplayer in #4507
• [VL] Pretty log native fallback reason by @Yohahaha in #4494
• [GLUTNE-4147][CH] Support GetArrayStructFields function for CH by @taiyang-li in #4210
• [VL] Update velox backend limitation docs for Spark3.4 by @ulysses-you in #4522
• [GLUTEN-4454][CH] Support write mergetree data for CH backend by @zzcclp in #4457
• [VL] Daily Update Velox Version (2024-01-25) by @rui-mo in #4513
• [VL] Fix Filter node falling back to Spark whenever there is Timestamp column in input (child node) by @mskapilks in #4441
• [GLUTEN-4510][CORE] DatasetTransformer should skip collect ReusedExchangeExec node fallback info by @zwangsheng in #4527
• [GLUTEN-4521][CH] Fixed bugs in GraceMergingAggregatedTransform which makes data lost by @lgbo-ustc in #4525
• [VL] Remove fmt lib from vcpkg.json by @PHILO-HE in #4531
• [CORE][VL] Enable from_unixtime function and fix test failure by @PHILO-HE in #4520
• [VL] Fix wrong native plan string by @ulysses-you in #4532
• [VL] fix delta column mapping for struct type columns by @zhli1142015 in #4530
• [GLUTEN-4545] [CH] Rename Common/Logger.h to avoid conflict. by @baibaichen in #4546
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240127) by @lwz9103 in #4547
• [VL] Offload byte type scan by @rui-mo in #4538
• [GLUTEN-3432][VL] Add support for CartesianProductExec by @Surbhi-Vijay in #4022
• [GLUTEN-4141][VL] Fix HDFS not working in debian 11 by @surnaik in #4539
• [GLUTEN-3559][VL] Enable passing tests in Gluten Cast Suites for Spark 3.3, 3.4 by @surnaik in #4555
• [VL] Daily Update Velox Version (2024-01-26) by @rui-mo in #4528
• [GLUTEN-3559][VL] Enable GlutenStringFunctionsSuite by @surnaik in #4552
• [GLUTEN-3559][VL] Fix Failing UTs of GlutenQueryExecutionErrorsSuite by @surnaik in #4553
• [VL] Optimize resource management for driver/test code by @zhztheplayer in #4533
• [GLUTEN-4350][CH] Improve NativeReader's performance for small blocks by @lgbo-ustc in #4369
• [GLUTEN-4542][CH]Remove PullingPipelineExecutor in SubstraitFileSource by @lgbo-ustc in #4543
• [GLUTEN-3559][VL] Fix "cannot divide by 0" exception for remainder function by @ulysses-you in #4541
• [VL] Enable tests for partition key by @rui-mo in #4540
• [GLUTEN-4480][CH] Decouple LocalFiles from plan to improve driver generating substrait plan by @exmy in #4481
• [GLUTEN-4213][CORE] Refactoring pull out project in SortExecTransformer by @liujiayi771 in #4497
• [VL][CI] Fix fallback to columnar shuffle in celeborn test by @PHILO-HE in #4474
• [GLUTEN-3559][VL] Fix GlutenCollectionExpressionSuite by @surnaik in #4561
• [CORE] Change Hadoop Version for DP 2.2 by @surnaik in #4537
• [GLUTEN-4548][VL] Ubuntu 18 Support by @felixloesing in #4549
• [GLUTEN-4475][VL] Allow offloading Spark hour function by @zwangsheng in #4495
• [VL] Put fmt lib back to vcpkg.json and bump the version by @PHILO-HE in #4557
• [CORE] Add arch suffix for package jar by @Yohahaha in #4529
• [Core][Minor] Do not transform SortAggExec if columnar hashagg is disabled by @surnaik in #4550
• [VL][CORE] Refactor and fix the columnar cartesian product by @ulysses-you in #4564
• [CORE] Cleanup unnecessary splitInfo in SubstraitContext and ReadRel by @exmy in #4563
• [CORE] Fix Spark34's shim descriptor to 3.4.2 by @Yohahaha in #4574
• [VL] Remove some legacy code in a CMakeLists.txt by @PHILO-HE in #4577
• [VL] Daily Update Velox Version (2024-01-30) by @JkSelf in #4570
• [GLUTEN-4141][VL] Support Gluten Build in Debian 12 by @surnaik in #4535
• [VL] Enable ilike tests by @rui-mo in #4584
• [CORE] Add junitxml path of scalatest-maven-plugin for backends-velox, spark33 and spark34 by @Yohahaha in #4573
• [GLUTEN-3559][VL] Fix GlutenStringExpressionsSuite by @surnaik in #4579
• [CORE] Extract common code into FilterExecTransformerBase by @ulysses-you in #4578
• [GLUTEN-1439][CH] Refactor: move some backend-specific expression transformer to cpp-ch/local-engine/Parser to prevent duplicated expression evaluation by @taiyang-li in #4562
• [VL] Enable all tests of GlutenParquetFileFormatV1Suite/GlutenParquetFileFormatV2Suite for Spark33 and Spark34 by @Yohahaha in #4580
• [VL] Support C2R and R2C between broadcast relations by @zhztheplayer in #4544
• [VL] Daily Update Velox Version (2024-01-31) by @JkSelf in #4588
• [GLUTEN-3582][CORE][VL][CH] Refactor filter pushdown logic by @baibaichen in #4582
• [GLUTEN-4558][VL] Enable V1WriteCommandSuite for Spark34 by @Yohahaha in #4566
• [CORE] Minor refactor of use ValidationResult for supportWriteFilesExec by @Yohahaha in #4589
• [GLUTEN-4596][CH] Clear unused codes by @lgbo-ustc in #4597
• [CORE][CELEBORN] Sync the columnar and celeborn shuffle writer by @ulysses-you in #4590
• [CORE] Add schema validation for broadcast exchange by @zhztheplayer in #4608
• [VL] Daily Update Velox Version (2024-02-01) by @JkSelf in #4604
• [CORE] Remove substrait-spark module by @ulysses-you in #4609
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240202) by @lwz9103 in #4612
• [VL] Fix setup script for centos8 by @liujiayi771 in #4606
• [GLUTEN-4603][CH]Handle complex type in native reader by @lgbo-ustc in #4605
• [VL] Upgrade folly version in vcpkg to align with velox by @PHILO-HE in #4591
• [VL] fix wrong result for delta deletion vector by @zhli1142015 in #4614
• [VL] Daily Update Velox Version (2024-02-02) by @JkSelf in #4616
• [VL]Support map_entries function in Gluten by @yma11 in #4595
• [GLUTEN-4751][VL] Enable storage partitioned join UTs with correct settings by @yma11 in #4524
• [GLUTEN-3582][CH] Remove Not Support PushDownFilters by @baibaichen in #4624
• [VL] Enable partition path lower unit test by @JkSelf in #4473
• [GLUTEN-4615][CH] Cover more cases in RewriteDateTimestampComparisonRule by @lgbo-ustc in #4617
• [GLUTEN-4213][UT] Overwrite udf/udf-group-by.sql in spark34 by @liujiayi771 in #4629
• [GLUTEN-3582][CH] Remove ArrowParquetBlockInputFormat and OptimizedParquetBlockInputFormat by @baibaichen in #4626
• [CORE] Remove a redundant is-adaptive check by @zhztheplayer in #4618
• [CORE] Fix limit with offset for Spark3.4 by @ulysses-you in #4607
• [VL] Fix setup script for Velox dependencies by @PHILO-HE in #4623
• [GLUTEN-4592][TEST] Unify Gluten Spark32 Unit Test for both VL & CH BE with GLUTEN_TEST constant as prefix by @zwangsheng in #4581
• [CORE] Record ignored tests by test framework by @Yohahaha in #4635
• [VL] Daily Update Velox Version (2024-02-04) by @JkSelf in #4631
• [GLUTEN-4632][CH] Support asynchronous reading of hdfs by @lgbo-ustc in #4644
• [CORE] Move PullOutPreProject after AddTransformHintRule by @liujiayi771 in #4619
• [GLUTEN-4560][SUB-4] Let test framework track ignored tests and revise test function name by @zwangsheng in #4638
• [GLUTEN-4108][VL] Fix build & runtime error when vcpkg is on in debug mode by @PHILO-HE in #4434
• [GLUTEN-4650][CORE] Fix BatchScanExecShim otherCopyArgs by @ulysses-you in #4639
• [GLUTEN-4636][VL] Fix build script for Velox on MacOS by @zdx19981006 in #4640
• [VL] Use the same result type for decimal round by @rui-mo in #4621
• Bump com.h2database:h2 from 2.1.214 to 2.2.220 in /gluten-ut/spark34 by @dependabot in #4567
• [GLUTEN-4213][CORE] Refactoring pull out project in HashAggregateExecTransformer by @liujiayi771 in #4628
• [VL] Use unscaled value as partition value for decimal type by @rui-mo in #4641
• [GLUTEN-4560][SUB-2] Refactor Gluten Spark33 Unit Test by @zwangsheng in #4648
• [VL] Save and load configurations for micro benchmark by @marin-ma in #4646
• [CORE] Improve rewrite spark plan rules framework by @ulysses-you in #4645
• [GLUTEN-4653][TEST][VL] Disable spark and gluten ui in velox backend unit test by @zwangsheng in #4654
• [VL] Do not fallback write files if output columns contain Spark internal metadata by @ulysses-you in #4661
• [VL] Daily Update Velox Version (2024-02-06) by @JkSelf in #4658
• [VL] Daily Update Velox Version (2024-02-07) by @JkSelf in #4666
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240208) by @lwz9103 in #4673
• [VL] Daily Update Velox Version 2024-02-08 by @GlutenPerfBot in #4674
• [VL] Daily Update Velox Version (2024-02-12) by @FelixYBW in #4688
• [VL] Daily Update Velox Version (2024-02-13) by @FelixYBW in #4690
• [VL] Daily Update Velox Version (2024-02-14) by @FelixYBW in #4693
• [GLUTEN-4695][VL] Fix array type Substrait signature by @liujiayi771 in #4686
• [VL] Daily Update Velox Version (2024-02-15) by @FelixYBW in #4697
• [VL] Daily Update Velox Version (2024-02-16) by @FelixYBW in #4700
• [VL] Daily Update Velox Version (2024-02-18) by @FelixYBW in #4710
• [VL] update docker script in readme by @FelixYBW in #4683
• [GLUTEN-4717][VL] Adapting the bind reference of agg that contains subquery in agg expressions by @liujiayi771 in #4705
• [GLUTEN-4670][VL] Fix split time metric for celeborn by @kerwin-zk in #4671
• [CORE] Add gen columnar shuffle exchange interface by @ulysses-you in #4712
• [VL] Allow try_sum to fallback to Spark by @mskapilks in #4687
• [VL] Daily Update Velox Version 2024_02_20 by @GlutenPerfBot in #4716
• [VL] Support change columnar shuffle commpression level by @ulysses-you in #4714
• [CORE] Untangle C2R / R2C transforming code by @zhztheplayer in #4620
• [VL] Enable array_repeat & array_except function by @PHILO-HE in #4504
• [CORE] Code refactoring for HashAggregateExecBaseTransformer by @liujiayi771 in #4719
• [GLUTEN-4668][CH] Merge two phase hash-based aggregate into one aggregate in the spark plan when there is no shuffle by @zzcclp in #4669
• [VL] Enable weekday & date_from_unix_date Spark functions by @PHILO-HE in #4721
• [VL] Daily Update Velox Version (2024_02_21) by @GlutenPerfBot in #4729
• [GLUTEN-4713][CORE] Not trim the Alias from resultExpressions when determining whether a post-project is needed by @liujiayi771 in #4726
• [VL][DOC] Update velox-backend-support-progress.md by @PHILO-HE in #4722
• [VL] Support multi-children count with row construct by @ulysses-you in #4720
• [GLUTEN-3559][VL] Fix unit tests in GlutenSQLQueryTestSuite suite - Part 3 by @vibhaska in #4655
• [Core] Enable push-based shuffle by @marin-ma in #4662
• [Gluten-4585][VL] Support spark.sql.files.ignoreMissingFiles=true by @zhli1142015 in #4725
• [GLUTEN-3559][VL] Overwrite scalar-subquery-select.sql test by @ayushi-agarwal in #4727
• [VL] Allow updating native hdfs credential by @ulysses-you in #4736
• [GLUTEN-3432][VL] Add support for BroadcastNestedLoopJoinExec by @Surbhi-Vijay in #4565
• [GLUTEN-4279][CH]Bug fix hour diff by @KevinyhZou in #4288
• [CORE] Add metadata exclusion list for native write by @ulysses-you in #4735
• [GLUTEN-4741][CORE] Align the ProjectExecTransformer#verboseStringWithOperatorId of Gluten with the ProjectExec of Spark by @zwangsheng in #4742
• [GLUTEN-4660][CH]asynchronous shuffle read by @lgbo-ustc in #4664
• [GLUTEN-4754][CORE] Let PullOutProjectHelper#eliminateProjectList return idempotent results by @zwangsheng in #4755
• [GLUTEN-4713][CORE] Fix invalid children caused by std::move in RowVectorStream by @liujiayi771 in #4753
• [VL] Daily Update Velox Version (2024_02_22) by @GlutenPerfBot in #4739
• [CH] Don't recursively sync submoudles by @baibaichen in #4758
• [GLUTEN-4749][CH] Support to purge mergetree data for CH backend by @zzcclp in #4756
• [GLUTEN-4452] [CH] fix may get wrong hash table when multi joins in a task by @shuai-xu in #4453
• [Velox] soft affinity support placing duplicate reading to same executors by @zhli1142015 in #4407
• [GLUTEN-3559][VL] Rewrite GlutenInsertSuite test cases with default values by @Surbhi-Vijay in #4737
• Revert "[GLUTEN-4660][CH]Asynchronous shuffle read (#4664)" by @baibaichen in #4767
• [VL] Daily Update Velox Version (2024_02_23) by @FelixYBW in #4765
• [GLUTEN-4642][CH]Fix cse issue in aggregate by @taiyang-li in #4651
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240225) by @lwz9103 in #4769
• [CORE] No need to compare all the agg mode in checkAggFuncModeSupport by @liujiayi771 in #4761
• [CORE] Remove tag TRANSFORM_SUPPORTED and rename transform-hint to fallback-hint: Phase 1 by @zhztheplayer in #4731
• [GLUTEN-4748][VL] Fix JNI array elements memory leak by @zhztheplayer in #4757
• [VL] Fix Gluten cpp compile warning by @liujiayi771 in #4770
• [VL] CI: Use existing Velox repo for building by @zhztheplayer in #4610
• [GLUTEN-4458] [CH] fallback when has IN in join by @shuai-xu in #4523
• [GLUTEN-4594][SUB-3] Refactor Gluten Spark34 Unit Test by @zwangsheng in #4744
• [GLUTEN-4763][VL] Add RewriteTypedImperativeAggregate rule for collect_list by @liujiayi771 in #4764
• [VL] Daily Update Velox Version (2024_02_26) by @GlutenPerfBot in #4775
• [VL] Fix Iceberg getPartitionValueString by @liujiayi771 in #4777
• [GLUTEN-4506][CORE] Fallback filter/project if nested expressions number reaches a configured threshold by @PHILO-HE in #4599
• [VL] Add write path option for generic benchmark by @Yohahaha in #4784
• [GLUTEN-4039] [VL] Add ntile window function support in Gluten by @JkSelf in #4776
• Add OpenSSF Badge in README by @weiting-chen in #4792
• [VL] Daily Update Velox Version (2024_02_27) by @GlutenPerfBot in #4787
• [GLUTEN-4789] [VL] Wrong preceding/following convert in window function node by @WangGuangxin in #4788
• [GLUTEN-1632][CH]Daily Update Clickhouse Version (20240227) by @lwz9103 in #4786
• [CORE] Support pull out project for WindowExec by @liujiayi771 in #4663
• [VL] Daily Update Velox Version (2024_02_28) by @GlutenPerfBot in #4795
• [VL] Preserve JNI libraries after driver/executor process exits in debug mode by @zhztheplayer in #4743
• [GLUTEN-2031][VL] Enable lag window function by @PHILO-HE in #2737
• [GLUTEN-4398][IT] Add Golden Files for TPC-H Spark32 + Gluten Execution Plan by @zwangsheng in #4432
• [VL] Make glog config be session level by @Yohahaha in #4785
• [GLUTEN-4807][CH] Fix 'Unsupported phase for state function' error when merging hash aggregate with the bloom filter agg by @zzcclp in #4810
• [VL] Daily Update Velox Version (2024_02_29) by @GlutenPerfBot in #4802
• [GLUTEN-4587][VL] Add config to force fallback on scan of complex type by @yma11 in #4778
• release version to v1.1.1 by @weiting-chen in #4838

New Contributors

• @chenxu14 made their first contribution in #3805
• @yangzhg made their first contribution in #3733
• @ychris78 made their first contribution in #3893
• @lzjqsdd made their first contribution in #3907
• @surnaik made their first contribution in #4074
• @felipepessoto made their first contribution in #4167
• @srinivasst made their first contribution in #4145
• @ayushi-agarwal made their first contribution in #4023
• @fyp711 made their first contribution in #4166
• @xumingming made their first contribution in #4208
• @HannanKan made their first contribution in #4169
• @vibhaska made their first contribution in #4143
• @boneanxs made their first contribution in #4262
• @PZD-CHINA made their first contribution in #4341
• @SinghAsDev made their first contribution in #4447
• @kevincmchen made their first contribution in #4439
• @mskapilks made their first contribution in #4221
• @felixloesing made their first contribution in #4549
• @zdx19981006 made their first contribution in #4640
• @dependabot made their first contribution in #4567

Full Changelog: v1.1.0...v1.1.1