[Good First Issue]StarRocks Hands-on Tasks 2024

[wangsimo0]

Hi Rockstars,

This is a list of proposed Hands-on tasks. If you're new to StarRocks and eager to engage with the community, here are some issues that are well-suited for you to dive into :) These issues are suitable for gaining hands-on experience and becoming familiar with StarRocks development. Also this is an open list, you are welcome to propose more tasks.

Please @kateshaowanjou or @wangsimo0 to book the issue, and add a comment in the issue you picked, so the issue won't be assigned to others. And always discuss with the community about the design before actually developing, some of the issues are really big, don't hesitate to seek help from the community.

External Catalog related issues

Information Schema in External Catalog

In version 3.2 and later, StarRocks enhances compatibility with more BI tools by supporting the information_schema database in External Catalog. This feature serves as a valuable tool for obtaining structured information. While several views within information_schema currently return empty, efforts are underway to optimize support for these views to ensure comprehensive coverage.
StarRocks aligns with MySQL's pattern in supporting information_schema, as it follows the MySQL protocol. We better maintain the compatibility with MySQL, provide as much information as we can, and optimize for efficiency to minimize time consumption. consumed.

• Columns view
• Views view

Trino's Compatibility Issues

In version 3.0 and later, StarRocks supports Trino's SQL_dialect mode; however, ongoing enhancements are necessary to further optimize this functionality.

New Functions

• inverse_normal_cdf and normal_cdf @241600489
  [function] inverse_normal_cdf and normal_cd #38989
• typeof @MicePilot
  [function] typeof #36245
• regexp_split @yangzho12138
  Better Trino compatibility request #37089
• boolor_agg,boolxor_agg,booland_agg
  Support for BOOLOR_AGG, BOOLXOR_AGG, BOOLAND_AGG aggregate function #22949
• from_iso8601_date(string),from_iso8601_timestamp(string)
  [function]from_iso8601_date,from_iso8601_timestamp #40877
• array_agg in window function
  [window function] array_agg in window function #40881 @mygrsun
• cardinality in HLL data type
  [function]cardinality(HLL) #40879
• count(distinct) window function
  [Feature Request] windows funciton support statement count(distinct col) over #46105

Function Mapping

	Trino's function/expression	StarRocks' function/expression	comment	assginee
	map_agg(key, value) → map<K,V>	map()
	show schemas from <catalog_name>	Show databases from <catalog_name>
	array_sort(array(T), function(T, T, int)) -> array(T)	array_sortby(, array0 [, array1...])	This one needs to pay attention to the input order.
	sequence(start, stop)sequence(start, stop, step)In integers data type	array_generate([start,] end [, step])
	last_day_of_month(x) → date	last_day(x,'month');
	map_from_entries(array(row(K, V))) -> map(K, V)	map_from_arrays.	This one needs to pay attention to the transformation. SELECT map_from_entries(ARRAY[(1, 'x'), (2, 'y')]); equals to SELECT map_from_arrays([1,2],['x','y']);
	current_catalog	catalog()		thanks to @macroguo-ghy
	current_schema	database()		thanks to @macroguo-ghy
	slice(x, start, length) → array	array_slice(input, offset, length)
	approx_set(x) → HyperLogLog	HLL_HASH(column_name)
	empty_approx_set() → HyperLogLog	HLL_EMPTY()
	merge(HyperLogLog) → HyperLogLog	HLL_RAW_AGG(hll)

Other Enhancements

• Apache Ranger's policy translator

StarRocks support using Hive service in Ranger to control access towards hive tables. However we discover there are still some community users want to manage all the privs in StarRocks ranger service. So we need a translator(maybe a script)

• Add catalog information in FE's query_detail @happut

After enabling collect query details using admin set frontend config("enable_collect_query_detail_info"="true") user can get query detail using curl -uroot: http://172.26.81.138:8030/api/query_detail?event_time=<unixtimestamp_value> , the information is like ...."database":"simo","sql":"insert into abc values (1,2),(2,3)","user":"root"....
There is no catalog information. Like "catalog":"defaut_catalog"

Apache Hudi & Delta Lake Compatibilities

• Add Hudi sink （✨ HIGH priority）
• Add Delta Lake sink （✨ HIGH priority）

More Connectors

• Oracle catalog
• Kudu catalog @predator4ann
• StarRocks catalog
• Greenplum catalog
• SQLSever catalog
• Clickhouse catalog
• Trino catalog
• DB2 catalog
• Druid catalog
• Oceanbase catalog
• SAP Hana catalog

More Compatibilities

• Hive UDF compatible
• Spark SQL compatible structure
• Hive SQL compatible structure
• Impala SQL compatible structure

[alberttwong]

I'd add iceberg tagging and branch query

[alberttwong]

#37959

[241600489]

I want to pick #38989 @wangsimo0

[mygrsun]

I want to pick #40881 @wangsimo0

[yangzho12138]

I want to pick #37089 @wangsimo0

[kateshaowanjou]

I want to pick #37089 @wangsimo0
You need to also comment under the issue #37089 so I can assign it to you.
If you have any issues during the development process, I can introduce you to the relevant discussion group. https://853921.ma3you.cn/articles/b12e90J/