varchar tasks

[bluestreak01]

MUST HAVE

• Run ingestion benchmark with varchar
• Run Clickbench with varchar instead of String
• plan to switch users over to VARCHAR
• Documentation: Documentation should be changed to prefer VARCHAR over STRING at all places.
• Web console support. @jerrinot: feat(web-console): create table dialog supports VARCHAR ui#281
• TextImport deals incorrectly with quotes when importing into a VarChar column @jerrinot feat(ilp): breaking change💥 - ILP and TextImport create VARCHAR instead of STRING columns #4476

NICE TO HAVE

• JIT support for varchar_col equals, substring equals, starts with, regexp (this might be moved to MUST HAVE, depending on benchmarking)
• add varchar variants of io.questdb.griffin.engine.functions.regex SQL functions
• to_uppercase(varchar) currently produces a string. this is unexpected. not sure if this is a blocker.
• web console to understand rnd_varchar(): @jerrinot rnd_varchar function sql-grammar#34
• Varchar equivalent of RndStrRndListFunctionFactory

COMPLETED

• add varchar variants of SQL functions: quote_ident, split_part @nwoolmer varchar tasks #4360
• Validate out of the box behaviour for all protocols: REST, PgWire, ILP. ILP should create VARCHAR columns by default + compatibility flag to revert back to STRING. @jerrinot feat(ilp): breaking change💥 - ILP creates VARCHAR instead of STRING columns #4475
• alter table alter column type @ideoma feat(sql): support usage of ALTER table SQL to convert column types #4413
• support for VARCHAR in WhereClauseParser (timestamp search by VARCHAR) @jerrinot @bluestreak01 @jerrinot chore(core): WHERE parser support for VARCHAR intervals #4403
• AbstractTextLexer doesn't calculate ASCII flag, so all inserted varchars have us.isAscii() == false. We need to fix that @bluestreak01 in chore(sql): ensure that ASCII flag on VARCHAR is enforced in tests #4401
• varchar update bug @puzpuzpuz fix(sql): fix incorrect results returned by GROUP BY queries with int or symbol key #4353

create table xyz as (select rnd_str(2,5,1) a, timestamp_sequence(0, 1000) ts from long_sequence(10000)) timestamp(ts) partition by DAY;
alter table xyz add column b varchar;
update xyz set b = a;

select count(*) from xyz where a is not null;
select count(*) from xyz where b is not null;

• support in InTimestampTimestampFunctionFactory @jerrinot chore(sql): InTimestampTimestampFunctionFactory Varchar support #4415
• add varchar variants of str aggregate functions such as min/max
• JIT support for varchar_col IS NULL/IS NOT NULL @puzpuzpuz chore(sql): add JIT compilation support for varchar null check #4357
• cast varchar to geohashes @mtopolnik
• cast varchar to ipv4 @jerrinot @mtopolnik
• string vs varchar consistency when casting to timestamp @jerrinot fix(sql): internal error when inserting null string as a designated timestamp #4408
• add varchar variants of SQL functions: trim, ltrim, rtrim, strpos, position, replace @mtopolnik chore(sql): add some missing VARCHAR functions #4430
• add varchar variant of SQL function: starts_with @mtopolnik chore(core): return Utf8SplitString from getVarcharA/B #4453
• introduce single varchar key unordered map; the map can leverage stable pointers infra introduced recently to avoid copying varchars into map memory - instead it'll be storing pointers to mmapped memory @jerrinot chore(core): unordered map specialized in using single varchar key #4435
• Unify access to VARCHAR column, whether inlined or split, in a single class. @mtopolnik chore(core): return Utf8SplitString from getVarcharA/B #4453
• length(varchar) is dog-slow: we should calculate the raw size - number of continuation bytes instead of doing proper UTF-8 parsing; ideally that should be done with SWAR @mtopolnik perf(sql): improve performance of length(VARCHAR) #4442

[nwoolmer]

Related issue: #4262