Skip to content

Contributor meetings

Jump to bottom Edit New page
Manfred Moser edited this page May 24, 2024 · 55 revisions

The Trino project organizes regular meetings with contributors to improve collaboration with maintainers and among contributors. This space acts is used to plan upcoming meetings and provide meeting minutes after the event.

General information

  • Trino Contributor Congregations are in-person meetings, typically run the day after Trino Fest or Trino Summit.
  • Trino Contributor Calls are virtual events, typically scheduled for every fourth Thursday of the month.
  • Anyone can attend.
  • Attendees can request invites from the Trino dev rel team (Manfred Moser (@mosabua), Cole Bowden (@colebow), Brian Olsen (@bitsondatadev)) on slack.
  • Attendees can also just join on the day, connections details are announced just prior to the event on slack or on this page.
  • Event dates are announced on slack, LinkedIn, and added to the Trino events calendar.
  • Topics can be suggested prior to the event by updating this page or during the meeting.
  • 📹 Video recordings are posted on the dedicated YouTube playlist starting with the April 2024 call.
  • Meeting notes and other details are captured on this page.

Trino Contributor Call, July 2024

Date to be determined.

Potential topics:

Trino Contributor Congregation, 14 June 2024

In person only event in Boston following Trino Fest. Contact Manfred Moser for invite and registration info. 9:00 - 14:00 EST

Topics, notes and action items

  • Iceberg aggregation pushdown presented and discussed by osscm
  • Incremental refresh on materialized views in Iceberg and beyond osscm
  • Open Policy Agent standard policies for relational data domain, related tooling and more vagaerg, dprophet
  • Trino Gateway update with roadmap and planning discussion, mosabua
  • Trino K8s operator project, osscm and mosabua

Trino Contributor Call, 23 May 2024

📹 Video recording with time stamps on YouTube

Attendees

mosabua, electrum, dain, nineinchnick, brianwmunz, dprophet, findinpath, xkrogen, jkylling, alprusty, marton-bod, vgankidi, osscm, vagaerg, walterddr, lxynov, mgorsk1 Praveen Sadhu, Josh Yeh

Topics, notes and action items

Trino Contributor Call, 24 Apr 2024

Attendees

martint, mosabua, sajjoseph, wendigo, nineinchnick, brianwmunz, bitsondatadev, xkrogen, virajjasani, stoty, kmurra

Topics, notes and action items

📹 Video recording with time stamps on YouTube

  • Status of upgrade to required Java 22
    • Trino works with Java 22
    • Trino 444 uses Java 22 in docker container
    • Requirement for Java 22 in general to follow in 446, 447, or 448
    • Java 23 also used in testing
    • No issues reports so far
  • Discuss Phoenix connector usage
    • Security issue cause Trino project to contemplate removal
    • martint, mosabua and bitsondatadev discuss options about deprecating, reducing impact or removing connector
    • wendigo explain current and past approaches and issues, looking for help
    • stoty and virajjasani explain options and offer to help
    • see https://github.com/trinodb/trino/pull/20739 and others
    • mosabua to follow up on how to proceed with wendigo and martint
    • we are looking for people to help and people who use the connector
    • reasonable for connector to require phoenix 5.2.0 and explicitly declare and manage dependencies for newer hadoop, phoenix and hbase
  • Incremental refresh materialized views (#18673, #20959)
    • mosabua explains that work is ongoing but nothing is there to report
  • Lateral column alias support
    • kmurra explains proposal and discusses with martint
    • various complications such as shadowing aliases, usage in aggregation and window functions, and more
    • kmurra will file issue with research on how it works on other engines and more
    • kmurra to reach out to SQL spec workgroup
    • martint will help with input and eventually review of PR for planner and parser
  • Trino REST API improvement
    • sajjoseph explains nextURI data in HTTP header values for usage with blue/green deployment and more behind load balancer or Trino Gateway
    • he will file issue to discuss more and probably send a PR
    • same for some performance improvements

Trino Contributor Call, 21 Mar 2024

9am PST

Attendees

martint, electrum, dain, mosabua, sajjoseph, jkylling, amoghmargoor, marton-bod, vgankidi, osscm, wendigo, monimiller, oneonestar, alprusty, nineinchnick, Praveen2112, brianwmunz, xingyuanlin, yathi, manoj narayanan, kasun indrasiri and others

Topics, notes and action items

  • Request for speaker submissions for Trino Fest - call out by mosabua
  • Trino Contributor Congregation after Trino Fest in person in Boston - call out to contact mosabua about attedance
  • Discuss plans to move to Java 22, explained by mosabua, wendigo, martint, dain:
    • https://trino.io/blog/2024/03/13/java-22
    • https://github.com/trinodb/trino/issues/20980
    • Build and runtime already work with Java 22
    • One of next release will ship docker container with Java 22
    • A few releases later we will switch to Java 22 as requirement, looking for feedback and testers
    • going to include native code via aircompressor using foreign function support
    • Java 22 should reduce locking problems from gzip
    • JDBC driver and CLI will continue to stay with Java 8
    • various other code base segments will see new language and library feature adoption
    • V2 protocol might see adoption of new language and library features
    • preview features will only be adopted if we see significant benefits
    • we will also adopt Java 23, 24, 25 and so on soon after they are released
  • Update on file system lead and related work - update from electrum
    • removal of Hadoop/Hive library usage completed in new native file system support, massive project and refactor
    • new FileSystemAPI is well designed, simple, clean, tested, no unneeded complexity
    • electrum acts as lead
    • not all legacy features moved
    • looking for feedback from testing
    • docs will be updated more by mosabua
    • see https://trino.io/docs/current/object-storage.html
    • old hdfs code will only be for HDFS, other object storage usage will use new
    • currently planning to have all system disabled by default and require manual activation, looking for feedback on that behavior
    • s3 security mapping and other improvements in progress or based on input
  • OpenTelemetry
    • proven to be VERY useful
    • great TCB episode, https://trino.io/episodes/57
    • looking for practical experience and PRs to add in other important places
  • Merged separation of IR and AST - discussed by martint
    • separation of IR (internal representation) and AST (abstract syntax tree)
    • massive internal refactor
    • fixed many latent, dormant bugs
    • brings numerous performance improvements
    • great code simplification and improvement
    • separated as of 442 release, after series of very large PR,
    • more cleanup work still ongoing, opportunity for future changes on IR, brings performance improvement,
    • avoids lots of duplicate transforms,
    • also removes internal caching needs,
    • still a month or two of work ongoing, looking for feedback after that in terms of people working on separate forks and so on
  • Discuss OkHTTP related regression around redirect - mosabua, electrum and wendigo
    • https://github.com/trinodb/trino/issues/21026
    • Brought issue in Trino Gateway sync
    • Two separate fixes now available
    • electrum and mosabua to follow up and sync and help with review and decision making,
    • redirect with auth can be considered a security risk, might want to restore old behavior,
    • might also have to go through clients and update them if possible, needs to potentially be made explicit,
    • also chat about http client in jdbc driver and other clients,
    • jetty client is too heavy, jdbc driver should probably use jvm http client
  • Iceberg Aggregate Pushdown - mosabua and osscm will lead discussion to establish approach and then drive into more features and other connectors
  • Call for reviewers, contributors, and maintainers - from mosabua, contact for guidance
  • mosabua to plan next call in a few weeks
  • Next call will be recorded and available on youtube channel

Trino Contributor Call, 1 Feb 2024

9am PST

Topics

Attendees

martint, electrum, dain, mosabua, sajjoseph, rice668, alok, i-93, colebow, jkylling, amoghmargoor, marton-bod, vgankidi, osscm

Notes and action items

  • Update on Java 21 provided, all done, no negative feedback from community
  • Test improvements initiative and developer guide for testing is in progress and living document for further improvements
  • Trino Gateway completed release 5, lots more improvements and progress ongoing, dev sync every second Wednesday, release 6 coming soon
  • Trino Kubernetes operator - mosabua to start conversation to move forward similar to Trino Gateway subproject, looking for initial code contributions and then ongoing support and maintenance
  • Default time precision change to 6
    • PR https://github.com/trinodb/trino/pull/20290
    • amoghmargoor will link to github issue
    • mosabua to update to roadmap issue and add tasks with martint and others
    • feature will need a switch since this will be NOT be backwards compatible
    • lots of impact on connectors, functions, and so on
    • testing need will be significant
  • Client protocol
    • no active work going on according to dain
    • parallel transfer to clients probably best to use object storage filesystem as proxy, parallel write by trino, parallel read by client
    • mostly aimed at python and related workloads
    • streaming in parallel from memory is not restartable
    • lots of discussion about Arrow/ADBC, not suitable as protocol for Trino since has limited scope below what Trino offers in terms of data types and such, also tightly tied to Spark only, very limited interoperability, security issues
    • in the long run Trino might end up with arrow support for import/export or so, only for interoperability due to problems of Arrow,
    • Arrow not suitable for native use in Trino
  • Separate aspect is trino to trino cluster communication
    • jdbc just a current approach
    • potential to create separate protocol for trino to trino
    • communication in cluster is implementation details, arrow not suitable at all, no benefits
  • Parquet column encryption
    • in progress with amoghmargoor,
    • blocked by hadoop dependency,
    • trino filesystem no longer uses hadoop deps
    • amoghmargoor will reimplement in collab with electrum
    • mosabua followed up - details at https://github.com/trinodb/trino/pull/20069
  • Alluxio caching
    • PR for Delta Lake from jkylling is ready for merge
    • PR for docs in progress with mosabua
    • Iceberg and Hive support coming in quick follow up PRs
    • Rubix removal PR is ready as well
    • Potential for everything to land together in 439
    • mosabua to follow up on coordination
  • Iceberg agg pushdown
    • osscm working with findepi on multiple PRs
    • including work in Iceberg project, and ideas for future PRs
    • electrum to help
  • Secondary index and aggregate index, Iceberg
    • osscm working with Iceberg and Trino community
    • rice668 working on another(?) implementation
    • mosabua to connect and start thread with electrum and findepi
  • Rewrite partial top-n node to LimitNode or LastNNode
    • PR from rice668
    • Assumption of all files being sorted and having correct ordering might be problematic
    • martint to chime in with input and potential ideas, https://github.com/trinodb/trino/pull/18384
    • mosabua to help coordinate meeting / sync

Prior events

Prior events include various calls and the first Trino Contributor Congregation at Trino Summit 2022.

Add a custom footer
Add a custom sidebar
Clone this wiki locally