revive this project

[mgorsk1]

• added facets:
  • trino.metadata
  • trino.queryStatistics
  • trino.queryContext
• implement properly queryCreated method of trino EventListener
• extended events with columnLineage and schema DatasetFacetsa
• aligned Job namespace with OL spec expectations
• added docker-compose
• added pom sorter
• bumped trino version
• bumped java version
• bumped ol version
• updated readme

tested with commited docker-compose:

trino:

trino> create table memory.default.mariusz13 as select * from tpch.sf1.nation limit 1;
CREATE TABLE: 1 row

Query 20240322_112141_00000_vcdx9, FINISHED, 1 node
Splits: 19 total, 19 done (100.00%)
1.80 [25 rows, 0B] [13 rows/s, 0B/s]

mock api:

2024-03-22 11:21:41 5.15.0 INFO 5000 received request:

  {
    "method" : "POST",
    "path" : "/api/v1/lineage",
    "headers" : {
      "Host" : [ "olapi:5000" ],
      "Content-Type" : [ "application/json" ],
      "Content-Length" : [ "590" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "protocol" : "HTTP_1_1",
    "localAddress" : "172.19.0.3:5000",
    "remoteAddress" : "172.19.0.2:49510",
    "body" : {
      "eventTime" : "2024-03-22T11:21:41.662Z",
      "producer" : "https://github.com/takezoe/trino-openlineage",
      "schemaURL" : "https://openlineage.io/spec/2-0-2/OpenLineage.json#/$defs/RunEvent",
      "eventType" : "START",
      "run" : {
        "runId" : "c0a4ee42-6ed0-38e2-b1c2-e701c151eb6d",
        "facets" : { }
      },
      "job" : {
        "namespace" : "trino",
        "name" : "20240322_112141_00000_vcdx9",
        "facets" : {
          "sql" : {
            "_producer" : "https://github.com/takezoe/trino-openlineage",
            "_schemaURL" : "https://openlineage.io/spec/facets/1-0-0/SQLJobFacet.json#/$defs/SQLJobFacet",
            "query" : "create table memory.default.mariusz13 as select * from tpch.sf1.nation limit 1"
          }
        }
      }
    }
  }

2024-03-22 11:21:43 5.15.0 INFO 5000 no expectation for:

  {
    "method" : "POST",
    "path" : "/api/v1/lineage",
    "headers" : {
      "Host" : [ "olapi:5000" ],
      "Content-Type" : [ "application/json" ],
      "Content-Length" : [ "707" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "protocol" : "HTTP_1_1",
    "localAddress" : "172.19.0.3:5000",
    "remoteAddress" : "172.19.0.2:49510",
    "body" : {
      "eventTime" : "2024-03-22T11:21:43.463Z",
      "producer" : "https://github.com/takezoe/trino-openlineage",
      "schemaURL" : "https://openlineage.io/spec/2-0-2/OpenLineage.json#/$defs/RunEvent",
      "eventType" : "COMPLETE",
      "run" : {
        "runId" : "c0a4ee42-6ed0-38e2-b1c2-e701c151eb6d",
        "facets" : { }
      },
      "job" : {
        "namespace" : "trino",
        "name" : "20240322_112141_00000_vcdx9",
        "facets" : {
          "sql" : {
            "_producer" : "https://github.com/takezoe/trino-openlineage",
            "_schemaURL" : "https://openlineage.io/spec/facets/1-0-0/SQLJobFacet.json#/$defs/SQLJobFacet",
            "query" : "create table memory.default.mariusz13 as select * from tpch.sf1.nation limit 1"
          }
        }
      },
      "inputs" : [ {
        "namespace" : "tpch",
        "name" : "sf1.nation"
      } ],
      "outputs" : [ {
        "namespace" : "memory",
        "name" : "default.mariusz13"
      } ]
    }
  }

[mgorsk1]

cc @takezoe would you be open to the idea of us moving this code to either Trino or OpenLineage repos?

[takezoe]

would you be open to the idea of us moving this code to either Trino or OpenLineage repos?

Yes, I'm open to that idea. Actually, we have discussed it in the openlineage project before (OpenLineage/OpenLineage#1288) but I didn't have enough time to work on it at that time.