Arrow's lifetime is tied to connection's lifetime

[mgrazianoc]

When a Statement is prepared from within a Connection by calling conn.prepare(...), the method takes its InnerConnection field (db) and calls InnerConnection::prepare by passing a reference to itself. If success, the new generated statement will have a lifetime that can live at most the same amount as conn: Connection (which was passed as a reference to InnerConnection).

We can query the duckdb::arrow_batch::Arrow data by calling Statement::query_arrow, which does its wizardry to fetch the data and generate a new Arrow struct. It does that by giving it its own ownership. In other terms, Arrow will live at most the same amount that conn Connection.

I'm not sure if I understood correctly, but you can only have an duckdb::arrow_batch::Arrow object as long as the conn Connection is alive. Is that intentional?

[wangfenjin]

The Arrow object here is an Iterator that you can call next() to collect arrow RecordBatch which is the real results, and the final results can out live the conn.

That's what we need to as if you close conn, you can't steam the result anymore.

[mgrazianoc]

I see. I guess that's where the difference between Arrow's arrow and Duckdb's arrow starts. I wonder if there's a pattern to keep the current behavior and also extend the arrow's life cycle without the need to copy anything

[wangfenjin]

We shouldn’t copy, we should move