Return table configuration parameters in TableMetadata

[eevans]

This adds attributes to TableMetadata for table config parameters (compression, compaction, default time to live, speculative retries, etc). However, because the number and disposition of parameters differs across versions of Cassandra, it (currently) only does so for Cassandra versions >= 3. The rationale for this (such as it is), was to add support moving forward for recent/popular versions, and avoid contributing any more conditional behavior to this area of the code than was necessary. It's not perfect though, users of older versions will see uninitialized struct members – some which are valid for their version of Cassandra, and some which are not.

TL;DR I didn't see an obvious Good Answer here, so consider this my attempt to exploit Cunningham's Law; Happy to iterate on this if there are ideas for making it better

[martin-sucha]

I'd rather not add features that can be implemented in users' code.
There is a possibility that some CQL server won't have some of these columns (I have not checked whether AWS Keyspaces or Scylla have all of them). If that would be the case, it will break getting of the schema metadata.
It seems to me that you could just SELECT ... FROM system_schema.tables and system_schema.views in user code.

What is your use case for adding those parameters to gocql's TableMetadata?

[eevans]

I'd rather not add features that can be implemented in users' code.
There is a possibility that some CQL server won't have some of these columns (I have not checked whether AWS Keyspaces or Scylla have all of them). If that would be the case, it will break getting of the schema metadata.
It seems to me that you could just SELECT ... FROM system_schema.tables and system_schema.views in user code.

That's also true of everything else that TableMetadata contains (that you could query system_schema in code). As a rule, users are discouraged from poking around the system tables, and the other drivers I am familiar with provide abstractions for returning this information (I was surprised to learn that this one did not).

It's probably also worth pointing out that most of these same properties exist in MaterializedViewMetadata. It seems inconsistent that they would be in one, and not the other.

What is your use case for adding those parameters to gocql's TableMetadata?

I'm sorry, do you mean: Why do I need these settings? Or, why did I propose TableMetadata specifically?

---

These settings are all key/value pairs that follow the WITH in DDL statements (ala CREATE ... WITH <key> = <val> AND ...). Perhaps they could be returned as a map using an Options member or similar (the Java driver does something similar, see TableMetadata#getOptions)? This could return nil on databases where it doesn't make sense.

Thoughts?

[martin-sucha]

That's also true of everything else that TableMetadata contains (that you could query system_schema in code).

Fair point.

As a rule, users are discouraged from poking around the system tables,

I don't see anything bad with querying the system tables in user code. Do you have some link to docs to support the statement that it is discouraged?

and the other drivers I am familiar with provide abstractions for returning this information (I was surprised to learn that this one did not).

Fair point. What the driver could provide and the user code can't at the moment is caching and invalidating the cache when things change (although it doesn't work right now, see #1516 and #1186)

It's probably also worth pointing out that most of these same properties exist in MaterializedViewMetadata. It seems inconsistent that they would be in one, and not the other.

This is a good point. I didn't notice they are in MaterializedViewMetadata already.

I'm sorry, do you mean: Why do I need these settings? Or, why did I propose TableMetadata specifically?

I was curious about what you use the settings for, as I haven't needed to manipulate them programmatically yet.

These settings are all key/value pairs that follow the WITH in DDL statements (ala CREATE ... WITH = AND ...). Perhaps they could be returned as a map using an Options member or similar (the Java driver does something similar, see TableMetadata#getOptions)? This could return nil on databases where it doesn't make sense.

It would certainly make sense to have the same interface as in MaterializedViewMetadata to be consistent. Although I'm not sure whether the options differ between databases. The approach with map as in the Java driver seems a little bit more flexible since the options / types could differ between server versions. Maybe we should switch MaterializedViewMetadata to use a map as well?

@mmatczuk does Scylla support exactly the same options as Cassandra or are there some differences? Thoughts?

[Zariel]

I think a bunch of stuff like this could live in gocql-contrib

…

On 11 May 2021, 7:53 PM +0100, Martin Sucha ***@***.***>, wrote: > That's also true of everything else that TableMetadata contains (that you could query system_schema in code). Fair point. > As a rule, users are discouraged from poking around the system tables, I don't see anything bad with querying the system tables in user code. Do you have some link to docs to support the statement that it is discouraged? > and the other drivers I am familiar with provide abstractions for returning this information (I was surprised to learn that this one did not). Fair point. What the driver could provide and the user code can't at the moment is caching and invalidating the cache when things change (although it doesn't work right now, see #1516 and #1186) > It's probably also worth pointing out that most of these same properties exist in MaterializedViewMetadata. It seems inconsistent that they would be in one, and not the other. This is a good point. I didn't notice they are in MaterializedViewMetadata already. > I'm sorry, do you mean: Why do I need these settings? Or, why did I propose TableMetadata specifically? I was curious about what you use the settings for, as I haven't needed to manipulate them programmatically yet. > These settings are all key/value pairs that follow the WITH in DDL statements (ala CREATE ... WITH = AND ...). Perhaps they could be returned as a map using an Options member or similar (the Java driver does something similar, see TableMetadata#getOptions)? This could return nil on databases where it doesn't make sense. It would certainly make sense to have the same interface as in MaterializedViewMetadata to be consistent. Although I'm not sure whether the options differ between databases. The approach with map as in the Java driver seems a little bit more flexible since the options / types could differ between server versions. Maybe we should switch MaterializedViewMetadata to use a map as well? However, is there a @mmatczuk does Scylla support exactly the same options as Cassandra or are there some differences? Thoughts? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.