Bring Metadata Library back!

[dhirajsb]

Is your feature request related to a problem? Please describe.
Since both Python client and Go service library share the schema models for mlmd types, there should be only one source of truth for creating/using types and for testing.

Describe the solution you'd like
Bring back the yaml metadata library definitions and mapper from the older GraphQL code base to restore the metadata library feature.
The library can be used for:

• Generating Go and Python types to keep them aligned
• Initializing mlmd on REST service startup
• Optionally Go/Python/ROBOT framework test code could also be potentially generated to validate that all fields are used and supported

Describe alternatives you've considered
Keeping ml metadata schema synced in separate Go and Python codes and tests is not maintainable in the long run as the number of types increase.

Additional context
None

[tarilabs]

I think this will be of convenience as we will eventually modify our entities (eg ModelArtifact) for instance to add new fields etc.
Or even for new entities.

Optionally Go/Python/ROBOT framework test code could also be potentially generated to validate that all fields are used and supported

On other hands, as explained in the meeting, this won't necessarily detect other types of misalignment such as https://github.com/opendatahub-io/model-registry/issues/206

Based on these, I would conclude this is important but not our top priority atm; unless I missed something!

[lampajr]

I think centralizing the MLMD types setup will avoid issues like https://github.com/opendatahub-io/model-registry/issues/204

Given that IMO the crucial point here is more on the testing part:

• I mean, wherever we put this logic of types setup (e.g., a separate command in the MR Go binary as Dhiraj already did in the first implemention IIRC), we MUST assure that all tests (in Go and Python) will use that logic to setup the types in the test envs.

• Therefore python tests should be slightly adapted to make use of this logic during the startup, e.g., using TestContainers also there? as already proposed (right?) and then call the new command for the types setup.

I am just thinking loud here, so haapy to hear other proposals or considerations.

[dhirajsb]

@lampajr summarized it correctly. Basically, use the library as the source of truth for schema where possible, either directly or using source generated from the library.