[EPIC] Re-Introspection

[janpio]

We want to remove one of the remaining big pain points of Introspection: If you manually adjusted your data model after introspection (to rename fields or models, or add default IDs for example) these changes are currently lost once you introspect again ("re-introspect", for example, because your database was changed manually or migrated by another tool).

This project consists of many individual problems that we are tackling one by one:

• [Re-Introspection] Keep manual changes to relation field names #2503: Keep manual changes to relation field names
• [Re-Introspection] Keep @map and @@map  #2545: Keep @map and @@map
• [Re-Introspection] Keep defaults that are not persisted in the database (cuid() and uuid()) #2534: Keep defaults that are not persisted in the database
• [Re-Introspection] Keep order of models of existing Prisma schema #1997: Keep order of models of existing Prisma schema
• [Re-Introspection] Keep order of fields of existing Prisma schema #2968: Keep order of fields of existing Prisma schema
• [Re-Introspection] Comments in the schema are removed by re-introspection #3052: Keep comments in schema
• [Re-Introspection] Keep @updatedAt #3335: Keep @updatedAt

We might also decide that some of them are not worth the effort.

Some more issues are listed related to this under: topic: re-introspection

[janpio]

With 2.1.0 we shipped the experimental flag --experimental-reintrospection that can be used in npx prisma introspect --experimental-reintrospection.

Behind that flag we implemented the first two pieces of Re-Introspection, #2503 and #2545

Please leave feedback on how this feature works for you. We are interested in both positive and negative feedback, so we know if this feature is already ready for production.

[kennytraction]

Thanks for putting the effort into this feature. It is greatly appreciated!

We have only had a few issues with re-introspection:

• Renamed relations which is fixed and working in v2.1.1
• @updatedAt is not maintained. @default(now()) and @updatedAt during introspection #2075
• Error with renamed relations to the same table. I have created an issue and explained it here: Re-introspection error for renamed relations when there are multiple relations to the same table #2844

[janpio]

I tested prisma introspect --experimental-reintrospection on the database @nikolasburk posted in another issue, where he used @@map and renamed relation fields: #2425 (comment) All changes were kept 🚀

[janpio]

Quick update:

We just merged the last planned feature here 🚀

If you install the dev version with npm install @prisma/cli@dev you will get the state we are planning to launch with 2.6.0 as part of the prisma introspect functionality. Further improvement would be merged into the main feature then in the future directly if we find anything else to do.

For now you can still test it with: npx prisma introspect --experimental-reintrospection

It would be great if you all could test this again, before we go into the final integration with #3331 and see if your use cases are covered now - and let us know if not.

(Pinging some people from this and other related issues for visibility: @Akxe @ExiaSR @theneshofficial @kennytraction @theRocket @louisrli @harryttd)

[kennytraction]

@janpio When will a new dev package be available? It seems the current one (2.6.0-dev.9) doesn't have @updateAt changes. That is the only remaining issue we have.

[bkrausz]

Sounds like you're on the tail end of dev work here, but any chance of getting eyes on #3076? tl;dr - reintrospection is incorrect in a particularly bad way when using partial unique indexes, having Prisma ignore them rather than incorrectly assume they're globally unique would be a large improvement in usability for us.

[janpio]

@kennytraction Ahem, there should be a 2.6.0-dev.11 now - seems I was to quick to post here.

[janpio]

@bkrausz Can you explain the "having Prisma ignore them rather than incorrectly assume they're globally unique would be a large improvement in usability for us." bit over in #3076 please? I thought this was just about adding new functionality - which is prioritized versus the other things we are working on and would not land very high in the list right now, if it is not or we need to improve our understanding we can probably fast track that part.

[kennytraction]

@kennytraction Ahem, there should be a 2.6.0-dev.11 now - seems I was to quick to post here.

No problem. With that build it seems all of our issues have been addressed.

Thanks!

[do4gr]

Since re-introspection is in a good state now we will turn this into the default behavior, meaning
prisma introspect will take in the existing schema and use it to enrich the schema taken from the database. If the previous schema contains errors, the command will abort and return the errors.

prisma introspect --force will allow you to ignore the previous schema and return a schema purely based on the database.

We just need product sign off on the naming for the extra flag here:

• --force
• --overwrite
• --clean

Are all options we have considered.

[Akxe]

How about --ignore-schema? It is to me more descriptive.

[albertoperdomo]

After some internal discussions, we've settled on using --force.

Feedback on the alternative options considered:

• --overwrite - both introspection and re-introspection overwrite the existing file so this seems inaccurate and be misinterpreted
• --clean - could be interpreted in multiple ways, i.e. clean up the file, start from clean slate, etc.
• --ignore-schema - describes behavior accurately but feels verbose.

[Akxe]

@albertoperdomo You can always provide an alias: --ignore-schema, or alias -i

This corresponds to the way npm shorten install to i, update to u and so on...

[janpio]

Finished via #3445 which closed #3331 🚀

Re-Introspection will now be the default behavior when you run prisma introspect and already content in your schema.prisma file.