Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All data will be lost as migration files have changed #7398

Closed
webnoob opened this issue Jun 1, 2021 · 15 comments
Closed

All data will be lost as migration files have changed #7398

webnoob opened this issue Jun 1, 2021 · 15 comments
Labels
bug/0-unknown Bug is new, does not have information for reproduction or reproduction could not be confirmed. kind/bug A reported bug. team/schema Issue for team Schema. topic: mysql topic: windows
Milestone

Comments

@webnoob
Copy link

webnoob commented Jun 1, 2021

Bug description

When creating a new migration, I "randomly" get something like:
image

How to reproduce

Not sure exactly. It's happening after creating new migrations but not every time. More information on what actually triggers this notification would be helpful in me determining how to reproduce it. How does it determine the files have changed, because the content is the same according to my source control?

Expected behavior

The latest migration I've just added is just adding: isAdmin Boolean? to a table. I wouldn't expect ALL my data to need clearning.

Prisma information

I don't want to share this publicly.

Environment & setup

  • OS: Windows
  • Database: Mysql
  • Node.js version: 14.17.0

Prisma Version

prisma               : 2.21.0
@prisma/client       : 2.21.0
Current platform     : windows
Query Engine         : query-engine e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ..\..\node_modules\@prisma\engines\query-engine-windows.exe)
Migration Engine     : migration-engine-cli e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ..\..\node_modules\@prisma\engines\migration-engine-windows.exe)
Introspection Engine : introspection-core e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ..\..\node_modules\@prisma\engines\introspection-engine-windows.exe)
Format Binary        : prisma-fmt e421996c87d5f3c8f7eeadd502d4ad402c89464d (at ..\..\node_modules\@prisma\engines\prisma-fmt-windows.exe)
Default Engines Hash : e421996c87d5f3c8f7eeadd502d4ad402c89464d
Studio               : 0.371.0
Preview Features     : orderByAggregateGroup, selectRelationCount
@webnoob webnoob added the kind/bug A reported bug. label Jun 1, 2021
@pantharshit00
Copy link
Contributor

Did you modify any migration files? Looks like a bug in checksum logic if that is not the case.

Also, are you using MySQL natively on window or via docker? MySQL on windows by default lowercase all tables name which might be the cause. Read: #6972

@pantharshit00 pantharshit00 added bug/0-unknown Bug is new, does not have information for reproduction or reproduction could not be confirmed. team/schema Issue for team Schema. topic: mysql topic: windows labels Jun 1, 2021
@webnoob
Copy link
Author

webnoob commented Jun 1, 2021

Nothing modified, no.

MySQL native on windows so yes, it could be case sensitivity being an issue. It's been a pain on numerous other tickets over the past 1.5 years. I suspected this might be the case when reporting.

@tomhoule
Copy link
Contributor

tomhoule commented Jun 2, 2021

I think this is probably due to git rewriting the newline characters when checking out files on windows vs unix. That changes the checksums. There is another issue for that: #7101

Do you think it could be the case here? It could have happened if you used migrate inside and then outside (or vice versa) docker or WSL for example.

@webnoob
Copy link
Author

webnoob commented Jun 2, 2021

It's possible but I'm the only one who adds migrations and I use the same environment each time. Sometimes I will be able to make it work without the modification error and other times not.

I'll try and run some tests to see if that's the case.

@webnoob
Copy link
Author

webnoob commented Jun 7, 2021

I've just had this reported again with a migration I've added.

image

I've compared the local version with the version in source control and they both have CRLF as the line feeds.

Current:
image

Source:
image

With this in mind, I can't see how the line feeds would be causing a problem unless Prisma is doing something with them internally.

For info. This migration was created by me. I committed it and pushed it and haven't removed / re-cloned the repo in the meantime so this is the same file Prisma created during the migrate command.

@pantharshit00
Copy link
Contributor

Related? #7101

@webnoob
Copy link
Author

webnoob commented Jun 7, 2021

@tomhoule mentioned the same thing above my post which is why I posted the screenshots showing the downloaded file from source and my local version both have CRLF as the line feeds which makes me think it's not related. I'd be happy for it to be the case though.

Also happy to keep testing once a fix for the EOL issue is sorted to see if it happens.

@tomhoule
Copy link
Contributor

tomhoule commented Jun 7, 2021

Could it be something removing/adding trailing newlines at the end of files?

@webnoob
Copy link
Author

webnoob commented Jun 7, 2021

I have no routines or anything which would modify those files and am not doing so manually either 🤔
I wouldn't even say it's Git as those files would show as changed AFAIK...

@tomhoule
Copy link
Contributor

tomhoule commented Jun 7, 2021

Hmm, that's mysterious. Could there be an encoding question at play, where parts of your setup is using say utf8, and another utf-16?

@webnoob
Copy link
Author

webnoob commented Jun 7, 2021

From what I can see, everything is in UTF-8 ...

I'd be happy to run a dev build with some advanced logging in if it would help.

@webnoob
Copy link
Author

webnoob commented Jun 14, 2021

I have a sneaky suspicion that this might be related to swapping branches.

i.e

  • Add migration and deploy in branch 1. Commit and push changes.
  • Switch to branch 2 then do some stuff (generally I'm just merging in branch 1 changes)
  • Switch back to branch 1 and add migration

Not 100% sure but I've just added a migration without issue and I know I haven't switched branches since my last migration. When I can commit / push this batch of changes, I'll switch branches and do another test to see if it triggers the error.

tomhoule added a commit to prisma/prisma-engines that referenced this issue Jul 19, 2021
See code for documentation on the how and why.

Relevant issues:

- prisma/prisma#7398
- prisma/prisma#7101
tomhoule added a commit to prisma/prisma-engines that referenced this issue Jul 19, 2021
See code for documentation on the how and why.

Relevant issues:

- prisma/prisma#7398
- prisma/prisma#7101
tomhoule added a commit to prisma/prisma-engines that referenced this issue Jul 19, 2021
See code for documentation on the how and why.

Relevant issues:

- prisma/prisma#7398
- prisma/prisma#7101
tomhoule added a commit to prisma/prisma-engines that referenced this issue Jul 23, 2021
See code for documentation on the how and why.

Relevant issues:

- prisma/prisma#7398
- prisma/prisma#7101
do4gr pushed a commit to prisma/prisma-engines that referenced this issue Aug 3, 2021
…2101)

* Encapsulate checksum handling into a module in MigrationConnector

* Make checksum comparison try matching with lf and crlf line endings

See code for documentation on the how and why.

Relevant issues:

- prisma/prisma#7398
- prisma/prisma#7101
@Jolg42 Jolg42 added this to the 2.29.0 milestone Aug 3, 2021
@webnoob
Copy link
Author

webnoob commented Aug 3, 2021

Just to confirm, this is definitely a line ending issue and is related to me switching branches. I've not switched then added a migration until just now and got the same issue.

Going through each migration and changing the line ending to LF has resolved it.
Looking forward to 2.29.0 🥇

@Jolg42
Copy link
Member

Jolg42 commented Aug 3, 2021

Thanks @webnoob for the confirmation

A fix just got released in the dev channel on npm (not recommended for production).

If you can try this specific version 2.29.0-dev.20 to confirm it's fixing the issue for you and let us know it will be greatly appreciated! 💚

It will be released next Tuesday in our biweekly release.

@webnoob
Copy link
Author

webnoob commented Aug 16, 2021

Confirming that 2.29.0 has resolved this issue. One of my colleagues had the issue so I told him to update to .29 to see if it was fixed. It solved it immediately.

Nice job guys, thanks.

@webnoob webnoob closed this as completed Aug 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug/0-unknown Bug is new, does not have information for reproduction or reproduction could not be confirmed. kind/bug A reported bug. team/schema Issue for team Schema. topic: mysql topic: windows
Projects
None yet
Development

No branches or pull requests

4 participants