[Obsidian-git] SHA check failed!

[GJRobert]

Please be sure to mention:

• whether you are using Node or the Browser

Obsidian app, iOS version, in one of its plugin called obsidian-git. Guess it's counted in the Browser category?

• how you are using it (if you are using a bundler what bundler; if you are using a <script> tag what CDN URL)

A Javascript plugin of Obsidian mentioned above: obsidian-git

---

This obsidian-git plugin makes Obsidian vaults of Markdown notes able to do Git actions, which can be executed on major OS platforms.

Recently on iOS, we are encountering this issue: denolehov/obsidian-git#503
i.e. when pushing changes to the remote repo,
an error message reads:

InternalError: An internal error caused this
command to fail. Please file a bug report at
https://github.com/isomorphic-git/isomorphic-
git/issues with this error message: SHA check
failed! Expected
27eb43e79ec4b1236d2ba1b27e738f279420bd93
, computed
b77dOb17f624d487b84cc67d4510794aeb34981
9

(SHA values vary with different instances)

Could you provide some insights so that we can have better information to fix on the obsidian-git side, or try to workaround/fix the situations? (I'm one of users of Obsidian, not developer of obsidian-git)

Feel free to correct me if this issue does not belong here.

Best regards

[dbischof90]

I second this, I just got the same error in the same setup. Removing and re-adding the same branch did not work.
Started showing up for me about 7h ago.

[jcubic]

I don't have this is the issue with isomorphic git, since it's pretty stable, there were no recent changes. You need to track what have changed since last 7h.

[dbischof90]

I didn't really change something in my setup. I'm also just reporting that I got a pop-up telling me that this is something I should open an issue about in this Github and saw that someone else had the same issue than me on the same day. I see that neither obsidian-git nor this project has received updates in the last days but that can also mean that there is a bug in there that hasn't been caused by the last commit. "Pretty stable" does not mean void of bugs.

My problem was not replicateable with other git clients and after copied out my changed files, deleted the repository, cloned it again, applied the changes as before I could push. I did not dive into the source code but judging from the hash error it feels a bit like that the state during which the file was pushed was modified during the commit (e.g. the hash was computed while I was still typing) and that lead to a hash that did not correspond to the hash that was commited last into the repo.

Is that possible? If yes, then file buffers would need to be locked before starting the commit. That could also be something of interest to obsidian-git in that case. Just trying to find a productive solution approach here and collect some info we can pass on to the obsidian-git devs.

[GJRobert]

Hi @jcubic, do you know any of the developers of isomorphic-git? Or can anyone provide insight about what may be going on with this issue? Why would the SHA be corrupted? What does that mean? Is there any possible solution if this happens?

I guess @dbischof90 is talking about the right direction? But I lack the knowledge about how these things work. Any clues are appreciated.

[jcubic]

@GJRobert there are no isomorphic-gt developers, this project doesn't have a main author that writes all the code and it's driven by the community. I'm a maintainer but I don't write much code myself, I mostly do code review and merging Pull Requests.

So if you want to fix this you will need to debug this yourself, I can't help you, I don't have any idea what can be wrong with the code. I suggest reading the code and trying to reproduce the issue somehow, you will probably need to set up dev env with all the libraries run together and try to tinker with the code and see how it behaves.

[jcubic]

@dbischof90 your hypothesis is good as any hypothesis, I would just try to setup the dev env of the libraries used and try to modify the code to add a lock and see if this will solve the issue. Unfortunately, this library doesn't have a main author that will do this kind of debugging for free, so you will need to do this on your own. You already have Obsidian and Obsidian-git installed so it should be easier for you to set up the environment to test this.

[GJRobert]

@jcubic Thank for your clarification~

[dead-end]

Hi @GJRobert, I did not work much with isomorphic-git, but this is my understanding of the issue.

The error occurs when the GitIndex class tries to read the .git/index file with the staging information. The file consists of 3 parts:

1. a fixed size header
2. a list of staging informations for each file
3. a hash which is computed over the first two parts

When the GitIndex reads the .git/index file, it computes the hash over the part 1. and part 2. and compares the result with the part 3. which is the hash, when the .git/index was written. So the hash value is used to check if the .git/index file is corrupted, since it was written.

To find your bug, you should first open an editor and see if .git/index looks like an index file. Then go through your code and check if someone tries to write to the index file.

This is a good articel about the index file: Understanding Git — Index

[Tsai5413]

same problems here.
I deleted all files and tested them a few times, and errors only happened when I added other plug-ins. Task, dataview, highlighter, etc...

[amxmln]

This might be related to Safari, or the Apple JS engine, if such a thing exists. I just had a report of this issue come in from someone using Safari on the desktop. Might also explain why this started cropping up now all of a sudden without the project changing.

[austinwilcox]

Has there been any headway made on this issue? I am also experiencing this problem.

[gkibt]

I am new to Obsidian and Obsidian Git, and I am currently testing their overall functions. I have 3 sync testing groups:

• iCloud sync only
• Obsidian Git only
• iCloud sync and Obsidian Git working together

I tested these groups on the following devices: my iPhone 12, my iPad Air 4, and my Windows 10 PC. All of them are upgraded to the latest versions.

I find it quite interesting that this SHA error occurs only when iCloud and Obsidian Git are working together. Moreover, the error appears on both iPadOS/iOS and my Windows 10 PC in the same group. The Obsidian Git-only groups on my devices have not experienced this error so far. Cloning the repository again could solve the problem temporarily, but it eventually happens again. I suspect iCloud might be causing the problem, but I'm not entirely sure. Maybe...?

[devzero]

I am new to Obsidian and Obsidian Git, and I am currently testing their overall functions. I have four sync testing groups:

• iCloud sync only
• Obsidian Git only
• iCloud sync and Obsidian Git working together

I tested these groups on the following devices: my iPhone 12, my iPad Air 4, and my Windows 10 PC. All of them are upgraded to the latest versions.

I find it quite interesting that this SHA error occurs only when iCloud and Obsidian Git are working together. Moreover, the error appears on both iPadOS/iOS and my Windows 10 PC in the same group. The Obsidian Git-only groups on my devices have not experienced this error so far. Cloning the repository again could solve the problem temporarily, but it eventually happens again. I suspect iCloud might be causing the problem, but I'm not entirely sure. Maybe...?

I experience this issue reliably and do not use iCloud at all

[jcubic]

You need to report this to Obsidian git, if you're not a developer they need to debug this on their side. We don't have a way to debug their software. Just because the error say to report the issue to isomorphic-git it doesn't mean that we can fix the issue.

[jcubic]

But this project some time ago introduced a big by merging the wrong PR, it seems right but it was causing for very slow statusMatrix. Maybe all the Obsidian Git has to do is update dependencies. But you need to report this to the Obsidian Git extension, we can't fix this on our part.

[dbischof90]

When was that? Looking at their package.json, the current version references 1.24.2, so that is somewhat recent.

[chmac]

I have found this issue although I'm using isomorphic-git directly and not ObisidianGit. I get this issue reliably on Safari, but not on any other browser. I suspect something has changed on Safari that results in this issue. I have been using my setup for several years without issue until sometime in the last few months when it started throwing this error on every push operation. I noticed first on Safari on iOS, but I can also reproduce on Safari Desktop.

One thing to add, I'm also using the @isomorphic-git/lightning-fs filesystem.

If I find time to debug this further, I'll post updates.

[chmac]

I believe this error is being generated here:

isomorphic-git/src/storage/readObject.js

Lines 69 to 71 in d72cf91

	throw new InternalError(
	`SHA check failed! Expected ${oid}, computed ${sha}`
	)

I have realised that in my case, it is related to the size of the repository. On a repository which is ~90MB and has 2k commits, this error gets thrown on Safari. But in other browsers, or on smaller repositories, I don't get this issue. In my case I can reduce my repo size, so I'll give up debugging this further now.

[chmac]

Well, I did some further digging anyway. In my case, when I create new commits, they are being truncated. So what's actually being saved to the object file is invalid, it's missing the last few bytes. I'm not sure why though.

[chmac]

In my case, the issue seems to be related to file size. With a file of ~72Kb, I see this error. When I reduce the size of the file, the issue goes away.

[jcubic]

Maybe the problem is with the limit of the local filesystem that uses IndexedDB. I'm not sure what are the limits of IndexedDB.

[chmac]

Maybe the problem is with the limit of the local filesystem that uses IndexedDB. I'm not sure what are the limits of IndexedDB.

I had the same thought, but there are larger files than the object file which is causing problems. The original file is larger because it's uncompressed. So it seems like that's not the cause. Very strange that this only happens in Safari though.