Some files do not show * when edited and do not save or autosave, no warning when renaming that buffer data will be lost

[rdhutchins]

Description

I had a .txt file open and was editing it. Autosave was on. After some time I decided to rename the file to .md using the left side file tree. I only noticed sometime later that I had lost all my edits - the file had reverted to the data it had when I first opened it. I turned off autosave and tested again with another file (I added a single line and saved it with ctrl-s). Upon rename, it happened again!! I don't know where the initially loaded text was even saved?? Somewhere in memory? I couldn't find any autosave or backup files in the directory.

Reproducing

See description

Zettlr Version

Stable (most recent version)

Specify version

No response

Installation Method

From the Website or GitHub

Your Platform

• Windows
• macOS
• Linux

Architecture

• Intel 64bit
• ARM 64bit

Operating System Version

Debian 12

Additional Information

I'm really sad :(

[boring-cyborg]

Thanks so much for opening up your first issue here on the repository! 🎉
We would like to warmly welcome you to the community behind the app! ☺️
We'll check in soon and have a look at your issue. In the meantime, you can
check your issue and make sure it aligns with our contribution guidelines!
Here's the comprehensive list:

NOTE: Please do not share screen captures of buggy behavior on YouTube.
If you have uploaded a video on YouTube and linked it already, don't
worry! But, we would like to ask you to remove the video from YouTube
and upload it directly to GitHub instead, by editing your comment.
Read more here.

Enhancements

An enhancement takes a feature and improves or alters its behaviour. Please
make sure to argue how your proposition will aid non-technical text workers,
and why it can't be emulated easily with other features or apps!

Feature requests

Feature requests introduce whole new features into the app. This requires a
lot of work, so these might be turned down if the implementation costs
supersede the benefits we expect to see from implementing it. Please do not
be disappointed if that happens. It likely has nothing to do with your great
request but simply with us and our missing resources!

You can of course always ask someone to implement this feature, because a PR
with a working new feature has much higher chances of being merged! :)

Bug reports

Please note that one of the main reasons for why bug reports cannot be
addressed is that there's not enough information for us to find and fix the
bug you describe, so make sure you try to pinpoint the bug as close as
possible.

The ideal bug report for us has two qualities:

1. The bug is always reproducible, at least within a certain context.
2. We know exactly what specifically goes wrong, and there is consensus on
   what should happen instead.

Please note that if you encounter behaviour that does not align with your
expectations of what would happen, this might as well be simply intended
behaviour and we need to simply clarify why the behaviour is the way it is.
This is not to be considered a bug and such issues may be closed! Suggest an
enhancement instead!

But now, have a great day and thank you again!

[rdhutchins]

I am using zettlr 3.0.5 which I installed from .deb yesterday.

[rdhutchins]

Ctrl-z/Undo does not work to get the modified text back

[rdhutchins]

Okay it's actually that Ctrl-S/save isn't working. I tried modifying a file, hitting save, closing it and opening it again and the changes are not saved. So my bad for not checking that zettlr actually saved changes but also... What??

[rdhutchins]

The freshly renamed .md files also do not save. However, if I close zettlr and restart, they do save and rename preserves the changes.

So there is some weird edge case where modified text is in the buffer but is not saved either by autosave or manual save and there is no warning this is happening. In terminal all files show as user rw.

[rdhutchins]

after restarting, when a file is modified it has an asterisk by its name and if I try to rename it I get a warning: Cannot rename file: Please save your changes first.

I do not understand why some files were/are NOT showing the asterisk when I edit them in the buffer and do not save.

[rdhutchins]

After closing and reopening zettlr a few times, all of my files are now showing the asterisk when edited and save without trouble. I did not rename all of them; some started working in their original txt file name. I couldn't figure out a pattern for why some worked and others didn't.

[rdhutchins]

I noticed that the * wasn't showing up as I edited the buffer but I thought that was just because of autosave. :(

Maybe related to #4232 ?

[nathanlesage]

I'm really sad :(

I can fully understand this :(

In 3.0.5 (or, rather, 3.0.3 which is the last update that got reasonable changes) there were quite a few smaller bugs across the file saving process, but I think I got rid of most of them. However, it'll be a bit until the next version can be released. That being said, should you encounter this issue more often, feel free to try out the current nightlies, they should be likely comparatively stable already.

See also related issue: #4628

[rdhutchins]

Yes that looks like the same issue. For what it's worth, in my case also these WERE new files to zettlr AND they were created on a non-*nix system (a portable typing device). It's possible they had the other line endings initially and somehow over a few opens zettlr changed them/somehow became able to track changes properly.

[rdhutchins]

ZettlrTest3.txt

See attached. This file was created on a Pomera DM100 writing device. It has 6 lines, 4 with text, 2 are empty/blank lines. I transferred two copies of it to my computer.

For both copies, when opened in Zettlr (Debian linux 3.0.5) -- with autosave off -- Zettlr does not recognize changes to the file when I edit it. No * is displayed.

For one file, I try to manually save anyway with Ctrl-S. The timestamp for that file updates (checked from terminal ls -lh) but the visible file contents do not change (checked with cat).

HOWEVER. After the forced Ctrl-S, the number of bytes on that copy has reduced from 222 to 220. In vim, the original copy does not display any visible end of line characters. But the new copy shows ^M (carriage return) at the end of the 4 lines with text. The two blank lines do NOT show the ^M. I suspect those two carriage returns were removed somehow.

I close Zettlr and open it again.

Now the file that was modified DOES display the * when edited and DOES save the change. After saving, when opened in vim, the ^M line endings on the 4 text lines are also gone.

The file that was NOT modified or forced a Ctrl-S still DOES NOT display the * when edited.

I close Zettlr and open it again.

The file that was NOT modified or forced a Ctrl-S still DOES NOT display the * when edited.

I force a Ctrl-S and observed in terminal that this file too has reduced by 2 bytes.

When Zettlr is closed and reopened, this file now correctly shows the * when edited.

---

Also of note, when I transfer a file from the device that does NOT have ANY EMPTY LINES (i.e. only a single line, or only text lines with no empty lines separating them) it opens and edits in Zettlr normally.

[rdhutchins]

Confirmed with cat -E:

• Original file from the device has ^M$ at the end of each line
• Zettlr does not recognize changes, but after a forced Ctrl-S the text lines remain ^M$ but the blank lines have changed to $
• After closing and reopening Zettlr, changes are recognized and asterisk displays. After Ctrl-S, all line endings have changed to $ (no more ^M)

[rdhutchins]

I ran the latest release I found:
./Zettlr-3.1.0-beta.1-x86_64.AppImage

Now when I try to edit one of these foreign device files, I get the same sync error warning another user reported (see attached). In console the error says:
Error occurred in handler for 'documents-authority': RangeError: Applying change set to a document with the wrong length
at k.apply (/tmp/.mount_ZettlrOD1bTl/resources/app.asar/.webpack/main/index.js:2:2585953)
at v.pushUpdates (/tmp/.mount_ZettlrOD1bTl/resources/app.asar/.webpack/main/index.js:2:1899484)
at /tmp/.mount_ZettlrOD1bTl/resources/app.asar/.webpack/main/index.js:2:1893085
at WebContents. (node:electron/js2c/browser_init:2:78183)
at WebContents.emit (node:events:514:28)

Which is... an improvement... At least you couldn't keep editing and think things were okay.

[rdhutchins]

Oops forgot to add the screenshot of the sync error message.

[rdhutchins]

Tried with the latest nightly from https://nightly.zettlr.com/ (April 8).
It gives the same sync error and same console error.

[nathanlesage]

Heya, sorry that I only respond now. This does give me a hint that maybe we also have to account for the line endings when comparing the actual file contents from disk, so there may be a residual error in the remove change detection code as well. I'll see to when I have the time to go over it once more and dig around.

[Ardhanm]

I get the same "Document out of sync" error.
Here's a possible clue that I just found. It seems to be related (at least partially) to EOL.
I get the error when the files are using windows style line endings (CR LF), imported from another project into Zettlr workspace. As soon as I convert all of them to Linux style (LF), the problem is solved.
Hope this helps.

[dstark]

I get the error when the files are using windows style line endings (CR LF), imported from another project into Zettlr workspace. As soon as I convert all of them to Linux style (LF), the problem is solved.

On Windows 11 with Zettlr 3.1.1, the issue also seems to abate if the content of the "Document out of sync" file is copied into a newly created file. That's less ideal as a fix the more files are involved but may also provide a clue.

[nathanlesage]

I get the same "Document out of sync" error. Here's a possible clue that I just found. It seems to be related (at least partially) to EOL. I get the error when the files are using windows style line endings (CR LF), imported from another project into Zettlr workspace. As soon as I convert all of them to Linux style (LF), the problem is solved. Hope this helps.

Does this refer to the 3.1.1? Because I specifically fixed that particular issue for the newest release.

[dstark]

Does this refer to the 3.1.1?

@nathanlesage, yes, sorry, that seems to be correct with Zettlr 3.1.1 on both Windows 10 and 11. A screenshot is below.

Also, on trying to import a single file, the Import directory dialog box first appears. On pressing Select folder, the Open file dialog box appears, allowing for a specific file to be selected for import.

[nathanlesage]

I'll have to take another look. For 3.1.1 I fixed an issue from 3.0.5 in that line endings were improperly detected, but for 3.1.1 I've seen the file system watchdog spitting out spurious messages, so I'm currently a bit afraid that chokidar might … choke. But I need to see a bit more info before I can start looking into it. I'm taking any feedback that helps resolve this issue!

[dstark]

I'll have to take another look.

Thanks so much!

[dstark]

I'm taking any feedback that helps resolve this issue!

Occasionally, I'm also receiving the same notice with *.md files that are already loaded in Zettlr (3.1.1). My best guess is that this occurs when I use a file on one machine, close it out, and then pick up working on it on another machine at another time. It hasn't seemed regular enough to pin down exactly why and when it always happens, but it definitely is something I'm running into every now and again.

And … I'm not sure if it's at all related, but the "Duplicate file" option in the right-click menu seems not to be doing anything at the moment—whether with one of the files raising this error or with any other.

[dstark]

I need to see a bit more info before I can start looking into it. I'm taking any feedback that helps resolve this issue!

@nathanlesage I just opened a conflicted Markdown file from the GitHub desktop application in Zettlr to resolve the conflicts. When the file attempted to auto-save, I got the same error. The log shows the following:

[07:08:29] [Error] [R] [Main Window (e0308453-5a3b-446e-ab3e-e17907c90a94)] Pushing updates failed: Error invoking remote method 'documents-authority': RangeError: Applying change set to a document with the wrong length Error: Error invoking remote method 'documents-authority': RangeError: Applying change set to a document with the wrong length (index.js:2)

I'm not sure if that's helpful at all but wanted to pass it along just in case.

[dstark]

[07:08:29] [Error] [R] [Main Window (e0308453-5a3b-446e-ab3e-e17907c90a94)] Pushing updates failed: Error invoking remote method 'documents-authority': RangeError: Applying change set to a document with the wrong length Error: Error invoking remote method 'documents-authority': RangeError: Applying change set to a document with the wrong length (index.js:2)

@nathanlesage, I just got this same error (minus the initial bit, which I suspect is a window/document identifier (? - [12:21:01] [Error] [R] [Main Window (b62ad328-54a3-41f1-a736-50384d697965)]) when picking up on a Windows 10 machine work that I'd left off on a Windows 11 machine. Zettlr 3.1.1 on both.

After another error like this, the log also includes the following:

[12:35:48] [Warning] [R] [Main Window (b62ad328-54a3-41f1-a736-50384d697965)] Could not apply delta, either local or remote not available [object Proxy] undefined 74536b19-1a8c-4223-b94c-89fefe285ca5 (index.js:2)
[12:35:48] [Warning] [R] [Main Window (b62ad328-54a3-41f1-a736-50384d697965)] Could not apply delta, either local or remote not available [object Proxy] undefined 74536b19-1a8c-4223-b94c-89fefe285ca5 (index.js:2)

Before opening Zettlr on the Windows 10 machine, I had pushed changes to GitHub from the Windows 11 machine and then pulled the changes down on the Windows 10 machine.

But on trying to edit another file, the same (visual) error message in Zettlr leaves only the following in the log:

[12:29:20] [Info] [R] [Main Window (b62ad328-54a3-41f1-a736-50384d697965)] null (index.js:2)

[nathanlesage]

Was Zettlr running while you invoked the git commands? There are still some issues when git changed many files at once.

[dstark]

Was Zettlr running while you invoked the git commands?

No. I updated via GitHub only before opening and after closing Zettlr.

[dstark]

@nathanlesage, when coming across this error, I had the thought to try

1. Closing all files in the same workspace as the file that isn't saving.
2. Unload the workspace.
3. Restart Zettlr.
4. Reload the workspace. And
5. Attempt to edit the file again.

The error I get then is identical (besides the main window hash) to this one.

I've then tried to

• Repeat steps 1--2.
• Delete the workspace's .ztr-directory file. And
• Repeat steps 3--5.

I get the same error and ones like the first two ("Could not apply delta …") quoted here.

I've then tried to

• Repeat steps 1--2.
• Ensure there's no .ztr-directory file in the folder.
• Create a copy of the non-saving file via Explorer (i.e., so the directory has both File.md and File - Copy.md).
• Repeat steps 3--4. And
• Attempt to edit either the original (File.md) or new (File - Copy.md) file.

Here again, the same kinds of errors occur with both files. And if I try the option to Duplicate file either on the non-saving files (or seemingly any others?), I get the error

[07:03:50] [Error] [Application] Error received while running command: this._app.documents.getOpenDirectory is not a function | Native Error: TypeError; this._app.documents.getOpenDirectory is not a function Stack Trace: TypeError: this._app.documents.getOpenDirectory is not a function -->     at h.run ([path-to-my-Zettlr-installation]\resources\app.asar\.webpack\main\index.js:2:1826263) -->     at j.run ([path-to-my-Zettlr-installation]\resources\app.asar\.webpack\main\index.js:2:1843392) -->     at [path-to-my-Zettlr-installation]\resources\app.asar\.webpack\main\index.js:2:1842197 -->     at WebContents.<anonymous> (node:electron/js2c/browser_init:2:78397) -->     at WebContents.emit (node:events:514:28)

So, it seems the only way to make Zettlr "happy" with the file again and render it save-able is to create a brand new file and copy contents into it. This seems to work if one creates the file either within Zettlr or within Explorer.

I'm not sure if any of this provides useful information or clues, but I thought it might be useful to try making a workspace look to Zettlr as much as possible like something it had never seen before and then seeing what happened.

(Interestingly, if I copy a whole directory and thus have both Original Directory and New Directory and I try to load New Directory as a workspace while I also have Original Directory, Zettlr replaces Original Directory's workspace with one for New Directory rather than loading them both at the same time.)

[dstark]

But on trying to edit another file, the same (visual) error message in Zettlr leaves only the following in the log:

[12:29:20] [Info] [R] [Main Window (b62ad328-54a3-41f1-a736-50384d697965)] null (index.js:2)

@nathanlesage, I'm not sure this is actually true. On further examination, it looks like--when an entry like this appears--another error of the "pushing updates failed" variety follows. But for some reason, in that circumstance, the "pushing updates failed" error is just delayed in getting written to the log.

I've also seen a new error type I hadn't noticed before (again with the same GUI message):

[11:41:36] [Error] [R] [Main Window (b62ad328-54a3-41f1-a736-50384d697965)] RangeError: Invalid line number 5 in 1-line document (index.js:2)

In this case, the line number doesn't seem to correspond to any of the edits I try to make (sometimes before and sometimes after the line number shown in the error).

Not sure if any of this is useful but wanted to pass it along in case so. Thanks so much for looking into this issue!

[dstark]

@nathanlesage, I think I may have come across a series of steps to be able to consistently reproduce at least some variety of this error on Windows 10:

1. Create a new file in Zettlr, and add some text. Ensure there aren't any blank lines in the document. Single line breaks are okay, but no full paragraphs.
2. Save the file, and close Zettlr.
3. Open the file in another text editor. Add text in a new full paragraph (i.e., with a blank line preceding it). Save the file, and close the text editor.
4. Reopen Zettlr, reopen the file, and try to make a change. The "Document out of sync" warning appears.
5. Close Zettlr.
6. Reopen the file in the other text editor used in step 3 above. Remove the blank line. You can leave the text on its own new line. It just needs to not be in a full paragraph by itself.
7. Save the file and close the text editor.
8. Reopen Zettlr, reopen the file, and try to make a change. Zettlr will save the document as expected.

Zettlr doesn't have an issue with files that have blank lines if I've created the file with that installation of Zettlr (e.g., on that machine). But if the blank lines have been added (or the file with them has been created) elsewhere (e.g., in another text editor or in another installation of Zettlr on another machine), this seems to reliably reproduce the "Document out of sync" GUI error in Zettlr 3.1.1.

[rdhutchins]

Yes, that is what I noticed. Files with text and line breaks but NO blank lines between paragraphs open fine in Zettlr even if created on the other device (each line has text with \r\n endings and on save Zettlr converts to \n endings). But if there is a blank line (\r\n only on a line) Zettlr chokes. There must be some special casing for blank lines somewhere, maybe they're hard coded as 1 character but in this case they're 2?

[dstark]

@rdhutchins and @Ardhanm, thanks so much for your thoughts. Adding these to the pile, the issue---at least in my case---seems to derive from two things:

1. Using an application other than Zettlr to test modifications of a file in Windows (e.g., here). It looks like that application was introducing Windows-style line breaks into files that Zettlr wrote out with Unix-style line breaks and that Zettlr wasn't able to handle the conversion of Windows-style breaks upon reopening the file.
2. Using GitHub to store project repositories. On Windows, Git seems to default to converting Unix-style line breaks to Windows-style breaks whenever commits get pushed up to GitHub. That meant I was only ever committing files with Unix-style breaks (so Zettlr was fine) but pulling down files on my other machine that had, in the meantime, had their breaks converted to Windows-style, which Zettlr doesn't like. On doing, git config --global core.autocrlf false and then—for each of the current repositories—git rm -rf --cached . and git reset --hard HEAD (per this guide), this seems (tentatively but hopefully) to resolve the issue—as long as I don't do the first thing above.

[dstark]

2. On doing, git config --global core.autocrlf false and then—for each of the current repositories—git rm -rf --cached . and git reset --hard HEAD (per this guide), this seems (tentatively but hopefully) to resolve the issue—as long as I don't do the first thing above.

Actually, this isn't quite sufficient. Git's core.eol apparently defaults to native (e.g., here), which means that CRLF smuggles its way in when running with Git on Windows. So, on Windows, one has to do both
git config --global core.autocrlf false and git config --global core.eol lf on each Windows machine hooked up to the repository to stop that machine from introducing CRLFs that Zettlr will then dislike.

It may not be news to anyone else, but in case it's useful to someone who stumbles upon this thread, you should be able to check each local repository to see if any files have CRLF endings with git ls-files --eol. And if you see CRLF endings shown for any *.md files, you can do

git rm --cached -r .
git reset --hard
git ls-files --eol

And that should show that Git has converted the CRLFs Windows prefers to the LFs Zettlr expects (rather than needing to copy and paste contents into new files and save those back over old files to set things working again).

As an interesting (and related?) aside, Zettlr seems to handle just fine other file types (e.g., *.json) that have CRLFs. But it's mainly (or solely?) *.md files that cause the error being discussed in this thread.