Chinese font not rendered with the correct font variation

[nowayhaze]

There is a regression that happens between versions 1.0.207 and 1.0.209 (tested with the same text). Chinese characters in 1.0.207 were rendered correctly under the zh-CN mainland standard. By 1.0.209 the same exact text renders improperly.

(This seems to be a very similar issue to that was resolved in Windows Terminal recently microsoft/terminal#4508)

Environment

Joplin version: 1.0.209
Platform: Windows
OS specifics: Windows 10 64-bit Build 18363

Steps to reproduce

1. Enter Chinese text with correct font variation (for example: 遍) in 1.0.207
2. Open up Joplin 1.0.209, and the text shows the wrong font variation.

1.0.207:

1.0.209:

[nowayhaze]

I tried editing the CSS for both joplin and markdown and it seems
the font-variant-east-asian CSS property isn't working on either, maybe the property is ignored or hard-coded in Joplin? If so, the hard-coding must have changed between the two versions.

[laurent22]

I can't replicate on macOS. Just to be sure could you check with 1.0.216? I assume it still happens but worth checking anyway.

[nowayhaze]

Yeah, on 1.0.216 same thing happens.

[nowayhaze]

I checked the same notes synced on the Android app and the characters display correctly. So it seems this problem is limited to Windows right now.

[laurent22]

I don't know enough about this to fix the issue. I've tried setting "font-variant-east-asian" to various values, but that didn't change anything. If you have some idea of what CSS should be applied to fix the issue, please let me know.

[nowayhaze]

Okay, so I tried some more custom CSS, and I have a hacky solution for the rendered-markdown window. The editor window is still unresolved. It looks like part of the problem is the default font. What is the default font Joplin uses for the rendered markdown? Was it changed/hard-coded between the two versions?

Rendered-markdown Window

When I force the font in the the CSS (either in the note or in userstyle.css), it actually does work to change it to the correctly localized variant:

* {
  font-family: Microsoft Yahei UI
  font-variant-east-asian: simplified;
}

Although this doesn't match the default font of Joplin, I could live with this in the rendered window since it's my default font for Chinese in Windows. Latin-based texts don't look very good, but a cumbersome solution is to set a custom <style> tag for each note or section I want to write in Chinese.
So as for as the rendered window is concerned, we have a usable CSS-based solution, but ultimately, based on the Arabic issue #3492, I think somewhere in the code someone messed up or hard-coded a font setting between versions 1.0.207 and 1.0.209. EDIT: I wonder if the default font used now is missing variants or extended character sets, which might simultaneously explain the boxes that appear for the Arabic issue, and that the font-variant-east-asian setting doesn't work unless a new font-family is specified.

Editor Window

Now, there is the editor window to deal with. If I force the editor font family to be "Microsoft Yahei", it screws up the cursor completely in the text. That is, in the following photo, I'm selecting all of and only the Chinese text in the final line. (There are no real spaces at the end of that line.)

This may be an issue related to the fact that Joplin wants only monospaced fonts for the editor.
As mentioned before, because Chinese fonts are not very good for rendering Latin-based texts, even if the cursor problem is resolved, I would rather not have a global Chinese font setting screwing up all the other notes. This is unlike the rendering window solution, where the precise font can be set on a per note basis.

[nowayhaze]

Okay, update. I decided to checkout and build a few of the commits between 1.0.207 and 1.0.209, and I have identified that fa76c5d as the commit when the regression first takes place. Unfortunately, the only change in that commit is upgrading Electron from 7.1.12 to 8.2.5. Which means it's either out of Joplin's hands or something with the fonts is misconfigured for Electron 8. With a cursory search, I have not found any Electron issues directly related to font-variant-east-asian.

[nowayhaze]

Amazing, as of 1.5.7, the all issues seem to be resolved. I know that there were some big UI changes between the versions and when I have some time I will investigate which version it was that fixed the issue.

In the meantime this issue can be closed.

[nowayhaze]

This issue was fixed between versions 1.4.10 and 1.4.11. In the changelog for 1.4.11, "Improved: Upgrade to Electron 10" was listed. This is likely a bug with Electron and not Joplin. I'm not going to investigate further right now to confirm, but possibly this PR was the fix: electron/electron/pull/23247