Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skip UTF-8 BOM also #381

Merged
merged 2 commits into from Jan 28, 2023
Merged

skip UTF-8 BOM also #381

merged 2 commits into from Jan 28, 2023

Conversation

kmuto
Copy link
Contributor

@kmuto kmuto commented Jan 28, 2023

I noticed that reading a UTF-8 encoded file with a BOM causes an error. Unfortunately, major Windows software adds a BOM to UTF-8 files.
UTF-16 support is fixed in #277 , but in addition, skipping the UTF-8 BOM (ef bb bf, https://en.wikipedia.org/wiki/Byte_order_mark ) also solves the problem, I believe.

A sample is attached.

before:

OK(withoutBOM) あ
ERROR(withBOM) toml: line 1: expected '.' or '=', but got '\ufeff' instead

after:

OK(withoutBOM) あ
OK(withBOM) あ

mini.zip

@arp242 arp242 merged commit 1a6ca6e into BurntSushi:master Jan 28, 2023
@arp242
Copy link
Collaborator

arp242 commented Jan 28, 2023

Thanks!

@kmuto
Copy link
Contributor Author

kmuto commented Jan 30, 2023

@arp242
Thanks for the merge!
Have you decided when you plan to release the next one? If you are keeping the releases at 3 month intervals, I think it's about the right time. 😊

@kmuto kmuto mentioned this pull request May 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants