Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide support for non utf-8 encodings #6064

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Utkarsh1308
Copy link
Contributor

coala currently only supports utf-8 encodings. Before merging this
we need to add support for non utf-8 encodings to coala-utils.

Some bears like SpellCheckBear have external linters which only
support utf-8 encodings. To solve this the bears should describe
the encoding it can handle followed by coala creating a temporary
file in the linter's accepted encoding (Converting utf-16 to utf-8)

Some encodings have glyphs which are not in unicode and can't be
transcoded to utf16/32. We should write test to catch such
exceptions.

Closes #3862

For short term contributors: we understand that getting your commits well
defined like we require is a hard task and takes some learning. If you
look to help without wanting to contribute long term there's no need
for you to learn this. Just drop us a message and we'll take care of brushing
up your stuff for merge!

Checklist

  • I read the commit guidelines and I've followed
    them.
  • I ran coala over my code locally. (All commits have to pass
    individually.
    It is not sufficient to have "fixup commits" on your PR,
    our bot will still report the issues for the previous commit.) You will
    likely receive a lot of bot comments and build failures if coala does not
    pass on every single commit!

After you submit your pull request, DO NOT click the 'Update Branch' button.
When asked for a rebase, consult coala.io/rebase
instead.

Please consider helping us by reviewing other peoples pull requests as well:

The more you review, the more your score will grow at coala.io and we will
review your PRs faster!

@Utkarsh1308
Copy link
Contributor Author

Utkarsh1308 commented Aug 21, 2019

There is a problem with detect_encoding in decoding utf-32 files which is causing tests for handling utf-32 encoded files to fail.

These should get fixed when https://gitlab.com/coala/coala-utils/merge_requests/99 gets merged

@Utkarsh1308 Utkarsh1308 force-pushed the charset branch 9 times, most recently from 92132d0 to 42becaa Compare August 26, 2019 05:34
@Utkarsh1308
Copy link
Contributor Author

Utkarsh1308 commented Aug 26, 2019

99 got merged. Tests are passing now
I think appveyor is failing because the latest version of coala-utils is not getting installed.

@jayvdb please review

coala currently only supports utf-8 encodings. Before merging this
we need to add support for non utf-8 encodings to coala-utils.

Some bears like SpellCheckBear have external linters which only
support utf-8 encodings. To solve this the bears should describe
the encoding it can handle followed by coala creating a temporary
file in the linter's accepted encoding (Converting utf-16 to utf-8)

Some encodings have glyphs which are not in unicode and can't be
transcoded to utf16/32. We should write test to catch such
exceptions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Add setting charset
2 participants