Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No support for non-utf8 paths #21

Open
wisp3rwind opened this issue Nov 2, 2018 · 1 comment
Open

No support for non-utf8 paths #21

wisp3rwind opened this issue Nov 2, 2018 · 1 comment

Comments

@wisp3rwind
Copy link
Collaborator

beets-alternatives currently does not support non-utf8 paths, while beets does (I didn't really test any of this, that's only guessed from reading the code).

When the Python 3 PR is applied, the only remaining problem should be path storage in the database. The issue is that beets' flexible attributes are treated as sqlite TEXT. While sqlite appears to support arbitrary binary data in its TEXT columns as long as no SQL operations tries to interpret it as actual encoded text, Python's sqlite3.Connection.text_factory interprets the data as unicode strings (grep suggests that bees does not change text_factory from the default).
Whether the default text_factory supports surrogate escapes is not documented (and from a quick look at the cpython code, I couldn't tell either). Thus, supporting such paths can probably not be solved by using a custom beets.dbcore.types.Type subclass, but would require changes to beets itself. Maybe using the backslashreplace error handler with the utf8 codec could also work (which might bring up issues with backwards-compatibility, though).

Personally, I don't really care about non-ut8 paths, but wanted to track this here either way.

@geigerzaehler
Copy link
Owner

I don’t think there is a need for supporting non-UTF8 paths and it looks like implementing it would be hard. I’ll mark this as wontfix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants