-
-
Notifications
You must be signed in to change notification settings - Fork 525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Common Lisp trivial-utf-8 error: Invalid byte at start of character: 0xFC #1562
Comments
Can't look deeper now, but trying to decode UTF-16 with UTF-8 would produce this problem, yes. Did you specify the --encoding option? |
@svantevonerichsen6906 Thank you for your comment, I tried it, but looks like in PostgreSQL to PostgreSQL mode pgloader does not allow this parameter. When I add it in command line, I get following messages and pgloader stops:
And if I try to add it into WITH part, pgloader complains on config file parsing:
I showed only last 2 lines from WITH part which it was able to parse, next one is "encoding" and it fails on it because it is not among expected strings. So unfortunately it looks like "encoding" cannot help here. BTW, I used FreeTDS library with "client charset = UTF-8" so it should convert data to UTF-8 - it had been discussed already on StackOverflow. So it is surprising for me that Lisp seems to fail. Do you know if ODBC would work better here? |
Hi guys, I have encounter a problem which seems to come from trivial-utl-8 lisp library. At least I cannot find this error message anywhere in pgloader code or in PostgreSQL code. Only in this lisp library - https://github.com/fukamachi/trivial-utf-8/blob/master/trivial-utf-8.lisp#L110 - but this code seems to be abandoned for 13 years.
I am using pgloader to copy data in parallel from foreign tables created by tds_fdw extension, into target PostgreSQL tables. In PostgreSQL all works as expected. I can select data from FDW table, I can do INSERT INTO target_table SELECT * FROM fdw_table, all works. But when I start pgloader to copy data from all tables, it fails on one particular table with an error message - "A thread failed with error: Invalid byte at start of character: 0xFC" - see details below.
Underlying data in source database are encoded in UTF-16. Tds_fdw and PostgreSQL process data without problems. But looks like lisp library fails. Maybe someone would have some idea what to do with it?
Thank you very much.
did you test a fresh compile from the source tree?
yes, no change
did you search for other similar issues?
Yes, this seems to be a very rare issue coming from lisp itself.
how can I reproduce the bug?
Underlying data in source database are encoded in UTF-16. Tds_fdw and PostgreSQL processes data without problems. But lisp library fails.
pgloader output you obtain
data that is being loaded, if relevant
Cannot past client's data here
How the data is different from what you expected, if relevant
Lisp library fails on reading data, although in PostgreSQL itself all works.
The text was updated successfully, but these errors were encountered: