Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default encoding to UTF-8 in JdbcExecutionContextDao #3983

Closed
marschall opened this issue Aug 30, 2021 · 3 comments
Closed

Change default encoding to UTF-8 in JdbcExecutionContextDao #3983

marschall opened this issue Aug 30, 2021 · 3 comments

Comments

@marschall
Copy link
Contributor

JdbcExecutionContextDao serializes as ISO-8859-1

This limits us to the ISO-8859-1 repertoire, even simple things like the Euro sign € are not supported. Honestly it is quite confusing that we convert through a binary encoding at all given that JdbcExecutionContextDao stores as String/VARCHAR/CLOB/....

This would be breaking change for anybody currently using non-ASCII ISO-8859-1 characters.

@marschall marschall added the status: waiting-for-triage Issues that we did not analyse yet label Aug 30, 2021
@fmbenhassine
Copy link
Contributor

This is a good candidate for v5. UTF-8 seems a reasonable default to me (similar to JEP 400: UTF-8 by default, even if this is for a different purpose). What do you think?

This might require increasing the current default value of maxVarCharLength in JobRepositoryFactoryBean and DDL scripts.

@fmbenhassine fmbenhassine added in: core type: enhancement and removed status: waiting-for-triage Issues that we did not analyse yet labels Sep 2, 2021
@fmbenhassine fmbenhassine added this to the 5.0.0 milestone Sep 2, 2021
@marschall
Copy link
Contributor Author

This is a good candidate for v5. UTF-8 seems a reasonable default to me (similar to JEP 400: UTF-8 by default, even if this is for a different purpose). What do you think?

Yes, assuming we want to keep ExecutionContextSerializer byte-based (InputStream/OutputStream) and not switch it to char-based (Reader/Writer). If we switch them to be char-based we can get rid of the entire encoding.

Switching to UTF-8 likely will break code that uses DefaultExecutionContextSerializer as it is unlikely to produce valid UTF-8. We would have to change DefaultExecutionContextSerializer to produce Base64.

This might require increasing the current default value of maxVarCharLength in JobRepositoryFactoryBean and DDL scripts.

We specify the length correctly in char and not byte therefore there should be no change needed.

@fmbenhassine
Copy link
Contributor

fmbenhassine commented May 5, 2022

Switching to UTF-8 should also fix issues like #764 and #795.

@fmbenhassine fmbenhassine changed the title JdbcExecutionContextDao serializes as ISO-8859-1 Change default encoding to UTF-8 in JdbcExecutionContextDao May 17, 2022
@fmbenhassine fmbenhassine modified the milestones: 5.0.0, 5.0.0-M3 May 17, 2022
fmbenhassine added a commit to fmbenhassine/spring-batch that referenced this issue Jun 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants