test(core): verify Unicode and ICU versions cross-platform 🙀 #11418

srl295 · 2024-05-11T01:03:02Z

load versions from node and ICU4C++ and Blocks.txt
validate Node version against package.json's engine
for now, don't complain if ICU4C++ mismatches, but DO check Unicode versions.

Fixes: #10183

@keymanapp-test-bot skip

keymanapp-test-bot · 2024-05-11T01:03:10Z

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

srl295 · 2024-05-11T01:04:18Z

Example output

= test_all
== load_json loading tests/unit/ldml/nodeversions.json
== load_json loading ../../../../package.json
== test_unicode_versions
cxx_icu	73.1
cxx_icu_unicode	15.0
node_icu_unicode	15.0
node	18.16.0
node_icu	72.1
node_engine	^18.x
block_unicode_ver	15.1.0.txt

srl295 · 2024-05-13T14:45:45Z

So linux is using ICU from the environment, which only supports Unicode 14 not 15.

  stdout:
20:08:09   = test_all
20:08:09   == load_json loading tests/unit/ldml/nodeversions.json
20:08:09   == load_json loading ../../../../package.json
20:08:09   == test_unicode_versions
20:08:09   cxx_icu  70.1
20:08:09   cxx_icu_unicode  14.0
20:08:09   node_icu_unicode  15.0
20:08:09   node  18.17.1
20:08:09   node_icu  73.1
20:08:09   node_engine  ^18.x
20:08:09   block_unicode_ver  15.1.0.txt
20:08:09   stderr:
20:08:09   Test failed at ../../../tests/unit/ldml/test_unicode.cpp:110:
20:08:09   expected: 14
20:08:09   actual:   15

srl295 · 2024-05-13T23:17:16Z

So linux is using ICU from the environment, which only supports Unicode 14 not 15.

  stdout:
20:08:09   = test_all
20:08:09   == load_json loading tests/unit/ldml/nodeversions.json
20:08:09   == load_json loading ../../../../package.json
20:08:09   == test_unicode_versions
20:08:09   cxx_icu  70.1
20:08:09   cxx_icu_unicode  14.0
20:08:09   node_icu_unicode  15.0
20:08:09   node  18.17.1
20:08:09   node_icu  73.1
20:08:09   node_engine  ^18.x
20:08:09   block_unicode_ver  15.1.0.txt
20:08:09   stderr:
20:08:09   Test failed at ../../../tests/unit/ldml/test_unicode.cpp:110:
20:08:09   expected: 14
20:08:09   actual:   15

I'm going to take out the assert for the cxx_icu version, for now.

srl295 · 2024-05-14T15:06:16Z

all working but wasm

- load version data from node.js, Blocks.txt, and ICU4C - support wasm: copy package.json, nodeversions.json and Blocks.txt into the keyboard area so that they can be mounted under wasm also: - rename 'fallback' macro to KMN_FALLBACK to not conflict with hedley in utfcodec.hpp - fix ambiguous path type in tests Fixes: #10183

- incorporate change from #11483 Fixes: #10183

srl295 · 2024-05-23T22:44:31Z

OK ptal, web unrelated?

mcdurdin

Looks fine, just small nits

mcdurdin · 2024-05-24T02:49:45Z

core/tests/unit/ldml/meson.build

 # Build ldml test executable

+keyboard_build_path = join_paths(meson.current_build_dir(),'keyboards')


Suggested change

keyboard_build_path = join_paths(meson.current_build_dir(),'keyboards')

keyboard_build_path = meson.current_build_dir() / 'keyboards'

It's cleaner to use / now in meson path construction

mcdurdin · 2024-05-24T02:50:48Z

core/tests/unit/ldml/meson.build

@@ -94,14 +120,34 @@ if cpp_compiler.get_id() == 'emscripten'
  normalization_tests_flags += ['-lnodefs.js', '-sEXPORTED_RUNTIME_METHODS=[\'UTF8ToString\']']
 endif

-t = executable('test_context_normalization',
+tc = executable('test_context_normalization',


Suggested change

tc = executable('test_context_normalization',

test_context_normalization = executable('test_context_normalization',

Can we use the full process name as the variable name? Makes future maintenance so much easier

mcdurdin · 2024-05-24T02:50:58Z

core/tests/unit/ldml/meson.build

+
+# Build and run additional test_unicode test
+
+u = executable('test_unicode', 'test_unicode.cpp',


Suggested change

u = executable('test_unicode', 'test_unicode.cpp',

test_unicode = executable('test_unicode', 'test_unicode.cpp',

Ditto

Fixes: #10183 Co-authored-by: Marc Durdin <marc@durdin.net>

mcdurdin · 2024-05-26T22:50:08Z

@ermshiperete the API Verification build is failing, any ideas why? Log is inconclusive.

ermshiperete · 2024-05-27T10:29:25Z

@ermshiperete the API Verification build is failing, any ideas why? Log is inconclusive.

Ignore the API Verification for now. I thought it would work, but I must be overlooking something and it's still not working. I'll add a step to always greenbar until that's solved. Sorry for the noise.

ermshiperete · 2024-05-27T10:32:31Z

Example output

= test_all
== load_json loading tests/unit/ldml/nodeversions.json
== load_json loading ../../../../package.json
== test_unicode_versions
cxx_icu	73.1
cxx_icu_unicode	15.0
node_icu_unicode	15.0
node	18.16.0
node_icu	72.1
node_engine	^18.x
block_unicode_ver	15.1.0.txt

I'm wondering if it would be better to change the order of the output so that all ICU versions are together, all Unicode versions are together etc. ? That way it would be easier to spot which versions should be the same.

srl295 · 2024-05-27T13:25:05Z

Example output
= test_all
== load_json loading tests/unit/ldml/nodeversions.json
== load_json loading ../../../../package.json
== test_unicode_versions
cxx_icu	73.1
cxx_icu_unicode	15.0
node_icu_unicode	15.0
node	18.16.0
node_icu	72.1
node_engine	^18.x
block_unicode_ver	15.1.0.txt
I'm wondering if it would be better to change the order of the output so that all ICU versions are together, all Unicode versions are together etc. ? That way it would be easier to spot which versions should be the same.

Good idea. Maybe with some explanatory text, even.

I think I'll do this as a separate PR since this stack is getting longer.

keyman-server · 2024-05-27T18:05:20Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

srl295 · 2024-05-27T21:45:04Z

Example output
= test_all
== load_json loading tests/unit/ldml/nodeversions.json
== load_json loading ../../../../package.json
== test_unicode_versions
cxx_icu	73.1
cxx_icu_unicode	15.0
node_icu_unicode	15.0
node	18.16.0
node_icu	72.1
node_engine	^18.x
block_unicode_ver	15.1.0.txt
I'm wondering if it would be better to change the order of the output so that all ICU versions are together, all Unicode versions are together etc. ? That way it would be easier to spot which versions should be the same.

#11572

keyman-server · 2024-05-29T05:31:33Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

keyman-server · 2024-05-30T00:56:49Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

keyman-server · 2024-05-30T21:40:54Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

keyman-server · 2024-05-31T19:21:02Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

keyman-server · 2024-05-31T23:06:30Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

keyman-server · 2024-05-31T23:24:39Z

Changes in this pull request will be available for download in Keyman version 18.0.45-alpha

srl295 self-assigned this May 11, 2024

keymanapp-test-bot bot added this to the A18S2 milestone May 11, 2024

github-actions bot added the core/ Keyman Core label May 11, 2024

srl295 changed the title ~~Chore/common/10183-unicode-version~~ chore(core): test for Unicode version 🙀 May 11, 2024

srl295 changed the base branch from master to chore/core/8800-cxx17 May 13, 2024 23:14

srl295 force-pushed the chore/common/10183-unicode-version branch from c7b51ba to d3f8e07 Compare May 13, 2024 23:16

github-actions bot added the chore label May 13, 2024

Base automatically changed from chore/core/8800-cxx17 to master May 14, 2024 03:35

srl295 force-pushed the chore/common/10183-unicode-version branch from 490b510 to e5ee26a Compare May 23, 2024 14:33

srl295 changed the title ~~chore(core): test for Unicode version 🙀~~ test(core): verify Unicode and ICU versions cross-platform 🙀 May 23, 2024

srl295 added test Any acceptance test issue and removed chore labels May 23, 2024

test(core): build fix for test_unicode

f3a80f5

- incorporate change from #11483 Fixes: #10183

srl295 marked this pull request as ready for review May 23, 2024 22:44

srl295 requested review from mcdurdin and rc-swag as code owners May 23, 2024 22:44

mcdurdin requested changes May 24, 2024

View reviewed changes

test(core): update per review comments

e48f6c3

Fixes: #10183 Co-authored-by: Marc Durdin <marc@durdin.net>

srl295 requested a review from mcdurdin May 24, 2024 03:19

srl295 and others added 2 commits May 23, 2024 22:20

Merge branch 'master' into chore/common/10183-unicode-version

c39040f

Merge branch 'master' into chore/common/10183-unicode-version

554c27d

mcdurdin modified the milestones: A18S2, A18S3 May 24, 2024

mcdurdin approved these changes May 26, 2024

View reviewed changes

srl295 merged commit 1700953 into master May 27, 2024
17 of 18 checks passed

srl295 deleted the chore/common/10183-unicode-version branch May 27, 2024 13:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(core): verify Unicode and ICU versions cross-platform 🙀 #11418

test(core): verify Unicode and ICU versions cross-platform 🙀 #11418

srl295 commented May 11, 2024 •

edited

keymanapp-test-bot bot commented May 11, 2024 •

edited

srl295 commented May 11, 2024

srl295 commented May 13, 2024

srl295 commented May 13, 2024

srl295 commented May 14, 2024

srl295 commented May 23, 2024

mcdurdin left a comment

mcdurdin May 24, 2024

mcdurdin May 24, 2024

mcdurdin May 24, 2024

mcdurdin commented May 26, 2024

ermshiperete commented May 27, 2024

ermshiperete commented May 27, 2024

srl295 commented May 27, 2024

keyman-server commented May 27, 2024

srl295 commented May 27, 2024

keyman-server commented May 29, 2024

keyman-server commented May 30, 2024

keyman-server commented May 30, 2024

keyman-server commented May 31, 2024

keyman-server commented May 31, 2024

keyman-server commented May 31, 2024

		# Build ldml test executable

		keyboard_build_path = join_paths(meson.current_build_dir(),'keyboards')

	keyboard_build_path = join_paths(meson.current_build_dir(),'keyboards')
	keyboard_build_path = meson.current_build_dir() / 'keyboards'

	tc = executable('test_context_normalization',
	test_context_normalization = executable('test_context_normalization',


		# Build and run additional test_unicode test

		u = executable('test_unicode', 'test_unicode.cpp',

	u = executable('test_unicode', 'test_unicode.cpp',
	test_unicode = executable('test_unicode', 'test_unicode.cpp',

test(core): verify Unicode and ICU versions cross-platform 🙀 #11418

test(core): verify Unicode and ICU versions cross-platform 🙀 #11418

Conversation

srl295 commented May 11, 2024 • edited

keymanapp-test-bot bot commented May 11, 2024 • edited

User Test Results

Test Artifacts

srl295 commented May 11, 2024

srl295 commented May 13, 2024

srl295 commented May 13, 2024

srl295 commented May 14, 2024

srl295 commented May 23, 2024

mcdurdin left a comment

Choose a reason for hiding this comment

mcdurdin May 24, 2024

Choose a reason for hiding this comment

mcdurdin May 24, 2024

Choose a reason for hiding this comment

mcdurdin May 24, 2024

Choose a reason for hiding this comment

mcdurdin commented May 26, 2024

ermshiperete commented May 27, 2024

ermshiperete commented May 27, 2024

srl295 commented May 27, 2024

keyman-server commented May 27, 2024

srl295 commented May 27, 2024

keyman-server commented May 29, 2024

keyman-server commented May 30, 2024

keyman-server commented May 30, 2024

keyman-server commented May 31, 2024

keyman-server commented May 31, 2024

keyman-server commented May 31, 2024

srl295 commented May 11, 2024 •

edited

keymanapp-test-bot bot commented May 11, 2024 •

edited