Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundle does not work properly with new Intl ICU translations #300

Closed
bocharsky-bw opened this issue Mar 20, 2019 · 3 comments
Closed

Bundle does not work properly with new Intl ICU translations #300

bocharsky-bw opened this issue Mar 20, 2019 · 3 comments
Labels

Comments

@bocharsky-bw
Copy link
Member

  1. When I extract translations with $ bin/console translation:extract - the command put Intl ICU translations to the standard messages.en.xlf files instead of messages+intl-icu.en.xlf.
  2. Even when I have "Intl ICU" translations in messages+intl-icu.en.xlf - the $ bin/console translation:extract command put those translations into messages.en.xlf anyway, i.e. so I have duplications in both files.
  3. Xliff format does not support + sign in file ID, i.e: <file id="messages+intl-icu.en"> is invalid ID, at least PhpStorm says so:

Screen Shot 2019-03-20 at 22 50 23

4. The Web UI, i.e. `/admin/_trans/app` page does not display translations of `messages+intl-icu.en.xlf` as a separate domain, it displays them in `messages` domain. But if you edit those messages, they popup in messages.en.xlf 5. Probably something else...
@Guite
Copy link
Contributor

Guite commented Jan 23, 2020

Point 2 is solved by php-translation/symfony-storage#49 and symfony/symfony#35430

Point 1 is still a huge problem though. It would be great if the extractor could be enforced to append the +intl-icu suffix to all domains.

@tristanbes
Copy link

tristanbes commented Jan 26, 2020

Hey @Guite ; Just came accross the issue after discovering that translating a plural string has changed starting from Symfony 4.2

The problem is that it will render the string "as is" without the plural form
Outputs:

{COUNT, PLURAL, =0 {THERE ARE NO APPLES} ONE {THERE IS ONE APPLE...} OTHER {THERE ARE # APPLES!}}

and I didn't find in the documentation if there is any specific steps that I need to do to achieve what is done in the Symfony documentation (https://symfony.com/doc/4.4/translation/message_format.html#pluralization)

Steps to reproduce

// messages.en.xlf updated from the web-ui
<?xml version="1.0" encoding="utf-8"?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:2.0" version="2.0" srcLang="fr" trgLang="en">
  <file id="messages.en">
    <unit id="bu5PltH" name="example">
      <notes>
        <note category="file-source" priority="1">app/templates/ex/show.html.twig:219</note>
        <note category="state" priority="1">new</note>
      </notes>
      <segment>
        <source>example</source>
        <target>{count, plural, =0 {There are no apples} one {There is one apple...} other {There are # apples!}}</target>
      </segment>
    </unit>
{{ "example"|trans({'%count%': 3}) }}

snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
It will allow us to control if we want to use new ICU format (with
suffixed domain with "+intl-icu") or format.

Configuration has normalizer (making sure string is in lower case) and
validator (making sure it's a string and either "" or "icu").

Setting `new_message_format: ""` we'll end up with new translations in
container like: {domain}.{locale}.{ext}.
Setting `new_message_format: "icu"` we'll end up with new translations
in container like: {domain}+intl-icu.{locale}.{ext}.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
Add new argument to converter methods to make sure we're able to
recognize existing translations (and use correct domain - either
standard or ICU format).
In case translation is new we're going to use predefined format (using
new configuration option new_message_format).

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
Let the default configuration overlap with Symfony configuration option
default value.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
Because of MessageCatalogue interface limitation we're using NSA to
access raw messages data.
Additionally because defines() method uses isset() were force to check
raw messages data directly to check if key exits (translations from
source collection are all set to NULL).

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 24, 2021
We have:

1. Source message domain.
2. Target message domain.
3. Result message domain (this is tied to 2 of the above).

This solution assumes we'd always like to use ICU format (result message
domain) if we use it in any of source or target catalogue.

Additionally because of default NULL values in source catalogue
(internal extractor design to set NULL for "target" when creating
messages catalogue from source collection) we have to check if desired
catalogue's messages field has a key (within requested domain) or not.
Using MessageCatalogue::defines() would return false for catalogue
without translation (NULL value).

Lastly - make sure internal messages field utilised result message
domain for consistency with result field.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Aug 30, 2021
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
It will allow us to control if we want to use new ICU format (with
suffixed domain with "+intl-icu") or format.

Configuration has normalizer (making sure string is in lower case) and
validator (making sure it's a string and either "" or "icu").

Setting `new_message_format: ""` we'll end up with new translations in
container like: {domain}.{locale}.{ext}.
Setting `new_message_format: "icu"` we'll end up with new translations
in container like: {domain}+intl-icu.{locale}.{ext}.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
Add new argument to converter methods to make sure we're able to
recognize existing translations (and use correct domain - either
standard or ICU format).
In case translation is new we're going to use predefined format (using
new configuration option new_message_format).

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
Let the default configuration overlap with Symfony configuration option
default value.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
Because of MessageCatalogue interface limitation we're using NSA to
access raw messages data.
Additionally because defines() method uses isset() were force to check
raw messages data directly to check if key exits (translations from
source collection are all set to NULL).

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
We have:

1. Source message domain.
2. Target message domain.
3. Result message domain (this is tied to 2 of the above).

This solution assumes we'd always like to use ICU format (result message
domain) if we use it in any of source or target catalogue.

Additionally because of default NULL values in source catalogue
(internal extractor design to set NULL for "target" when creating
messages catalogue from source collection) we have to check if desired
catalogue's messages field has a key (within requested domain) or not.
Using MessageCatalogue::defines() would return false for catalogue
without translation (NULL value).

Lastly - make sure internal messages field utilised result message
domain for consistency with result field.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Jan 28, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
It will allow us to control if we want to use new ICU format (with
suffixed domain with "+intl-icu") or format.

Configuration has normalizer (making sure string is in lower case) and
validator (making sure it's a string and either "" or "icu").

Setting `new_message_format: ""` we'll end up with new translations in
container like: {domain}.{locale}.{ext}.
Setting `new_message_format: "icu"` we'll end up with new translations
in container like: {domain}+intl-icu.{locale}.{ext}.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
Add new argument to converter methods to make sure we're able to
recognize existing translations (and use correct domain - either
standard or ICU format).
In case translation is new we're going to use predefined format (using
new configuration option new_message_format).

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
Let the default configuration overlap with Symfony configuration option
default value.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
Because of MessageCatalogue interface limitation we're using NSA to
access raw messages data.
Additionally because defines() method uses isset() were force to check
raw messages data directly to check if key exits (translations from
source collection are all set to NULL).

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
We have:

1. Source message domain.
2. Target message domain.
3. Result message domain (this is tied to 2 of the above).

This solution assumes we'd always like to use ICU format (result message
domain) if we use it in any of source or target catalogue.

Additionally because of default NULL values in source catalogue
(internal extractor design to set NULL for "target" when creating
messages catalogue from source collection) we have to check if desired
catalogue's messages field has a key (within requested domain) or not.
Using MessageCatalogue::defines() would return false for catalogue
without translation (NULL value).

Lastly - make sure internal messages field utilised result message
domain for consistency with result field.

php-translation#300
snpy added a commit to kram-soft/symfony-bundle that referenced this issue Nov 24, 2022
bocharsky-bw pushed a commit that referenced this issue Jan 23, 2023
* Remove both "normal" and ICU domain from messages set

#300

* Add new configuration for new message format

It will allow us to control if we want to use new ICU format (with
suffixed domain with "+intl-icu") or format.

Configuration has normalizer (making sure string is in lower case) and
validator (making sure it's a string and either "" or "icu").

Setting `new_message_format: ""` we'll end up with new translations in
container like: {domain}.{locale}.{ext}.
Setting `new_message_format: "icu"` we'll end up with new translations
in container like: {domain}+intl-icu.{locale}.{ext}.

#300

* Add new field and getter based on recently added new config option

#300

* Make sure importer is ICU format aware

Add new argument to converter methods to make sure we're able to
recognize existing translations (and use correct domain - either
standard or ICU format).
In case translation is new we're going to use predefined format (using
new configuration option new_message_format).

#300

* Make sure new_message_format configuration is defined

Let the default configuration overlap with Symfony configuration option
default value.

#300

* Pass to converter new_message_format configuration

#300

* Make sure importer is aware of ICU domain format

Because of MessageCatalogue interface limitation we're using NSA to
access raw messages data.
Additionally because defines() method uses isset() were force to check
raw messages data directly to check if key exits (translations from
source collection are all set to NULL).

#300

* Let code "breath" a bit

#300

* Make sure we're always aware of ICU domain

#300

* Make sure we're aware of 3 different message domains

We have:

1. Source message domain.
2. Target message domain.
3. Result message domain (this is tied to 2 of the above).

This solution assumes we'd always like to use ICU format (result message
domain) if we use it in any of source or target catalogue.

Additionally because of default NULL values in source catalogue
(internal extractor design to set NULL for "target" when creating
messages catalogue from source collection) we have to check if desired
catalogue's messages field has a key (within requested domain) or not.
Using MessageCatalogue::defines() would return false for catalogue
without translation (NULL value).

Lastly - make sure internal messages field utilised result message
domain for consistency with result field.

#300

* Persist PHP CS Fixer changes

#300

* Remove both "normal" and ICU domain from messages set

#300

* Add new configuration for new message format

It will allow us to control if we want to use new ICU format (with
suffixed domain with "+intl-icu") or format.

Configuration has normalizer (making sure string is in lower case) and
validator (making sure it's a string and either "" or "icu").

Setting `new_message_format: ""` we'll end up with new translations in
container like: {domain}.{locale}.{ext}.
Setting `new_message_format: "icu"` we'll end up with new translations
in container like: {domain}+intl-icu.{locale}.{ext}.

#300

* Add new field and getter based on recently added new config option

#300

* Make sure importer is ICU format aware

Add new argument to converter methods to make sure we're able to
recognize existing translations (and use correct domain - either
standard or ICU format).
In case translation is new we're going to use predefined format (using
new configuration option new_message_format).

#300

* Make sure new_message_format configuration is defined

Let the default configuration overlap with Symfony configuration option
default value.

#300

* Pass to converter new_message_format configuration

#300

* Make sure importer is aware of ICU domain format

Because of MessageCatalogue interface limitation we're using NSA to
access raw messages data.
Additionally because defines() method uses isset() were force to check
raw messages data directly to check if key exits (translations from
source collection are all set to NULL).

#300

* Let code "breath" a bit

#300

* Make sure we're always aware of ICU domain

#300

* Make sure we're aware of 3 different message domains

We have:

1. Source message domain.
2. Target message domain.
3. Result message domain (this is tied to 2 of the above).

This solution assumes we'd always like to use ICU format (result message
domain) if we use it in any of source or target catalogue.

Additionally because of default NULL values in source catalogue
(internal extractor design to set NULL for "target" when creating
messages catalogue from source collection) we have to check if desired
catalogue's messages field has a key (within requested domain) or not.
Using MessageCatalogue::defines() would return false for catalogue
without translation (NULL value).

Lastly - make sure internal messages field utilised result message
domain for consistency with result field.

#300

* Persist PHP CS Fixer changes

#300

* Apply PHP CS fixer automated fix

* Fix main PHPUnit issue

We're now providing getter method name for new field.

* Fix catalogue counter

* Make assertion more wordy

* Update configuration to have passing tests

Add new ICU messages domain.
Make sure configuration is set to before-the-modification configuration.
New messages will not be added to ICU domain.

An additional tests must be run with new option set to ICU.
This however may require functional tests refactor.

* Apply PHP-CS-Fixer suggestion
@bocharsky-bw
Copy link
Member Author

Now can be closed thanks to #452

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants