[Translation] Plural handling in .po files completely broken since 4.3.3

[BjornTwachtmann]

Symfony version(s) affected: >=4.3.3

Description
When attempting to translate entries from .po files that are in plural form, translation fails and the original input string (with substitutions) is returned rather than the translated form.

How to reproduce
The following script and .po file illustrate the problem. They work correctly in symfony/translate 4.3.2 and are broken in >=4.3.3

test.po

msgid "Bug Free"
msgstr "Fehler Frei"

msgid "There is currently 1 bug"
msgid_plural "There are currently %count% bugs"
msgstr[0] "Es gibt derzeit ein Fehler"
msgstr[1] "Es gibt derzeit %count% Fehler"

test.php

<?php
require __DIR__.'/vendor/autoload.php';

$translate = new \Symfony\Component\Translation\Translator('de_DE');
$translate->addLoader('pofile', new \Symfony\Component\Translation\Loader\PoFileLoader());
$pofile = __DIR__.'/test.po';
$translate->addResource('pofile', $pofile, 'de_DE');

// This should translate to German, but just outputs the input string since v4.3.3
echo $translate->trans('There is currently 1 bug') . "\n";
echo $translate->trans('There are currently %count% bugs', ['%count%' => 3]) . "\n";

//This still works fine and outputs German in both cases (singular form)
echo $translate->trans('Bug Free') . "\n";

Possible Solution
This bug appears to have been introduced in #31266 . Reverting that code restores the original behaviour. Honestly I'm very confused as to how this was meant to work. Symfony, post this change, appears to be expecting you to ask for translations of plural forms as follows: $translate->trans('There is currently 1 bug|There are currently %count% bugs', ['%count%' => 3]). This is clearly unacceptable as it a) breaks existing behaviour (which is also standard behaviour across any app that uses .po files) in a patch release, and b) requires the calling code to know details of Symfony's internal representation of a translation.

[nicolas-grekas]

/cc @Stadly in case you want to have a look.

[Stadly]

I'm not entirely convinced that this is a bug. At least the old functionality was not correct, even though it seemed to work correctly in your case. I'll try to come back to it during the weekend.

[Stadly]

From https://symfony.com/doc/current/components/translation/usage.html:

This example illustrates the two different philosophies when creating messages to be translated:

$translator->trans('Symfony is great');
$translator->trans('symfony.great');

In the first method, messages are written in the language of the default locale (English in this case). That message is then used as the "id" when creating translations.

In the second method, messages are actually "keywords" that convey the idea of the message. The keyword message is then used as the "id" for any translations. In this case, translations must be made for the default locale (i.e. to translate symfony.great to Symfony is great).

The key point here is that the message is used as the id for translations.

$translate->trans('There is currently 1 bug') and $translate->trans('There are currently %count% bugs', ['%count%' => 3]) points to two different message ids, not the same message id in singular and plural.

Further down on the page, there is an example with plurals (using the old transChoice):

$translator->transChoice(
    '{0} There are no apples|{1} There is one apple|]1,Inf[ There are %count% apples',
    10,
    [],
    'messages',
    'fr_FR'
);

Either, you have to include both the singular and plural in the message:

echo $translate->trans('There is currently 1 bug|There are currently %count% bugs', ['%count%' => 1]) . "\n";
echo $translate->trans('There is currently 1 bug|There are currently %count% bugs', ['%count%' => 3]) . "\n";

Or you have to use a message keyword, and supply translation files for both English and German:

echo $translate->trans('bug.count', ['%count%' => 1]) . "\n";
echo $translate->trans('bug.count', ['%count%' => 3]) . "\n";

This is just how the Symfony Translation Component is designed to work.

[BjornTwachtmann]

@Stadly, what do the PO file look like for this example? I have separate PO files for all languages and don't see how it would work

echo $translate->trans('bug.count', ['%count%' => 1]) . "\n";
echo $translate->trans('bug.count', ['%count%' => 3]) . "\n";

[Stadly]

Edit

When using bug.count as message id, PO's native support for pluralization cannot be utilized. This would be the PO file:

msgid "bug.count"
msgstr "Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler"

Original comment (for reference)

Good question.

This might work, but I think it would be an invalid PO file. Also, it cannot be created by the PO file dumper:

msgid "bug.count"
msgstr[0] "There is currently 1 bug"
msgstr[1] "There are currently %count% bugs"

This might be better, but does not work currently. Also, it cannot be created by the PO file dumper:

msgid "bug.count"
msgid_plural ""
msgstr[0] "There is currently 1 bug"
msgstr[1] "There are currently %count% bugs"

Maybe the best solution is to use the message id bug.count|. This should work:

echo $translate->trans('bug.count|', ['%count%' => 1]) . "\n";
echo $translate->trans('bug.count|', ['%count%' => 3]) . "\n";

The | indicates that the message should be provided in both singular and plural. Then this would be the PO file:

msgid "bug.count"
msgid_plural ""
msgstr[0] "There is currently 1 bug"
msgstr[1] "There are currently %count% bugs"

[Stadly]

Or make it explicit that the message is pluralizable in the message keyword:

echo $translate->trans('bug.count.singular|bug.count.plural', ['%count%' => 1]) . "\n";
echo $translate->trans('bug.count.singular|bug.count.plural', ['%count%' => 3]) . "\n";

This would be the PO file:

msgid "bug.count.singular"
msgid_plural "bug.count.plural"
msgstr[0] "There is currently 1 bug"
msgstr[1] "There are currently %count% bugs"

[BjornTwachtmann]

@Stadly I'm quite uncomforable with the idea of leaking internal implementation details of symfony translate into my frontend code like that.

The way that symfony translate creates internal representations of the data should be transparent to the calling code, otherwise you just break encapsulation and create a massive amount of work for every user of the framework if the internal details change

[Stadly]

@BjornTwachtmann I completely agree.

The issue is not really with Symfony's internal representation of data, but with converting between the plural formats used by Symfony and PO.

Most translation formats don't have native support for pluralization, and the message id simply maps to a message string in Symfony's pluralization format, such as

bug.count => Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler
There is currently 1 bug|There are currently %count% bugs => Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler

PO, however, has native support for pluralization.

Let's consider this file:

$translate->trans('bug.count', ['%count%' => 1]);
$translate->trans('bug.count.singular|bug.count.plural', ['%count%' => 2])
$translate->trans('There is currently 1 bug|There are currently %count% bugs', ['%count%' => 3]);

If we don't want to utilize PO's native support for pluralization, we can use such a PO file (this is comparable to how most other translation formats work):

msgid "bug.count"
msgstr "Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler"

msgid "bug.count.singular|bug.count.plural"
msgstr "Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler"

msgid "There is currently 1 bug|There are currently %count% bugs"
msgstr "Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler"

In #31266, support for PO's native pluralization format was implemented. So now we also have the option to use such a PO file:

msgid "bug.count"
msgstr "Es gibt derzeit ein Fehler|Es gibt derzeit %count% Fehler"

msgid "bug.count.singular"
msgid_plural "bug.count.plural"
msgstr[0] "Es gibt derzeit ein Fehler"
msgstr[1] "Es gibt derzeit %count% Fehler"

msgid "There is currently 1 bug"
msgid_plural "There are currently %count% bugs"
msgstr[0] "Es gibt derzeit ein Fehler"
msgstr[1] "Es gibt derzeit %count% Fehler"

So all three message id approaches work in both PO translation formats and all other formats. By using bug.count.singular|bug.count.plural or There is currently 1 bug|There are currently %count% bugs, you may (but don't have to) take advantage of PO's native pluralization format. By using bug.count, you cannot utilize PO's native support for pluralization (but it still works fine).

[Stadly]

This issue can probably be closed now.