prefer-lowest fails installing vendors

[emodric]

I have a composer.json file which installs a couple of packages (amongst others) that require twig/twig in different versions: either ^3.0 or ^2.14 || ^3.3. Now while regular composer update works fine and latest twig/twig is installed, using --prefer-lowest just errors out and I can't seem to parse the output to figure out the problem.

In my understanding, Composer should install the lowest possible version that matches ^3.0 or ^2.14 || ^3.3, which would be 3.3.0, but it looks like it tries to install 2.14.0 and fails.

My composer.json:

{
    "minimum-stability": "dev",
    "prefer-stable": true,
    "require": {
        "netgen/layouts-ibexa": "~1.4.0"
    }
}

Output of composer diagnose:

Checking composer.json: WARNING
No license specified, it is recommended to do so. For closed-source software you may use "proprietary" as license.
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking github.com oauth access: OK
Checking disk free space: OK
Checking pubkeys: 
Tags Public Key Fingerprint: 57815BA2 7E54DC31 7ECC7CC5 573090D0  87719BA6 8F3BB723 4E5D42D0 84A14642
Dev Public Key Fingerprint: 4AC45767 E5EC2265 2F0C1167 CBBB8A2B  0C708369 153E328C AD90147D AFE50952
OK
Checking composer version: OK
Composer version: 2.2.6
PHP version: 8.1.2
PHP binary path: /usr/bin/php8.1
OpenSSL version: OpenSSL 1.1.1l  24 Aug 2021
cURL version: 7.74.0 libz 1.2.11 ssl OpenSSL/1.1.1l
zip: extension present, unzip present, 7-Zip present (7z)

When I run this command:

composer update --prefer-lowest

I get the following output:

Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - twig/twig v3.0.0 requires php ^7.2.9 -> your php version (8.1.2) does not satisfy that requirement.
    - twig/twig[v3.0.1, ..., v3.0.3] require php ^7.2.5 -> your php version (8.1.2) does not satisfy that requirement.
    - Root composer.json requires netgen/layouts-ibexa ~1.4.0 -> satisfiable by netgen/layouts-ibexa[1.4.x-dev (alias of dev-master)].
    - netgen/content-browser 1.4.x-dev is an alias of netgen/content-browser dev-master and thus requires it to be installed too.
    - netgen/content-browser-ibexa dev-master requires netgen/content-browser ~1.4.0 -> satisfiable by netgen/content-browser[1.4.x-dev (alias of dev-master)].
    - netgen/content-browser-ibexa 1.4.x-dev is an alias of netgen/content-browser-ibexa dev-master and thus requires it to be installed too.
    - netgen/layouts-ibexa dev-master requires netgen/content-browser-ibexa ^1.4 -> satisfiable by netgen/content-browser-ibexa[1.4.x-dev (alias of dev-master)].
    - netgen/layouts-ibexa 1.4.x-dev is an alias of netgen/layouts-ibexa dev-master and thus requires it to be installed too.
    - netgen/layouts-ibexa dev-master requires ibexa/admin-ui ^4.0 -> satisfiable by ibexa/admin-ui[v4.0.0, v4.0.1, v4.0.2, 4.0.x-dev (alias of dev-main)].
    - ibexa/admin-ui 4.0.x-dev is an alias of ibexa/admin-ui dev-main and thus requires it to be installed too.
    - ibexa/admin-ui[dev-main, v4.0.0, ..., 4.0.x-dev] require twig/twig ^3.0 -> satisfiable by twig/twig[v3.0.0, ..., 3.x-dev].
    - You can only install one version of a package, so only one of these can be installed: twig/twig[v1.34.0, ..., 1.x-dev, v2.4.0, ..., 2.x-dev, v3.0.0, ..., 3.x-dev].
    - netgen/content-browser dev-master requires twig/twig ^2.14 || ^3.3 -> satisfiable by twig/twig[v2.14.0, ..., 2.x-dev, v3.3.0, ..., 3.x-dev].
    - Conclusion: don't install twig/twig v2.14.0 (conflict analysis result)

And I expected this to happen:

I expected to have vendors installed with minimum supported versions.

[Seldaek]

Yeah it looks like it should select twig/twig 3.0.4. for the ibexa/admin-ui dependency as PHP is 8 supported by twig 3.0.4+.

It makes sense that twig 2.14 is invalid.. but I am not sure why it doesn't end up with twig/twig 3.3.0.

Forcing that in the root package makes it work, so it's definitely possible to resolve:

{
    "minimum-stability": "dev",
    "prefer-stable": true,
    "require": {
        "netgen/layouts-ibexa": "~1.4.0",
        "twig/twig": "^3.3"
    }
}

Maybe @naderman has a clue here?

[naderman]

Best guess: Try without the optimizer in case that has a prefer lowest bug? Would have to look when I have some more time otherwise.

[emodric]

@naderman Is this something I could try myself with release version of Composer?

[Seldaek]

@emodric run COMPOSER_POOL_OPTIMIZER=0 composer update --prefer-lowest - I tried it and it seems to indeed fix the issue. So I guess cc @Toflar (sorry :D)

This may be a funky one to debug though because it runs so slow.. But hopefully it can be reproduced with a subset of the packages, assuming it's a simple logic bug in the optimizer.

[emodric]

@Seldaek Thanks, using COMPOSER_POOL_OPTIMIZER=0 works.

If I can help with the testing potential fixes for this, don't hesitate to @ me :)

[Toflar]

I can help debugging next week :)

[Toflar]

After 3 hours of debugging the issue turned out to be rather simple to understand. The PoolOptimizer simply keeps only 2.14.0 for ^2.14 || ^3.3. I've exctracted the problem in a test case in #10579 and will try to solve it there :) I need to dig a bit deeper though.

[Seldaek]

Fixed by #10579