Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requirements error says 'but it conflicts with another require' without saying what the problem actually is #9393

Open
joachim-n opened this issue Nov 1, 2020 · 24 comments
Assignees
Labels
Milestone

Comments

@joachim-n
Copy link

My composer.json:

{
    "name": "drupal/git-clone-project",
    "description": "TODO",
    "type": "project",
    "license": "GPL-2.0-or-later",
    "homepage": "https://www.drupal.org/project/drupal",
    "support": {
        "docs": "https://www.drupal.org/docs/user_guide/en/index.html",
        "chat": "https://www.drupal.org/node/314178"
    },
    "repositories": [
        {
            "type": "path",
            "url": "repos/drupal",
            "options": {
                "symlink": true
            }
        },
        {
            "type": "path",
            "url": "repos/drupal/core"
        },
        {
            "type": "path",
            "url": "repos/drupal/composer/Plugin/ProjectMessage"
        },
        {
            "type": "path",
            "url": "repos/drupal/composer/Plugin/VendorHardening"
        },
        {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        }
    ],
    "require": {
        "composer/installers": "^1.9",
        "drupal/core-composer-scaffold": "^8.9",
        "drupal/drupal": "*",
        "drush/drush": "^10.3",
        "symfony/var-dumper": "^5.1"
    },
    "minimum-stability": "dev",
    "prefer-stable": true,
    "config": {
        "sort-packages": true
    },
    "extra": {
        "drupal-scaffold": {
            "locations": {
                "web-root": "web/"
            }
        },
        "installer-paths": {
            "web/core": [
                "type:drupal-core"
            ],
            "web/libraries/{$name}": [
                "type:drupal-library"
            ],
            "web/modules/contrib/{$name}": [
                "type:drupal-module"
            ],
            "web/profiles/contrib/{$name}": [
                "type:drupal-profile"
            ],
            "web/themes/contrib/{$name}": [
                "type:drupal-theme"
            ],
            "drush/Commands/contrib/{$name}": [
                "type:drupal-drush"
            ],
            "web/modules/custom/{$name}": [
                "type:drupal-custom-module"
            ],
            "web/themes/custom/{$name}": [
                "type:drupal-custom-theme"
            ]
        }
    }
}

Additionally need to do the following in the project folder before installing with Composer:

mkdir repos
cd repos
git clone --branch 9.2.x https://git.drupalcode.org/project/drupal.git

Output of composer diagnose:

Checking composer.json: WARNING
require.drupal/drupal : unbound version constraints (*) should be avoided
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking github.com rate limit: OK
Checking disk free space: OK
Checking pubkeys:
Tags Public Key Fingerprint: 57815BA2 7E54DC31 7ECC7CC5 573090D0  87719BA6 8F3BB723 4E5D42D0 84A14642
Dev Public Key Fingerprint: 4AC45767 E5EC2265 2F0C1167 CBBB8A2B  0C708369 153E328C AD90147D AFE50952
OK
Checking composer version: You are not running the latest stable version, run `composer self-update` to update (2.0.3 => 2.0.4)
Composer version: 2.0.3
PHP version: 7.4.2
PHP binary path: /Applications/MAMP/bin/php/php7.4.2/bin/php
OpenSSL version: OpenSSL 1.0.2o  27 Mar 2018
cURL version: 7.65.1 libz 1.2.8 ssl OpenSSL/1.0.2o
zip extension: OK

When I run this command:

composer require -W drupal/core-dev ^9.2

I get the following output:

Running composer update drupal/core-dev --with-all-dependencies
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Root composer.json requires drupal/drupal *, found drupal/drupal[9.2.x-dev] but it conflicts with another require.

And I expected this to happen:

I need to be told what the 'another require' actually is, otherwise I can't understand or fix the problem.

@Seldaek
Copy link
Member

Seldaek commented Nov 2, 2020

If you can, please check with #9360 applied to see if it fixes it. It hopefully does.

@Seldaek Seldaek added this to the 2.0 milestone Nov 2, 2020
@Seldaek Seldaek added the Bug label Nov 2, 2020
@joachim-n
Copy link
Author

I cloned composer from source, added https://github.com/naderman/composer.git as an origin and checked out the branch for #9360, pool-builder-unlock-consider-all-constraints.

Using the bin/composer from the git clone, I tried the command in my project again.

I still get:

Running composer update drupal/core-dev --with-all-dependencies
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Root composer.json requires drupal/drupal *, found drupal/drupal[9.2.x-dev] but these were not loaded, likely because it conflicts with another require.

@Seldaek
Copy link
Member

Seldaek commented Nov 2, 2020

Ok, might be another bug of the same kind then /cc @naderman

@joachim-n just to get you out of your pickle here, have you tried a full composer update? Does that resolve?

@naderman
Copy link
Member

naderman commented Nov 2, 2020

The simple workaround here should be to run the update command with drupal/drupal added as well, which means running require without an update first and then manually updating:

composer require --no-update drupal/core-dev ^9.2
composer update -W drupal/core-dev drupal/drupal

@joachim-n
Copy link
Author

I did:

composer update
composer require -W drupal/core-dev ^9.2

same result as before.

I then tried:

composer require --no-update drupal/core-dev ^9.2
composer update -W drupal/core-dev drupal/drupal

and that worked!!!

Thanks!

@joachim-n
Copy link
Author

It also works from scratch if I add drupal/core-dev to the composer.json (with version *) and then do composer update.

@Seldaek
Copy link
Member

Seldaek commented Nov 24, 2021

I can't repro this anymore with the latest snapshot, unsure if something changed in the package landscape or if we did actually fix the issue, but anyway closing.

@Seldaek Seldaek closed this as completed Nov 24, 2021
@jrobens
Copy link

jrobens commented Dec 16, 2023

Problem 1
- drupal/core-recommended 10.0.0 requires drupal/core 10.0.0 -> found drupal/core[10.0.0] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.1 requires drupal/core 10.0.1 -> found drupal/core[10.0.1] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.2 requires drupal/core 10.0.2 -> found drupal/core[10.0.2] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.3 requires drupal/core 10.0.3 -> found drupal/core[10.0.3] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.4 requires drupal/core 10.0.4 -> found drupal/core[10.0.4] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.5 requires drupal/core 10.0.5 -> found drupal/core[10.0.5] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.6 requires drupal/core 10.0.6 -> found drupal/core[10.0.6] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.7 requires drupal/core 10.0.7 -> found drupal/core[10.0.7] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.8 requires drupal/core 10.0.8 -> found drupal/core[10.0.8] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.9 requires drupal/core 10.0.9 -> found drupal/core[10.0.9] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.10 requires drupal/core 10.0.10 -> found drupal/core[10.0.10] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.0.11 requires drupal/core 10.0.11 -> found drupal/core[10.0.11] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.0 requires drupal/core 10.1.0 -> found drupal/core[10.1.0] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.1 requires drupal/core 10.1.1 -> found drupal/core[10.1.1] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.2 requires drupal/core 10.1.2 -> found drupal/core[10.1.2] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.3 requires drupal/core 10.1.3 -> found drupal/core[10.1.3] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.4 requires drupal/core 10.1.4 -> found drupal/core[10.1.4] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.5 requires drupal/core 10.1.5 -> found drupal/core[10.1.5] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.6 requires drupal/core 10.1.6 -> found drupal/core[10.1.6] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.1.7 requires drupal/core 10.1.7 -> found drupal/core[10.1.7] but these were not loaded, likely because it conflicts with another require.
- drupal/core-recommended 10.2.0 requires drupal/core 10.2.0 -> found drupal/core[10.2.0] but these were not loaded, likely because it conflicts with another require.
- Root composer.json requires drupal/core-recommended ^10 -> satisfiable by drupal/core-recommended[10.0.0, ..., 10.2.0].

@jrobens
Copy link

jrobens commented Dec 16, 2023

composer.json
Local + composer file

@jrobens
Copy link

jrobens commented Dec 16, 2023

composer_lock.txt

@jrobens
Copy link

jrobens commented Dec 16, 2023

Remove the lock file to get more sense
Problem 1
- zendframework/zend-feed[2.11.0, ..., 2.12.0] require php ^5.6 || ^7.0 -> your php version (8.3.0) does not satisfy that requirement.
- laminas/laminas-feed[2.12.0, ..., 2.12.3] require php ^5.6 || ^7.0 -> your php version (8.3.0) does not satisfy that requirement.
- symfony-cmf/routing[1.4.0, ..., 1.4.1] require php ^5.3.9|^7.0 -> your php version (8.3.0) does not satisfy that requirement.
- laminas/laminas-feed[2.13.0, ..., 2.14.1] require php ^7.3 || ~8.0.0 -> your php version (8.3.0) does not satisfy that requirement.
- longwave/laminas-diactoros 2.14.1 requires php ^7.3 || ~8.0.0 || ~8.1.0 -> your php version (8.3.0) does not satisfy that requirement.
- longwave/laminas-diactoros 2.14.2 requires php ^7.3 || ~8.0.0 || ~8.1.0 || ~8.2.0 -> your php version (8.3.0) does not satisfy that requirement.
- drupal/core[8.9.11, ..., 8.9.20] require php ^7.0.8 -> your php version (8.3.0) does not satisfy that requirement.
- drupal/core[9.0.10, ..., 9.0.14] require php ^7.3 -> your php version (8.3.0) does not satisfy that requirement.
- Root composer.json requires drupal/coffee_mediteran ^2.0 -> satisfiable by drupal/coffee_mediteran[2.0.0].
- drupal/core-recommended 10.0.2 requires drupal/core 10.0.2 -> satisfiable by drupal/core[10.0.2].
- Conclusion: don't install drupal/core 10.0.2 (conflict analysis result)
- drupal/core-recommended 10.0.3 requires drupal/core 10.0.3 -> satisfiable by drupal/core[10.0.3].
- Conclusion: don't install drupal/core 10.0.3 (conflict analysis result)
- drupal/core-recommended 10.0.4 requires drupal/core 10.0.4 -> satisfiable by drupal/core[10.0.4].
- Conclusion: don't install drupal/core 10.0.4 (conflict analysis result)
- drupal/core-recommended 10.0.5 requires drupal/core 10.0.5 -> satisfiable by drupal/core[10.0.5].
- Conclusion: don't install drupal/core 10.0.5 (conflict analysis result)
- drupal/core-recommended 10.0.6 requires drupal/core 10.0.6 -> satisfiable by drupal/core[10.0.6].
- Conclusion: don't install drupal/core 10.0.6 (conflict analysis result)
- drupal/core-recommended 10.0.7 requires drupal/core 10.0.7 -> satisfiable by drupal/core[10.0.7].
- Conclusion: don't install drupal/core 10.0.7 (conflict analysis result)
- drupal/core-recommended 10.0.8 requires drupal/core 10.0.8 -> satisfiable by drupal/core[10.0.8].
- Conclusion: don't install drupal/core 10.0.8 (conflict analysis result)
- drupal/core-recommended 10.0.9 requires drupal/core 10.0.9 -> satisfiable by drupal/core[10.0.9].
- Conclusion: don't install drupal/core 10.0.9 (conflict analysis result)
- drupal/core-recommended 10.0.10 requires drupal/core 10.0.10 -> satisfiable by drupal/core[10.0.10].
- Conclusion: don't install drupal/core 10.0.10 (conflict analysis result)
- drupal/core-recommended 10.0.11 requires drupal/core 10.0.11 -> satisfiable by drupal/core[10.0.11].
- Conclusion: don't install drupal/core 10.0.11 (conflict analysis result)
- drupal/core-recommended 10.1.0 requires drupal/core 10.1.0 -> satisfiable by drupal/core[10.1.0].
- Conclusion: don't install drupal/core 10.1.0 (conflict analysis result)
- drupal/core-recommended 10.1.1 requires drupal/core 10.1.1 -> satisfiable by drupal/core[10.1.1].
- Conclusion: don't install drupal/core 10.1.1 (conflict analysis result)
- drupal/core-recommended 10.1.2 requires drupal/core 10.1.2 -> satisfiable by drupal/core[10.1.2].
- Conclusion: don't install drupal/core 10.1.2 (conflict analysis result)
- drupal/core-recommended 10.1.3 requires drupal/core 10.1.3 -> satisfiable by drupal/core[10.1.3].
- Conclusion: don't install drupal/core 10.1.3 (conflict analysis result)
- drupal/core-recommended 10.1.4 requires drupal/core 10.1.4 -> satisfiable by drupal/core[10.1.4].
- Conclusion: don't install drupal/core 10.1.4 (conflict analysis result)
- drupal/core-recommended 10.1.5 requires drupal/core 10.1.5 -> satisfiable by drupal/core[10.1.5].
- Conclusion: don't install drupal/core 10.1.5 (conflict analysis result)
- drupal/core-recommended 10.1.6 requires drupal/core 10.1.6 -> satisfiable by drupal/core[10.1.6].
- Conclusion: don't install drupal/core 10.1.6 (conflict analysis result)
- drupal/core-recommended 10.1.7 requires drupal/core 10.1.7 -> satisfiable by drupal/core[10.1.7].
- Conclusion: don't install drupal/core 10.1.7 (conflict analysis result)
- drupal/core-recommended 10.2.0 requires drupal/core 10.2.0 -> satisfiable by drupal/core[10.2.0].
- Conclusion: don't install drupal/core 10.2.0 (conflict analysis result)
- drupal/core[8.9.0, ..., 8.9.10] require symfony-cmf/routing ^1.4 -> satisfiable by symfony-cmf/routing[1.4.0, 1.4.1].
- drupal/core[8.8.0, ..., 8.8.12] require zendframework/zend-feed ^2.12 -> satisfiable by laminas/laminas-feed[2.12.0, ..., 2.19.0], zendframework/zend-feed[2.12.0].
- drupal/core[9.4.15, ..., 9.5.11] require longwave/laminas-diactoros ^2.14 -> satisfiable by longwave/laminas-diactoros[2.14.1, 2.14.2].
- drupal/core-recommended 10.0.1 requires drupal/core 10.0.1 -> satisfiable by drupal/core[10.0.1].
- Conclusion: don't install drupal/core 10.0.1 (conflict analysis result)
- drupal/coffee_mediteran 2.0.0 requires drupal/core ^8.8 || ^9 -> satisfiable by drupal/core[8.8.0, ..., 8.9.20, 9.0.0, ..., 9.5.11].
- You can only install one version of a package, so only one of these can be installed: drupal/core[8.0.0, ..., 8.9.20, 9.0.0, ..., 9.5.11, 10.0.0, ..., 10.2.0].
- You can only install one version of a package, so only one of these can be installed: drupal/core[8.7.0, ..., 8.9.20, 9.0.0, ..., 9.5.11, 10.0.0, ..., 10.2.0].
- drupal/core-recommended 10.0.0 requires drupal/core 10.0.0 -> satisfiable by drupal/core[10.0.0].
- Root composer.json requires drupal/core-recommended ^10 -> satisfiable by drupal/core-recommended[10.0.0, ..., 10.2.0].

@Seldaek
Copy link
Member

Seldaek commented Dec 18, 2023

@jrobens I am not sure what command you are running, when I run an install I get this which clearly explains the lock file is outdated and incompatible with the composer.json requirements:

Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. It is recommended that you run `composer update` or `composer update <package name>`.
- Required package "drupal/context" is in the lock file as "4.1.0" but that does not satisfy your constraint "^5.0@RC".
- Required package "drupal/core-composer-scaffold" is in the lock file as "9.5.11" but that does not satisfy your constraint "^10".
- Required package "drupal/core-project-message" is in the lock file as "9.5.11" but that does not satisfy your constraint "^10".
- Required package "drupal/core-recommended" is in the lock file as "9.5.11" but that does not satisfy your constraint "^10".
- Required package "drupal/focal_point" is in the lock file as "1.5.0" but that does not satisfy your constraint "^2.0".
- Required package "drupal/viewsreference" is in the lock file as "1.8.0" but that does not satisfy your constraint "^2.0@beta".
- Required package "drush/drush" is in the lock file as "11.6.0" but that does not satisfy your constraint "^12".
- Required (in require-dev) package "drupal/core-dev" is not present in the lock file.
This usually happens when composer files are incorrectly merged or the composer.json file is manually edited.
Read more about correctly resolving merge conflicts https://getcomposer.org/doc/articles/resolving-merge-conflicts.md
and prefer using the "require" command over editing the composer.json file directly https://getcomposer.org/doc/03-cli.md#require-r

@jrobens
Copy link

jrobens commented Dec 18, 2023

Sorry, should have put the commands in. The process is building up the composer file with --no-update and then trying to run.

composer remove drupal/console
composer require 'drupal/core-dev:^10' --dev --no-update

composer require 'drupal/core-recommended:^10' 'drupal/core-composer-scaffold:^10' 'drupal/core-project-message:^10' --no-update

composer require 'drush/drush:^12' --no-update

Then

composer update 'drupal/core*' drupal/context drupal/focal_point drupal/animate_on_scroll drupal/viewsreference drush/drush -W

@Seldaek
Copy link
Member

Seldaek commented Dec 19, 2023

Yeah sorry I have no clue what's going on there. It's quite a pain as I cannot easily navigate the drupal packages' dependencies as with those that are on packagist.org.. But from what I gather the issue is that drupal/core_event_dispatcher requires drupal/core ^9.3, and drupal/core-recommended requires drupal/core ^10, so that's the conflict.

I don't know why it does not show this in the partial update though, that is very odd, but at least doing a full update shows you the problem. This happens somehow with complex setups like core-recommended which add indirection. If you change the require to drupal/core ^10 instead of core-recommended then it works. So that might allow us to build a smaller repro case to try and understand this..

@Seldaek Seldaek reopened this Dec 19, 2023
@Seldaek
Copy link
Member

Seldaek commented Dec 20, 2023

Ok I got this far:

# works fine without optimizer
COMPOSER_POOL_OPTIMIZER=0 composer update 'drupal/core*' drupal/viewsreference -W --dry-run
# fails with optimizer, somehow drupal/core v10 gets removed
composer update 'drupal/core*' drupal/viewsreference -W --dry-run

Using these files (rename .lock.json to .lock)

composer.json
composer.lock.json

So it seems the pool optimizer is causing this issue somehow (cc @Toflar) but I haven't gotten to the bottom of it.

@Toflar
Copy link
Contributor

Toflar commented Dec 20, 2023

Bah, debugging this is super time-consuming. But here's what I've found out so far (gotta stop for today):

So seems to be related to the locked packages logic.

@Seldaek
Copy link
Member

Seldaek commented Dec 20, 2023

cc @driskell as it's a drupal issue maybe you'll have an easier time reasoning about this.. I am so unfamiliar with the dependencies that it does not help :)

@driskell
Copy link
Contributor

driskell commented Dec 20, 2023

Usually when I see this I run composer why xxxxx --locked on the thing that got removed. Probably worth looking at way to add it to the message.

When I use the files from #9393 (comment) along with composer update 'drupal/core*' drupal/viewsreference -W --dry-run I get the following:

Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - drupal/core-recommended 10.2.0 requires drupal/core 10.2.0 -> found drupal/core[10.2.0] but these were not loaded, likely because it conflicts with another require.
    - drupal/core-recommended 10.1.7 requires drupal/core 10.1.7 -> found drupal/core[10.1.7] but these were not loaded, likely because it conflicts with another require.
    - Root composer.json requires drupal/core-recommended ^10.1.7 -> satisfiable by drupal/core-recommended[10.1.7, 10.2.0].

So I then run composer why drupal/core --locked to find out what is requiring it and the constraints:

$ composer why drupal/core --locked
drupal/address                       1.12.0     requires  drupal/core (^9.2 || ^10)
drupal/admin_toolbar                 3.4.2      requires  drupal/core (^9.2 || ^10)
drupal/animate_on_scroll             2.0.0      requires  drupal/core (^8 || ^9 || ^10)
drupal/autoservices                  1.0.1      requires  drupal/core (^9 || ^10)
drupal/block_class                   2.0.11     requires  drupal/core (^8 || ^9 || ^10)
drupal/block_title_link              1.1.5      requires  drupal/core (^8 || ^9 || ^10)
drupal/coffee                        1.3.0      requires  drupal/core (^9.2 || ^10)
drupal/coffee_mediteran              2.0.0      requires  drupal/core (^8.8 || ^9)
drupal/commerce                      2.33.0     requires  drupal/core (^9.3 || ^10)
drupal/context                       4.1.0      requires  drupal/core (^8.8 || ^9)
drupal/copyright_footer              3.0.1      requires  drupal/core (^9 || ^10)
drupal/core                          9.5.11     replaces  drupal/core-annotation (self.version)
drupal/core                          9.5.11     replaces  drupal/core-assertion (self.version)
drupal/core                          9.5.11     replaces  drupal/core-bridge (self.version)
drupal/core                          9.5.11     replaces  drupal/core-class-finder (self.version)
drupal/core                          9.5.11     replaces  drupal/core-datetime (self.version)
drupal/core                          9.5.11     replaces  drupal/core-dependency-injection (self.version)
drupal/core                          9.5.11     replaces  drupal/core-diff (self.version)
drupal/core                          9.5.11     replaces  drupal/core-discovery (self.version)
drupal/core                          9.5.11     replaces  drupal/core-event-dispatcher (self.version)
drupal/core                          9.5.11     replaces  drupal/core-file-cache (self.version)
drupal/core                          9.5.11     replaces  drupal/core-file-security (self.version)
drupal/core                          9.5.11     replaces  drupal/core-filesystem (self.version)
drupal/core                          9.5.11     replaces  drupal/core-front-matter (self.version)
drupal/core                          9.5.11     replaces  drupal/core-gettext (self.version)
drupal/core                          9.5.11     replaces  drupal/core-graph (self.version)
drupal/core                          9.5.11     replaces  drupal/core-http-foundation (self.version)
drupal/core                          9.5.11     replaces  drupal/core-php-storage (self.version)
drupal/core                          9.5.11     replaces  drupal/core-plugin (self.version)
drupal/core                          9.5.11     replaces  drupal/core-proxy-builder (self.version)
drupal/core                          9.5.11     replaces  drupal/core-render (self.version)
drupal/core                          9.5.11     replaces  drupal/core-serialization (self.version)
drupal/core                          9.5.11     replaces  drupal/core-transliteration (self.version)
drupal/core                          9.5.11     replaces  drupal/core-utility (self.version)
drupal/core                          9.5.11     replaces  drupal/core-uuid (self.version)
drupal/core                          9.5.11     replaces  drupal/core-version (self.version)
drupal/core-recommended              9.5.11     requires  drupal/core (9.5.11)
drupal/core_event_dispatcher         3.3.4      requires  drupal/core (^9.3)
drupal/crop                          2.3.0      requires  drupal/core (^9.3 || ^10)
drupal/ctools                        4.0.4      requires  drupal/core (^9.3 || ^10)
drupal/drulma                        2.0.0      requires  drupal/core (^8.8 || ^9 || ^10)
drupal/drulma_companion              2.1.6      requires  drupal/core (^9.3 | ^10)
drupal/encrypt                       3.1.0      requires  drupal/core (^8 || ^9 || ^10)
drupal/entity                        1.4.0      requires  drupal/core (^9.2|^10.0)
drupal/entity_class_formatter        2.0.0      requires  drupal/core (^9 || ^10)
drupal/entity_print                  2.13.0     requires  drupal/core (^9.4 || ^10.0)
drupal/entity_reference_revisions    1.10.0     requires  drupal/core (^9 || ^10)
drupal/entity_word                   3.1.1      requires  drupal/core (^8.8 || ^9 || ^10)
drupal/file_mdm                      2.6.0      requires  drupal/core (^9.3 | ^10)
drupal/file_mdm_exif                 2.6.0      requires  drupal/core (^9.3 | ^10)
drupal/file_mdm_font                 2.6.0      requires  drupal/core (^9.3 | ^10)
drupal/focal_point                   1.5.0      requires  drupal/core (^8.8 || ^9)
drupal/fontawesome                   2.26.0     requires  drupal/core (^9.4 || ^10)
drupal/fontawesome_iconpicker_widget 2.26.0     requires  drupal/core (^9.4 || ^10)
drupal/fontawesome_menu_icons        2.0.0      requires  drupal/core (^9.2 || ^10)
drupal/form_options_attributes       2.0.2      requires  drupal/core (^9 || ^10)
drupal/geofield                      1.57.0     requires  drupal/core (^8.8 || ^9 || ^10)
drupal/google_analytics              4.0.2      requires  drupal/core (^9.3 || ^10)
drupal/honeypot                      2.1.3      requires  drupal/core (^9.2 || ^10)
drupal/hook_event_dispatcher         3.3.4      requires  drupal/core (^9.3)
drupal/image_effects                 3.4.0      requires  drupal/core (^9.3 || ^10)
drupal/inline_entity_form            1.0.0-rc15 requires  drupal/core (^8.8 || ^9 || ^10)
drupal/jquery_ui                     1.6.0      requires  drupal/core (^9.2 || ^10)
drupal/jquery_ui_autocomplete        2.0.0      requires  drupal/core (^9.2 || ^10)
drupal/jquery_ui_menu                2.0.0      requires  drupal/core (^9.2 || ^10)
drupal/key                           1.17.0     requires  drupal/core (>=8.9 <11)
drupal/layout_paragraphs             2.0.4      requires  drupal/core (^9.2 || ^10)
drupal/leaflet                       10.2.4     requires  drupal/core (^9 || ^10)
drupal/leaflet_more_maps             2.1.3      requires  drupal/core (^8.8 || ^9 || ^10)
drupal/libraries_provider            2.0.1      requires  drupal/core (^9 || ^10)
drupal/linked_field                  1.5.0      requires  drupal/core (^8.8 || ^9 || ^10)
drupal/lp_fontawesome                5.15004.0  requires  drupal/core (^9 || ^10)
drupal/mail_login                    2.9.0      requires  drupal/core (^8 || ^9 || ^10)
drupal/mailsystem                    4.4.0      requires  drupal/core (^9 || ^10)
drupal/markdown                      3.0.0      requires  drupal/core (^9.1 || ^10)
drupal/mediteran                     1.32.0     requires  drupal/core (^9.2 || ^10)
drupal/menu_item_extras              3.0.2      requires  drupal/core (^9.4 || ^10)
drupal/menu_item_fields              1.10.0     requires  drupal/core (^9 || ^10)
drupal/menu_link_attributes          1.3.0      requires  drupal/core (^8 || ^9 || ^10)
drupal/metatag                       1.26.0     requires  drupal/core (^9.3 || ^10)
drupal/module_filter                 4.0.1      requires  drupal/core (^9 || ^10)
drupal/nagios                        1.19.0     requires  drupal/core (^8 || ^9 || ^10)
drupal/paragraph_blocks              3.1.8      requires  drupal/core (^8 || ^9 || ^10)
drupal/paragraphs                    1.16.0     requires  drupal/core (^9.3 || ^10)
drupal/pathauto                      1.12.0     requires  drupal/core (^9.3 || ^10)
drupal/profile                       1.7.0      requires  drupal/core (^9 || ^10)
drupal/role_paywall                  2.1.10     requires  drupal/core (^9 || ^10)
drupal/shield                        1.7.0      requires  drupal/core (^9.2 || ^10)
drupal/spamaway                      2.0.0      requires  drupal/core (^8 || ^9 || ^10)
drupal/state_machine                 1.10.0     requires  drupal/core (^9.2 || ^10)
drupal/swiftmailer                   2.4.0      requires  drupal/core (^8.9.11 || ^9.0.10 || ^10)
drupal/term_condition                2.0.3      requires  drupal/core (^9 || ^10)
drupal/tfa                           1.4.0      requires  drupal/core (^8 || ^9 || ^10)
drupal/token                         1.13.0     requires  drupal/core (^9.2 || ^10)
drupal/upgrade_status                4.0.0      requires  drupal/core (^9 || ^10)
drupal/views_add_button              2.0.8      requires  drupal/core (^8 || ^9 || ^10)
drupal/viewsreference                1.8.0      requires  drupal/core (^8.6|^9.0)
drupal/webform                       6.1.8      requires  drupal/core (^9.3)
drupal/xmlsitemap                    1.5.0      requires  drupal/core (^9.3 || ^10)
drush/drush                          11.6.0     conflicts drupal/core (< 9.2)

So here we can see that the following Drupal modules in the lock file are not marked compatible with Drupal 10 so it won't be allowed to be installed:

drupal/coffee_mediteran              2.0.0      requires  drupal/core (^8.8 || ^9)
drupal/context                       4.1.0      requires  drupal/core (^8.8 || ^9)
drupal/focal_point                   1.5.0      requires  drupal/core (^8.8 || ^9)
drupal/hook_event_dispatcher         3.3.4      requires  drupal/core (^9.3)
drupal/viewsreference                1.8.0      requires  drupal/core (^8.6|^9.0)
drupal/webform                       6.1.8      requires  drupal/core (^9.3)

Checking those, drupal/coffee_mediteran has no v10 available: https://www.drupal.org/project/coffee_mediteran
Interestingly, this is in the lock file but not in the composer.json, so maybe there's something missing in composer to cope with this? It's possible the locked contents are being used to filter requirements, even though they are candidate for removal?

I manually removed from lock file to get past but maybe I could've just added to the update command. Adding the other packages that are locked to incompatible versions, I then find below:

$ composer update 'drupal/core*' drupal/viewsreference drupal/coffee_mediteran drupal/context drupal/focal_point drupal/hook_event_dispatcher drupal/webform -W --dry-run
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - drupal/core-recommended 10.2.0 requires drupal/core 10.2.0 -> satisfiable by drupal/core[10.2.0].
    - drupal/core-recommended 10.1.7 requires drupal/core 10.1.7 -> satisfiable by drupal/core[10.1.7].
    - drupal/core 10.2.0 requires symfony/console ^6.4 -> found symfony/console[v6.4.0, v6.4.1] but these were not loaded, likely because it conflicts with another require.
    - drupal/core 10.1.7 requires symfony/console ^6.3 -> found symfony/console[v6.3.0, ..., v6.4.1] but these were not loaded, likely because it conflicts with another require.
    - Root composer.json requires drupal/core-recommended ^10.1.7 -> satisfiable by drupal/core-recommended[10.1.7, 10.2.0].

So we check that one:

$ composer why consolidation/site-process --locked
drush/drush 11.6.0 requires consolidation/site-process (^4.1.3 || ^5)

So yeh repeating a few times we find a few more things in lock that are not compatible, not in the composer.json but in the lock file:

$ composer update 'drupal/core*' drupal/viewsreference drupal/context drupal/focal_point drupal/hook_event_dispatcher drupal/webform drush/drush drupal/libraries_provider consolidation/log -W --dry-run

This then works.

So it's possible it's related to when filtering the pool by the locked but not-being-updated packages, it's not accounting for something that is potentially going to be removed and therefore allow other packages to install. 😬

Separate note, the drush/drush originally included consolidation/log but then removed it in later version - adding drush/drush to the update wasn't enough - clearly it didn't add it to the pool but it being locked meant it restricted the pool and broke it - and I had to add consolidation/log to the update - this then unlocks it, stops it filtering the pool, and then it subsequently removes it after drush/drush update is chosen

Toflar added a commit to Toflar/composer that referenced this issue Dec 20, 2023
@Toflar
Copy link
Contributor

Toflar commented Dec 20, 2023

#11766 here are my thoughts. It should fix the problem described by @jrobens but it will revert the logic introduced by @driskell 🙈 I guess it's better to forgo an optimization in favor of a bugfix but I understand that the additional optimization step was quite important to the Drupal community as well - so let's see what we can do together 😊

@Toflar
Copy link
Contributor

Toflar commented Dec 21, 2023

Okay, managed to debug exactly on what's happening after way too many hours 😅 Here's the exact reason on what's happening in the case described by @Seldaek in #9393 (comment):

  • drupal/webform is locked at 6.1.8.0 but allowed to be updated/removed because of -W
  • PoolOptimizer::optimizeImpossiblePackagesAway() removes drupal/core 10.2.0.0 because drupal/webform is locked at 6.1.8.0 and requires drupal/core in ^9.3.

The problem is that drupal/webform itself is going to be removed during the update process so actually, its requirements should not be considered and thus the drupal/core 10.2.0.0 should not get removed.

This exact case was actually sort of considered in the $isUnusedPackage here:

if (isset($this->requireConstraintsPerPackage[$packageName])) {

However, this doesn't work properly because $this->requireConstraintsPerPackage[$packageName]) actually does contain drupal/webform because this array is built before any optimizations are even applied. It is required by the root package drupal/spamaway 2.0.0.0 which would also get removed.

So the bug happens because the $this->requireConstraintsPerPackage (and probably other things) are used as if this array were to contain a definitive list of what is going to be required. Something that we never know at the stage of the PoolOptimizer as this is what the Solver will do - it's a typical problem of recursive optimizations.

So long story short: The current optimization step for locked packages is flawed because it is built on top of false assumptions.

I tried building a test case for that but I failed. My brain capacity is not sufficient to conjure some up some test case representing the situation that I described here but I hope it helps somebody else to create such a test file 😅

@Toflar
Copy link
Contributor

Toflar commented Dec 21, 2023

@naderman we might need your brain for the test 😉

@Toflar
Copy link
Contributor

Toflar commented Dec 21, 2023

Or well, actually I don't think we need a test, really. I think we have to remove the optimization step.

@naderman
Copy link
Member

@Toflar Definitely think we need a test case for this to avoid making this sort of mistake again in the future. But I don't think I'll have time for that before mid-January.

@Toflar
Copy link
Contributor

Toflar commented Dec 22, 2023

Okay, I've managed to extract the exact case we're encountering into a test 🥳 See #11772 - I've kept the exact package names and versions for easier reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants