Major update v5.0 #25

ffraenz · 2020-06-12T23:17:27Z

Checklist:

Submit pull requests to Composer to maintain plugin compatibility
Add support for composer/composer ^2.0 while maintaining support for composer/composer ^1.0 (Support for Composer 2 #23)
Add support for vlucas/phpdotenv ^5.0 while maintaining support for older versions (Update dependency vlucas/phpdotenv to v5 #29)
Move CI to GitHub actions
Refactor code to increase extensibility
Test the Composer plugin in the wild
Update README

.travis.yml

composer.json

References: - ffraenz/private-composer-installer#25 Resolves junaidbhura#19

mcaskill · 2020-09-07T04:51:06Z

I was working on upgrading junaidbhura/composer-wp-pro-plugins to support Composer v2 and this pull request was quite helpful to understand the internal changes made to Composer.

I believe I've reached a similar roadblock to you, based on tests of both plugins: in Composer v2, all packages get downloaded first then prepared/installed/updated/uninstalled/cleaned-up. This means changing the package dist URL in PRE_PACKAGE_* and PRE_FILE_DOWNLOAD events is no longer possible.

I bring this up because I'm curious to see what other people have been planning to do about this predicament. I'm under the impression that any Composer plugin that dynamically alters a package's dist URL (and by extension the cache key) will need a custom InstallerInterface or DownloaderInterface to override Composer's; I have not had the chance to test PRE_POOL_CREATE.

I'm available to help out with this plugin.

Thanks,

/cc @junaidbhura

ffraenz · 2020-09-07T07:19:44Z

Hi @mcaskill, indeed I hit a roadblock when I found out that the remote filesystem could no longer be exchanged in Composer 2 to replace the dist URL. I wrote to one of the Composer maintainers and worked out a set of changes to make an interface available to us. You can now simply call setProcessedUrl on PreFileDownloadEvent to alter the dist URL before downloading a package. Please have a look at the pull request composer/composer#8975.

I still need to incorporate the changes into this repository to finalize v5. I'm always open to suggestions and/or contributions to this plugin, so feel free to join if you're up to it!

mcaskill · 2020-09-07T15:51:27Z

You can now simply call setProcessedUrl on PreFileDownloadEvent to alter the dist URL before downloading a package.

I had not seen the that pull request at the time but I have implemented setProcessedUrl() like you did. The downloading of the ZIP file is, thanks to you, resolved and functional on my end.

The roadblock I'm at is the cache key in FileDownloader::download(). It is hashed before PRE_FILE_DOWNLOAD and PRE_PACKAGE_*, the URI's placeholders on the dist URL are replaced too late.

References: - ffraenz/private-composer-installer#25 Resolves junaidbhura#19

ffraenz · 2020-09-10T09:54:08Z

As far as I know the caching strategy does not change much moving forward to Composer 2. In this project I make sure a version number is always included in the dist URL (and in composer.lock) to bust caching when the version changes. Re-downloading a package due to changed env vars is a use case I do not plan to support as it kinda works against the package manager core concept. As Composer 2 is not released yet, there is still a chance to integrate a hook, if you think you need one.

References: - ffraenz/private-composer-installer#25 Resolves junaidbhura#19

src/PrivateComposerInstaller/Plugin.php

test/PrivateComposerInstaller/PluginTest.php

test/PrivateComposerInstaller/RemoteFilesystemTest.php

mcaskill · 2020-09-12T20:58:10Z

Hi @ffraenz,

I'm sorry, I have been poorly explaining the issue I'm encountering with both your package and the one I'm working on.

As far as I know the caching strategy does not change much moving forward to Composer 2. […]

Yes, the caching strategy hasn't changed. What has changed is the processing strategy.

Your unit tests are only testing what your plugin can do with a package.
They are not testing what Composer is doing with your plugin.

Example:

1. Package

ACME_KEY=foobar

{
    "type": "package",
    "package": {
        "name": "acme/anvil",
        "version": "1.0.0",
        "dist": {
            "type": "zip",
            "url": "https://acme.com/packages/anvil?key={%ACME_KEY}&version={%VERSION}"
        },
        "require": {
            "ffraenz/private-composer-installer": "^5.0"
        }
    }
}

In Composer 1.0:

Composer resolves dependencies, iterates packages while dispatching PRE_PACKAGE_* before PRE_FILE_DOWNLOAD, then finally writes to the lockfile.

During the PRE_PACKAGE_* events, you inject the package's version into the URL. This filtered dist URL is used in the lockfile and serves as the cache key of FileDownloader and the processed URL of PreFileDownloadEvent.

During the PRE_FILE_DOWNLOAD event, you replace any other placeholders in the URL. This filtered processed URL is the target download.

2. Breakdown

Dist URL: https://acme.com/packages/anvil?key={%ACME_KEY}&version={%VERSION}

Resolves packages

Dispatches PRE_PACKAGE_*

Lockfile URL: https://acme.com/packages/anvil?key={%ACME_KEY}&version=1.0.0

Cache Key: c49a8ac6c2b38148b07bded53fe49f1781fec37d

Dispatches PRE_FILE_DOWNLOAD

Download URL: https://acme.com/packages/anvil?key=foobar&version=1.0.0

Cache Path: acme/anvil/c49a8ac6c2b38148b07bded53fe49f1781fec37d.zip

Locks composer.lock

In Composer 2.0:

Composer resolves dependencies, writes to the lockfile, downloads the packages while dispatching PRE_FILE_DOWNLOAD, then iterates packages dispatching PRE_PACKAGE_*.

As a result, the package version is missing from the lockfile, the cache key, and the processed URL.

3. Breakdown

Dist URL: https://acme.com/packages/anvil?key={%ACME_KEY}&version={%VERSION}

Resolves packages

Lockfile URL: https://acme.com/packages/anvil?key={%ACME_KEY}&version={%VERSION}

Locks composer.lock

Cache Key: ee70db77098cc143c9d23d7a1858573f1b03b960

Dispatches PRE_FILE_DOWNLOAD

Download URL: https://acme.com/packages/anvil?key=foobar&version={%VERSION}

Cache Path: acme/anvil/ee70db77098cc143c9d23d7a1858573f1b03b960.zip

Dispatches PRE_PACKAGE_*

As of d91f7fa, your plugin throws MissingEnvException at step 7 because it can't find VERSION among the environment variables.

Even if we ensure the version is injected at this step…

4. Snippet

public function handlePreFileDownload(PreFileDownloadEvent $event): void
{
    $processedUrl = $event->getProcessedUrl();

    if ($event->getType() === 'package') {
        $package = $event->getContext();
        if ($package instanceof PackageInterface) {
            $filteredUrl = $this->filterDistUrl($processedUrl, $package);
            if ($filteredUrl !== $processedUrl) {
                $processedUrl = $filteredUrl;
                $package->setDistUrl($filteredUrl);
            }
        }
    }

    $filteredProcessedUrl = $this->filterProcessedUrl($processedUrl);

…the lockfile URL and cache key are already set.

At this point, on my end, Composer fails to unzip the archive and I end up with an empty folder at vendor/acme/anvil.

This is the roadblock I'm at.

I think symfony/flex might have some practical ideas on how to move forward.

test/PrivateComposerInstaller/RemoteFilesystemTest.php

…the plugin API Co-authored-by: Chauncey McAskill <chauncey@mcaskill.ca> Co-authored-by: Viktor Szépe <viktor@szepe.net>

Co-authored-by: Chauncey McAskill <chauncey@mcaskill.ca>

ffraenz · 2020-09-14T11:32:49Z

Hi @mcaskill, I completely missed the change in the Composer 2 workflow that renders the current strategy completely useless. Thank you for taking the time to break it down in a very detailed way. I guess I need to go back to the drawing board to see what we can do about it.

mcaskill · 2020-09-14T19:49:27Z

This recent issue is tangentially related with our situation: composer/composer#9209

Seldaek · 2020-09-15T07:59:00Z

@mcaskill please report an issue on composer about the cache key issue, that definitely seems like something we'd want to fix if possible.

szepeviktor · 2020-10-26T19:29:18Z

@ffraenz coverallsapp/github-action does not support PHP 😿

szepeviktor · 2020-10-26T19:30:10Z

Recently this PR got a green way php-coveralls/php-coveralls#296

alexkerber · 2020-10-26T19:38:01Z

Any updates with this? Pretty much stuck with
The "ffraenz/private-composer-installer" plugin was skipped because it requires a Plugin API version ("^1.0.0") that does not match your Composer installation ("2.0.0"). You may need to run composer update with the "--no- plugins" option. Installing dependencies from lock file (including require- dev) Verifying lock file contents can be installed on current platform. Your lock file does not contain a compatible set of packages. Please run composer update.

szepeviktor · 2020-10-26T19:43:31Z

name: "Merge intermediate results and finalize Coveralls report"

@ffraenz Actually you can use that GitHub action in the final step :)

ffraenz · 2020-10-26T19:45:43Z

@alexkerber A release supporting Composer 2 will be available shortly.

alexkerber · 2020-10-26T19:46:53Z

@alexkerber A release supporting Composer 2 will be available shortly.

We are stuck with several projects because of this issue :)

szepeviktor · 2020-10-26T21:32:53Z

Unit tests somehow did not reach this file.

mcaskill · 2020-10-26T23:56:42Z

MikeiLL · 2021-01-22T19:50:38Z

Any updates with this? Pretty much stuck with
The "ffraenz/private-composer-installer" plugin was skipped because it requires a Plugin API version ("^1.0.0") that does not match your Composer installation ("2.0.0"). etc...

Same roadblock here. For the time being I'm using


    {
    "type": "package",
    "package": {
      "name": "junaidbhura/advanced-custom-fields-pro",
      "version": "5.9.1",
      "type": "wordpress-plugin",
      "dist": {
        "type": "zip",
        "url": "https://www.advancedcustomfields.com"
      },
      "require": {
          "junaidbhura/composer-wp-pro-plugins": "*"
      }
    }
  }
  ],
  "require": {
    "junaidbhura/advanced-custom-fields-pro": "*",

As seen here.

szepeviktor · 2021-01-22T19:53:53Z

If you remove the vendor directory (or the global plugin) you should be good to go.

MikeiLL · 2021-01-22T20:11:22Z

If you remove the vendor directory (or the global plugin) you should be good to go.

I had tried removing just vendor/composer and vendor/ffraenz, which seemed not to work.

Would I be using ffraenz/private-composer-installer version 5 and vlucas/phpdotenv v5?

Also what do you mean, please by or the global plugin?

szepeviktor · 2021-01-22T21:00:07Z

Would I be using ffraenz/private-composer-installer version 5 and vlucas/phpdotenv v5?

https://github.com/ffraenz/private-composer-installer#dependencies

Also what do you mean, please by or the global plugin?

You may have installed this plugin globally.

Refactor plugin and tests, port plugin to composer 2, refs #23

e4336af

ffraenz self-assigned this Jun 12, 2020

ffraenz added 2 commits June 13, 2020 01:23

Fix minor test issue

742cdd0

Configure Travis test for Composer 2

6a858ca

ffraenz force-pushed the v5 branch from cf86340 to 6a858ca Compare June 12, 2020 23:45

ffraenz added 4 commits June 13, 2020 17:10

Improve on test coverage

3bbee49

Update travis config for Composer 2 tests

ab12dca

Update travis config for Composer 2 tests (2)

b967600

Add test coverage to Composer 2 case

d91f7fa

ffraenz mentioned this pull request Jun 14, 2020

Composer 2: Allow plugins to override the URL before triggering the download composer/composer#8975

Merged

ffraenz mentioned this pull request Aug 19, 2020

Can you place the .env file in a directory above composer.json? #26

Closed

ffraenz linked an issue Aug 19, 2020 that may be closed by this pull request

Can you place the .env file in a directory above composer.json? #26

Closed

szepeviktor suggested changes Aug 19, 2020

View reviewed changes

.travis.yml Outdated Show resolved Hide resolved

composer.json Outdated Show resolved Hide resolved

mcaskill added a commit to mcaskill/composer-wp-pro-plugins that referenced this pull request Sep 7, 2020

Add support for Composer v2

89b8c2d

References: - ffraenz/private-composer-installer#25 Resolves junaidbhura#19

mcaskill added a commit to mcaskill/composer-wp-pro-plugins that referenced this pull request Sep 9, 2020

Add support for Composer v2

2ec64eb

References: - ffraenz/private-composer-installer#25 Resolves junaidbhura#19

ffraenz linked an issue Sep 10, 2020 that may be closed by this pull request

Support for Composer 2 #23

Closed

mcaskill added a commit to mcaskill/composer-wp-pro-plugins that referenced this pull request Sep 11, 2020

Add support for Composer v2

42bd20d

References: - ffraenz/private-composer-installer#25 Resolves junaidbhura#19

mcaskill reviewed Sep 12, 2020

View reviewed changes

src/PrivateComposerInstaller/Plugin.php Outdated Show resolved Hide resolved

test/PrivateComposerInstaller/PluginTest.php Outdated Show resolved Hide resolved

test/PrivateComposerInstaller/RemoteFilesystemTest.php Outdated Show resolved Hide resolved

szepeviktor reviewed Sep 12, 2020

View reviewed changes

test/PrivateComposerInstaller/RemoteFilesystemTest.php Outdated Show resolved Hide resolved

ffraenz and others added 3 commits September 14, 2020 13:07

Apply suggestions from code review, fix support for version 1.1.0 of …

ed91abb

…the plugin API Co-authored-by: Chauncey McAskill <chauncey@mcaskill.ca> Co-authored-by: Viktor Szépe <viktor@szepe.net>

Apply suggestion from code review

01be9b0

Co-authored-by: Chauncey McAskill <chauncey@mcaskill.ca>

Finalize code review changes

90bc822

ffraenz added 3 commits October 26, 2020 20:17

Configure Coveralls with GitHub actions

7637a9d

Fix typo

4bc5a9d

Fix broken task reference

cf1e9b8

Rephrase Coveralls configuration with GitHub actions

31f395a

ffraenz added 4 commits October 26, 2020 21:30

Configure xdebug locally and in GitHub actions

2d210af

Update coding standards and fix CS issues

d64a5cf

Only run CS checks once

6d193a7

Use GitHub action to finalize Coveralls report

5479971

ffraenz added 6 commits October 26, 2020 22:45

Update GitHub actions to choose correct Composer dependency version

c7d1159

Fix type hints

2529741

Run integration test cases outside the plugin folder

2e69cfd

Force plugin mirroring in integration test case

6193676

Have separated integration test cases for different phpdotenv versions

3a09613

Update README

5113c7b

ffraenz merged commit ced8193 into dev Oct 26, 2020

ffraenz mentioned this pull request Oct 27, 2020

Support plugins served by EDD #22

Open

ffraenz deleted the v5 branch October 30, 2020 09:56

dsturm mentioned this pull request Oct 30, 2020

Failing to install plugins gtap-dev/gravityforms-composer-installer#10

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Major update v5.0 #25

Major update v5.0 #25

ffraenz commented Jun 12, 2020 •

edited

mcaskill commented Sep 7, 2020 •

edited

ffraenz commented Sep 7, 2020

mcaskill commented Sep 7, 2020

ffraenz commented Sep 10, 2020

mcaskill commented Sep 12, 2020 •

edited by ffraenz

ffraenz commented Sep 14, 2020

mcaskill commented Sep 14, 2020

Seldaek commented Sep 15, 2020

szepeviktor commented Oct 26, 2020

szepeviktor commented Oct 26, 2020

alexkerber commented Oct 26, 2020

szepeviktor commented Oct 26, 2020

ffraenz commented Oct 26, 2020

alexkerber commented Oct 26, 2020

szepeviktor commented Oct 26, 2020

mcaskill commented Oct 26, 2020 •

edited

MikeiLL commented Jan 22, 2021

szepeviktor commented Jan 22, 2021

MikeiLL commented Jan 22, 2021

szepeviktor commented Jan 22, 2021

Major update v5.0 #25

Major update v5.0 #25

Conversation

ffraenz commented Jun 12, 2020 • edited

mcaskill commented Sep 7, 2020 • edited

ffraenz commented Sep 7, 2020

mcaskill commented Sep 7, 2020

ffraenz commented Sep 10, 2020

mcaskill commented Sep 12, 2020 • edited by ffraenz

Example:

In Composer 1.0:

In Composer 2.0:

ffraenz commented Sep 14, 2020

mcaskill commented Sep 14, 2020

Seldaek commented Sep 15, 2020

szepeviktor commented Oct 26, 2020

szepeviktor commented Oct 26, 2020

alexkerber commented Oct 26, 2020

szepeviktor commented Oct 26, 2020

ffraenz commented Oct 26, 2020

alexkerber commented Oct 26, 2020

szepeviktor commented Oct 26, 2020

mcaskill commented Oct 26, 2020 • edited

MikeiLL commented Jan 22, 2021

szepeviktor commented Jan 22, 2021

MikeiLL commented Jan 22, 2021

szepeviktor commented Jan 22, 2021

ffraenz commented Jun 12, 2020 •

edited

mcaskill commented Sep 7, 2020 •

edited

mcaskill commented Sep 12, 2020 •

edited by ffraenz

mcaskill commented Oct 26, 2020 •

edited