feature #5390 feature #4024 added a list-files command (clxmstaab, …

…keradus)

This PR was merged into the 2.19-dev branch.

Discussion
----------

feature #4024 added a `list-files` command

as discussed in #4024 this PR proposes a `list-files` command, which from a user-perspective can be used to utilize `xargs` to get some basic parallelization into php-cs-fixer.

this is especially interessting for CI build cases, in which the php-cs-fixer builtin caching mechanism doesn't work (e.g. because git does use the correct file modification stamps on clone or the CI build is running within a readonly environment).

example usage:
```
vendor/bin/php-cs-fixer list-files | xargs -n 10 -P 8 vendor/bin/php-cs-fixer fix --config=.php_cs.dist --using-cache=no
```
`-n` defines how many files a single subprocess process
`-P` defines how many subprocesses the shell is allowed to spawn for parallel processing (usually similar to the number of CPUs your system has)

see xargs help page: https://wiki.ubuntuusers.de/xargs/

as can be seen in #4024 (comment) you might get a perf boost by a factor of 3-6x depending on the number of files beeing fixed, number of CPUs etc.

this isn't the most elegant way of how parallelzation can be applied on the php-cs-fixer case, but its a really simple one which works with very simple means and therefore has a rather low entry barrier.
you don't need fancy php extension or any rather complicated setup. simple bash means are enought to get a decent perf boost.
also its pretty simple from php-cs-fixer perspective to keep the maintenance burden as low as possible.

in case you would consider this change acceptable, I am willing to provide automated tests, docs and everything else required and missing atm.

any opinions?

Commits
-------

ed65a9e Update doc/usage.rst
c95d430 Update doc/usage.rst
81753b4 try getRealPath()
a3ea03a fix test on windows
a32a8ff fix typo
add9542 to provide multiple files to fix command, you need to specify config file
d3f420d fis CS
d107a1f fix typo
47acf3a Update doc/usage.rst
172d577 fix deprecation
3f6a255 fix CS
d9dbec3 try to fix windows compat
06beee5 fix test
ac8773d .php_cs -> .php-cs-fixer.php
925f47c added in/out test
4e93714 moved ListFilesTest project into test/Fixtures/
725cc86 added separate path for SplFileInfo and sfSplFileInfo
444500c docs
7e92ca7 fix test-expectation
2f9cd68 fix test
5d2104d added ListFilesCommandTest
1f6f32b rm un-used imports
44bbe92 use relative paths instead of real-path to shorten output
22315d9 remove getHelp(), seems to be only used for the FixCommand
8c87ac9 Update src/Console/Command/ListFilesCommand.php
2a25c8c alphabetical order
149d757 removed un-used imports
302733d added a list-files command

doc/usage.rst

@@ -2,6 +2,9 @@
 Usage
 =====
 
+The ``fix`` command
+-------------------
+
 The ``fix`` command tries to fix as much coding standards
 problems as possible on a given file or files in a given directory and its subdirectories:
 
@@ -121,6 +124,35 @@ fixed but without actually modifying them:
 By using ``--using-cache`` option with ``yes`` or ``no`` you can set if the caching
 mechanism should be used.
 
+The ``list-files`` command
+--------------------------
+
+The ``list-files`` command will list all files which need fixing.
+
+.. code-block:: console
+
+    $ php php-cs-fixer.phar list-files
+
+The ``--config`` option can be used, like in the ``fix`` command, to tell from which path a config file should be loaded.
+
+.. code-block:: console
+
+    $ php php-cs-fixer.phar list-files --config=.php-cs-fixer.dist.php
+
+The output is build in a form that its easy to use in combination with ``xargs`` command in a linux pipe.
+This can be useful e.g. in situations where the caching might mechanism not available (CI, Docker) and distributing
+fixing across several processes might speedup the process.
+
+Note: You need to pass the config to the ``fix`` command, in order to make it work with several files being passed by ``list-files``.
+
+.. code-block:: console
+
+    $ php php-cs-fixer.phar list-files --config=.php-cs-fixer.dist.php | xargs -n 10 -P 8 php php-cs-fixer.phar fix --config=.php-cs-fixer.dist.php --path-mode intersection -v
+
+* `-n` defines how many files a single subprocess process
+* `-P` defines how many subprocesses the shell is allowed to spawn for parallel processing (usually similar to the number of CPUs your system has)
+
+
 Rule descriptions
 -----------------
 

src/Console/Application.php

@@ -15,6 +15,7 @@
 use PhpCsFixer\Console\Command\DescribeCommand;
 use PhpCsFixer\Console\Command\FixCommand;
 use PhpCsFixer\Console\Command\HelpCommand;
+use PhpCsFixer\Console\Command\ListFilesCommand;
 use PhpCsFixer\Console\Command\SelfUpdateCommand;
 use PhpCsFixer\Console\SelfUpdate\GithubClient;
 use PhpCsFixer\Console\SelfUpdate\NewVersionChecker;
@@ -52,8 +53,10 @@ public function __construct()
 
         $this->toolInfo = new ToolInfo();
 
+        // in alphabetical order
         $this->add(new DescribeCommand());
         $this->add(new FixCommand($this->toolInfo));
+        $this->add(new ListFilesCommand($this->toolInfo));
         $this->add(new SelfUpdateCommand(
             new NewVersionChecker(new GithubClient()),
             $this->toolInfo,

src/Console/Command/ListFilesCommand.php

@@ -0,0 +1,96 @@
+<?php
+
+/*
+ * This file is part of PHP CS Fixer.
+ *
+ * (c) Fabien Potencier <fabien@symfony.com>
+ *     Dariusz Rumiński <dariusz.ruminski@gmail.com>
+ *
+ * This source file is subject to the MIT license that is bundled
+ * with this source code in the file LICENSE.
+ */
+
+namespace PhpCsFixer\Console\Command;
+
+use PhpCsFixer\Config;
+use PhpCsFixer\ConfigInterface;
+use PhpCsFixer\Console\ConfigurationResolver;
+use PhpCsFixer\ToolInfoInterface;
+use SplFileInfo;
+use Symfony\Component\Console\Command\Command;
+use Symfony\Component\Console\Input\InputInterface;
+use Symfony\Component\Console\Input\InputOption;
+use Symfony\Component\Console\Output\OutputInterface;
+
+/**
+ * @author Markus Staab <markus.staab@redaxo.org>
+ *
+ * @internal
+ */
+final class ListFilesCommand extends Command
+{
+    protected static $defaultName = 'list-files';
+
+    /**
+     * @var ConfigInterface
+     */
+    private $defaultConfig;
+
+    /**
+     * @var ToolInfoInterface
+     */
+    private $toolInfo;
+
+    public function __construct(ToolInfoInterface $toolInfo)
+    {
+        parent::__construct();
+
+        $this->defaultConfig = new Config();
+        $this->toolInfo = $toolInfo;
+    }
+
+    /**
+     * {@inheritdoc}
+     */
+    protected function configure()
+    {
+        $this
+            ->setDefinition(
+                [
+                    new InputOption('config', '', InputOption::VALUE_REQUIRED, 'The path to a .php-cs-fixer.php file.'),
+                ]
+            )
+            ->setDescription('List all files being fixed by the given config.')
+        ;
+    }
+
+    protected function execute(InputInterface $input, OutputInterface $output)
+    {
+        $passedConfig = $input->getOption('config');
+        $cwd = getcwd();
+
+        $resolver = new ConfigurationResolver(
+            $this->defaultConfig,
+            [
+                'config' => $passedConfig,
+            ],
+            getcwd(),
+            $this->toolInfo
+        );
+
+        $finder = $resolver->getFinder();
+
+        /** @var SplFileInfo $file */
+        foreach ($finder as $file) {
+            if ($file->isFile()) {
+                $relativePath = str_replace($cwd, '.', $file->getRealPath());
+                // unify directory separators across operating system
+                $relativePath = str_replace('/', \DIRECTORY_SEPARATOR, $relativePath);
+
+                $output->writeln(escapeshellarg($relativePath));
+            }
+        }
+
+        return 0;
+    }
+}

tests/Console/Command/ListFilesCommandTest.php

@@ -0,0 +1,56 @@
+<?php
+
+/*
+ * This file is part of PHP CS Fixer.
+ *
+ * (c) Fabien Potencier <fabien@symfony.com>
+ *     Dariusz Rumiński <dariusz.ruminski@gmail.com>
+ *
+ * This source file is subject to the MIT license that is bundled
+ * with this source code in the file LICENSE.
+ */
+
+namespace PhpCsFixer\Tests\Console\Command;
+
+use PhpCsFixer\Console\Application;
+use PhpCsFixer\Console\Command\ListFilesCommand;
+use PhpCsFixer\Tests\TestCase;
+use PhpCsFixer\ToolInfo;
+use Symfony\Component\Console\Tester\CommandTester;
+
+/**
+ * @internal
+ *
+ * @covers \PhpCsFixer\Console\Command\ListFilesCommand
+ */
+final class ListFilesCommandTest extends TestCase
+{
+    public function testListWithConfig()
+    {
+        $commandTester = $this->doTestExecute([
+            '--config' => __DIR__.'/../../Fixtures/ListFilesTest/.php-cs-fixer.php',
+        ]);
+
+        $expectedPath = './tests/Fixtures/ListFilesTest/needs-fixing/needs-fixing.php';
+        // make the test also work on windows
+        $expectedPath = str_replace('/', \DIRECTORY_SEPARATOR, $expectedPath);
+
+        static::assertSame(escapeshellarg($expectedPath).PHP_EOL, $commandTester->getDisplay());
+    }
+
+    /**
+     * @return CommandTester
+     */
+    private function doTestExecute(array $arguments)
+    {
+        $application = new Application();
+        $application->add(new ListFilesCommand(new ToolInfo()));
+
+        $command = $application->find('list-files');
+        $commandTester = new CommandTester($command);
+
+        $commandTester->execute($arguments);
+
+        return $commandTester;
+    }
+}

tests/Fixtures/ListFilesTest/.php-cs-fixer.php

@@ -0,0 +1,20 @@
+<?php
+
+$finder = PhpCsFixer\Finder::create()
+    ->in([
+        __DIR__.'/needs-fixing/',
+    ])
+    ->exclude([
+        __DIR__.'/excluded/',
+    ])
+;
+
+$config = new PhpCsFixer\Config();
+return $config
+    ->setUsingCache(false)
+    ->setRules([
+        '@Symfony' => true,
+    ])
+    ->setRiskyAllowed(true)
+    ->setFinder($finder)
+    ;

tests/Fixtures/ListFilesTest/excluded/needs-fixing.php

@@ -0,0 +1,8 @@
+<?php
+
+function abc() {}
+
+function def()
+{
+
+}

tests/Fixtures/ListFilesTest/needs-fixing/needs-fixing.php

@@ -0,0 +1,8 @@
+<?php
+
+function abc() {}
+
+function def()
+{
+
+}