[Help Wanted] Increasing test coverage of Command classes

[Seldaek]

As you can see on this image, Command classes have very poor coverage currently (16% of lines covered in total):

Edit: To see latest guidelines on what to work on see #10796 (comment)

Here some examples of how to write sane integration tests for most commands:

composer/tests/Composer/Test/Command/RunScriptCommandTest.php

Lines 92 to 113 in 176d258

	public function testCanListScripts(): void
	{
	$this->initTempComposer([
	'scripts' => [
	'test' => '@php test',
	'fix-cs' => 'php-cs-fixer fix',
	],
	'scripts-descriptions' => [
	'fix-cs' => 'Run the codestyle fixer',
	],
	]);
	$appTester = $this->getApplicationTester();
	$appTester->run(['command' => 'run-script', '--list' => true]);
	$appTester->assertCommandIsSuccessful();
	$output = $appTester->getDisplay();
	$this->assertStringContainsString('Runs the test script as defined in composer.json.', $output, 'The default description for the test script should be printed');
	$this->assertStringContainsString('Run the codestyle fixer', $output, 'The custom description for the fix-cs script should be printed');
	}

composer/tests/Composer/Test/Command/ConfigCommandTest.php

Lines 23 to 52 in 176d258

	/**
	* @dataProvider provideConfigUpdates
	* @param array<mixed> $before
	* @param array<mixed> $command
	* @param array<mixed> $expected
	*/
	public function testConfigUpdates(array $before, array $command, array $expected): void
	{
	$this->initTempComposer($before);
	$appTester = $this->getApplicationTester();
	$appTester->run(array_merge(['command' => 'config'], $command));
	$appTester->assertCommandIsSuccessful($appTester->getDisplay());
	$this->assertSame($expected, json_decode((string) file_get_contents('composer.json'), true));
	}
	public function provideConfigUpdates(): \Generator
	{
	yield 'set scripts' => [
	[],
	['setting-key' => 'scripts.test', 'setting-value' => ['foo bar']],
	['scripts' => ['test' => 'foo bar']],
	];
	yield 'unset scripts' => [
	['scripts' => ['test' => 'foo bar', 'lala' => 'baz']],
	['setting-key' => 'scripts.lala', '--unset' => true],
	['scripts' => ['test' => 'foo bar']],
	];

composer/tests/Composer/Test/Command/BumpCommandTest.php

Lines 28 to 47 in 70f2dd6

	$this->initTempComposer($composerJson);
	$packages = [
	$this->getPackage('first/pkg', '2.3.4'),
	$this->getPackage('second/pkg', '3.4.0'),
	];
	$devPackages = [
	$this->getPackage('dev/pkg', '2.3.4.5'),
	];
	$this->createInstalledJson($packages, $devPackages);
	if ($lock) {
	$this->createComposerLock($packages, $devPackages);
	}
	$appTester = $this->getApplicationTester();
	$appTester->run(array_merge(['command' => 'bump'], $command));
	$json = new JsonFile('./composer.json');
	$this->assertSame($expected, $json->read());

If you feel like helping, PRs are very much welcome, I don't think we absolutely need 100% coverage but having the most common use cases covered would already be very valuable. Please target the main branch if you want to contribute some, and feel free to announce it here if you start some larger chunk of work to avoid duplicating efforts.

[theoboldalex]

@Seldaek I'd love to take on some of this work. I have opened a first PR here #10932 and will start to work my way through the commands.

[Seldaek]

@theoboldalex great, thanks! Maybe wait until I get a chance to review the first one to make sure you don't run off and do more of them with the wrong assumptions. I'll try to get to it soon.

Also note the listing above is a bit outdated as I tried to work on this here and there when touching commands the last few months. But there is still plenty to be done :)

[ralflang]

No promises but I may make the occassional PR every now and then.
Might be a good learning exercise. I have been playing around with the Command, Plugin, Capable ... parts of the codebase last month.

[Seldaek]

Update, we're at 36% (up from 16!) already now:

[tomekpryjma]

I'd like to help out with this now and again - is there a way to see the current coverage?

[Seldaek]

@tomekpryjma nope sorry, gotta run the tests with --coverage-html foo yourself then check in foo/index.html for the output.

[Seldaek]

Now at 41%! Here are the classes still needing the most effort IMO:

• BaseDependencyCommand.php (needs to be tested via why/why-not commands)
• CheckPlatformReqsCommand.php (WIP Added CheckPlatformReqsCommand Test #11079)
• FundCommand.php
• HomeCommand.php (WIP Test home command #11254 probably only can test with the --show flag enabled)
• InstallCommand.php
• SuggestsCommand.php (WIP Tests for Audit/Remove/Suggests Commands #11162)
• ValidateCommand.php
• RemoveCommand.php
• UpdateCommand.php with --interactive flag (needs to run the command in interactive mode to test the uncovered bits, and use $applicationTester->setInputs() to configure/fake the user inputs)
• InitCommand.php & PackageDiscoveryTrait.php (needs to run the command in interactive mode to test the uncovered bits, and use $applicationTester->setInputs() to configure/fake the user inputs)
• ShowCommand.php (tested quite a bit but still only 40% covered as it is a large one)
• GlobalCommand.php (might be tricky to test as it requires some internal knowledge)

These need an install to be run to actual get a working vendor dir, using packages with type metapackage helps making them installable without actual source:

• StatusCommand.php
• ReinstallCommand.php

[giulio-Joshi]

Is this issue still relevant?

[Seldaek]

@giulio-Joshi yes for sure, not much happened since my last comment above yours.

[giulio-Joshi]

Thanks for your answer.
I was planning on adding some more tests in the area, but paused the activity since my related PR had no feedback.
I was wondering if there was something wrong about it.

[Seldaek]

No it's just been 8 days, I'll get to it :)

[yesdevnull]

Putting my hand up to work on tests for SuggestsCommand. Will submit a PR with my progress in the next ~24 hours, I currently have 100% for that command but I want to finesse the tests so they're not just chasing coverage.

PR created here: #11162.

[siims-biz]

@Seldaek What kind of skill/talent/tools are required/useful to contribute with testing?

[Seldaek]

@siims-biz knowledge of PHPUnit, some knowledge of composer (from user perspective good, internals better), the easy ones have been covered already though so if you aren't super familiar with all this I would say maybe rather find something else to work on, but up to you :)

[stof]

Now at 52%

[theoboldalex]

I'm going to make a start on testing ReinstallCommand tonight and will try to pick up StatusCommand after that if time permits.

[Seldaek]

54.4% now, thanks everyone involved so far!

Here are the classes still needing the most effort, but overall we are really in a much better spot now I would say with the most critical commands pretty well covered:

• BaseDependencyCommand.php (needs to be tested via why/why-not commands)
• UpdateCommand.php with --interactive flag (needs to run the command in interactive mode to test the uncovered bits, and use $applicationTester->setInputs() to configure/fake the user inputs)
• InitCommand.php & PackageDiscoveryTrait.php (needs to run the command in interactive mode to test the uncovered bits, and use $applicationTester->setInputs() to configure/fake the user inputs)
• ShowCommand.php (tested quite a bit but still only 40% covered as it is a large one)
• GlobalCommand.php (might be tricky to test as it requires some internal knowledge)

These need an install to be run to actual get a working vendor dir, using packages with type metapackage helps making them installable without actual source:

• StatusCommand.php

[rad8329]

Will take test cases for commands why and why-not

[rad8329]

Hi @Seldaek, I'd love to continue working on the above checklist, but I'm still waiting for Tests for base dependency command #11547 feedback, once I've received first feedback I could replicate that same approach in my upcoming PRs, thanks

[rad8329]

Will try to complete show command test coverage, currently it is 55%

[stof]

@Seldaek this should be re-opened. The PR #11547 contributed to this by adding more tests, but we are still not done yet.

[stof]

now at 58.11%:

[greew]

FYI - I'm writing a lot of test for ShowCommand :)