Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Git] Drop most test files & unsupported features from builds to reduce size #2769

Closed
wants to merge 1 commit into from

Conversation

andrerom
Copy link
Contributor

@andrerom andrerom commented Sep 21, 2019

Drops most tests and unsupported features (old broken elasticsearch 1.5 search prototype, this is removed in master and will likely come re-written next year as new package instead).

Main motivation:
A step to reduce size of install in order to in the end enable use on setups such as AWS Lambda. Same as done for Symfony 4.4 recently here: symfony/symfony#33579

Size change:

  • 7.5: 5.9mb compressed (download), 41.6mb extracted (disk)
  • this: 3.7mb compressed (download), 22.8mb extracted (disk)

Tested using git archive --prefix=test/ -o test.zip HEAD

⚠️ Impact:
Community bundles that relies on Test classes and fixtures in kernel will for testing have to force Composer to install using sources instead of tags (--prefer-source), at least for this package.

Todo:

  • Test against SolrBundle (relies on integration test)
  • Get Community to test their packages that rely on kernel test classes
  • TBD: Identify a minimal safe change for LTS branches and more aggressive stripping for master(?)

Motivation: Reduce Size of install in order to enable use on setups such as AWS Lambda.

Tested using "git archive --prefix=test/ -o test.zip HEAD":
- 7.5: 5.9mb compressed, 41.6mb extracted
- this: 3.7mb compressed, 22.8mb extracted

Todo:
- Test against SolrBundle
- Get Netgen to test their packages that rely on kernel test classes
@andrerom andrerom changed the title [TBD][Git] Drop most test files from builds to reduce size [WIP][Git] Drop most test files from builds to reduce size Sep 21, 2019
@andrerom andrerom changed the base branch from 7.5 to master September 29, 2019 17:16
@andrerom andrerom changed the title [WIP][Git] Drop most test files from builds to reduce size [8.x][Git] Drop most test files from builds to reduce size Sep 29, 2019
.gitattributes Show resolved Hide resolved
.gitattributes Show resolved Hide resolved
.gitattributes Show resolved Hide resolved
@andrerom andrerom changed the base branch from master to 7.5 September 30, 2019 15:13
@andrerom andrerom changed the title [8.x][Git] Drop most test files from builds to reduce size [Git] Drop most test files from builds to reduce size Sep 30, 2019
@andrerom
Copy link
Contributor Author

@pspanja / @emodric Any clear dependencies on test classes / fixtures on your sides?

@emodric
Copy link
Contributor

emodric commented Oct 1, 2019

@andrerom I know that at least Tags Bundle uses test classes and fixtures from kernel. This can probably be remedied by using source as the preferred install.

@gggeek
Copy link
Contributor

gggeek commented Oct 1, 2019

Agree: any downstream that needs Test stuff from Kernel should be able to set up composer to do a Source install in order to get all the goodies

@andrerom
Copy link
Contributor Author

andrerom commented Oct 1, 2019

ok, I'll look into the todos here and wrap this up if any changes are needed. Thanks for the feedback so far!

🗣 If anyone sees this differently, please let us know!

@andrerom andrerom changed the title [Git] Drop most test files from builds to reduce size [Git] Drop most test files & unsupported features from builds to reduce size Oct 1, 2019
@andrerom
Copy link
Contributor Author

Finally got around to test archive from this on solr bundle, it fails on first test with:


1) eZ\Publish\API\Repository\Tests\Values\User\Limitation\ContentTypeLimitationTest::testContentTypeLimitationAllow
Cannot create a repository with predefined user. Check the UserService or RoleService implementation. 

Exception: Symfony\Component\DependencyInjection\Exception\RuntimeException: Please add the class to service "ezpublish.api.storage_engine.legacy.connection" even if it is constructed by a factory since we might need to add method calls based on compile-time checks. in /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Compiler/CheckDefinitionValidityPass.php:48
Stack trace:
#0 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Compiler/Compiler.php(140): Symfony\Component\DependencyInjection\Compiler\CheckDefinitionValidityPass->process(Object(Symfony\Component\DependencyInjection\ContainerBuilder))
#1 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/ContainerBuilder.php(789): Symfony\Component\DependencyInjection\Compiler\Compiler->compile(Object(Symfony\Component\DependencyInjection\ContainerBuilder))
#2 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Base/ServiceContainer.php(190): Symfony\Component\DependencyInjection\ContainerBuilder->compile()
#3 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Base/ServiceContainer.php(139): eZ\Publish\Core\Base\ServiceContainer->getContainer()
#4 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Base/ServiceContainer.php(83): eZ\Publish\Core\Base\ServiceContainer->initializeContainer()
#5 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/SetupFactory/Legacy.php(481): eZ\Publish\Core\Base\ServiceContainer->__construct(Object(Symfony\Component\DependencyInjection\ContainerBuilder), '/Users/andrerom...', '/Users/andrerom...', true, true)
#6 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/SetupFactory/Legacy.php(437): eZ\Publish\API\Repository\Tests\SetupFactory\Legacy->getServiceContainer()
#7 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/SetupFactory/Legacy.php(341): eZ\Publish\API\Repository\Tests\SetupFactory\Legacy->getDatabaseConnection()
#8 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/SetupFactory/Legacy.php(321): eZ\Publish\API\Repository\Tests\SetupFactory\Legacy->createSchema('/Users/andrerom...')
#9 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/SetupFactory/Legacy.php(141): eZ\Publish\API\Repository\Tests\SetupFactory\Legacy->initializeSchema()
#10 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/tests/lib/SetupFactory/LegacySetupFactory.php(39): eZ\Publish\API\Repository\Tests\SetupFactory\Legacy->getRepository(false)
#11 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/BaseTest.php(55): EzSystems\EzPlatformSolrSearchEngine\Tests\SetupFactory\LegacySetupFactory->getRepository(false)
#12 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/Framework/TestCase.php(838): eZ\Publish\API\Repository\Tests\BaseTest->setUp()
#13 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/Framework/TestResult.php(693): PHPUnit\Framework\TestCase->runBare()
#14 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/Framework/TestCase.php(796): PHPUnit\Framework\TestResult->run(Object(eZ\Publish\API\Repository\Tests\Values\User\Limitation\ContentTypeLimitationTest))
#15 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/Framework/TestSuite.php(746): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
#16 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/Framework/TestSuite.php(746): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#17 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/Framework/TestSuite.php(746): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#18 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(652): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#19 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/TextUI/Command.php(206): PHPUnit\TextUI\TestRunner->doRun(Object(PHPUnit\Framework\TestSuite), Array, true)
#20 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/src/TextUI/Command.php(162): PHPUnit\TextUI\Command->run(Array, true)
#21 /Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/phpunit/phpunit/phpunit(61): PHPUnit\TextUI\Command::main()
#22 {main}

/Users/andrerom/Sites/git/ezplatform-solr-search-engine/vendor/ezsystems/ezpublish-kernel/eZ/Publish/API/Repository/Tests/BaseTest.php:74

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

@alongosz and @adamwojs Should we keep this for master only maybe? Or do we assume stable packages like solr fetches kernel from dev/source always?

@alongosz
Copy link
Member

alongosz commented Dec 2, 2019

Finally got around to test archive from this on solr bundle, it fails on first test with:


1) eZ\Publish\API\Repository\Tests\Values\User\Limitation\ContentTypeLimitationTest::testContentTypeLimitationAllow
Cannot create a repository with predefined user. Check the UserService or RoleService implementation. 

Exception: Symfony\Component\DependencyInjection\Exception\RuntimeException: Please add the class to service "ezpublish.api.storage_engine.legacy.connection" even if it is constructed by a factory since we might need to add method calls based on compile-time checks.
(...)

Should we keep this for master only maybe? Or do we assume stable packages like solr fetches kernel from dev/source always?

@andrerom the fact that test setup from Solr relies on test setup from Kernel is a mistake. Usually those namespaces are autoload-dev (obviously not the case for Kernel), so they shouldn't even be accessible. So, no, I would not make such assumption. The direction here should be rather fixing test setup for Kernel and Solr.
The only thing that should be exposed from Kernel is SetupFactory (as the test API), other portions of test matrix should not depend on Kernel. They could depend on some internal dev package though if we see that there's no way to keep it DRY otherwise.

Though, I'm not following how the mentioned failure is related to reduced build. Are some classes missing which results in ezpublish.api.storage_engine.legacy.connection not being built?

@andrerom
Copy link
Contributor Author

andrerom commented Dec 2, 2019

Actually I'm not entirely sure. Might actually be unrelated.
I have limited time to look at this so if anyone feels like don't hesitate to take over, otherwise I'll try to dig into that at a later point.

@andrerom andrerom closed this May 12, 2020
@andrerom andrerom deleted the drop_files_from_builds branch May 12, 2020 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
4 participants