Post merge test strategy

Our current workflow is sub-optimal. It is difficult to develop (no fixtures, no feature isolation) and very hard to test (no automated regression tests).

Current issues

• Some tests are very slow due to database
• Effectively no functional tests for sulu-standard - regression testing is done by hand
• Speed: Speed of tests made worse when we merge the repositories
• Speed: Speed of tests made worse by running for both MySQL and Postgres
• Complexity: Some tests are very complicated (in terms of mocking) - is this a problem?
• No fixtures: We have no fixtures. Testing bugs is made difficult because data has to be created by hand everytime.

Proposal

• Speed: Do not test with either MySQL or Postgres, use sqlite instead (database dropping and recreation can be almost instantaneous).
  • MySQL and Postgres can both be tested instead in Sulu-Standard as regression tests.
• Speed: Replace any functional tests with Behat tests in sulu-standard where possible.
• Add full functional tests to sulu-standard
• Fixtures: Create a test website bundle which provides a complex, fully-featured site.

Actions

These actions could be taken after the repositories are merged into one:

• Refactor SuluTestBundle to use the SymfonyCMF testing component
  • Refactor existing database tests to use sqlite via. Testing component
  • Add a standard Sulu kernel and configuration to SuluTestBundle
• Add behat to sulu-standard
  • Behat and Mink should be added as dependencies to SuluStandard, also a browser implementation
  • We should write a full suite of tests
    • One suite per bundle
• Fixtures: Implement a test site bundle which can generate fixtures and provides direct, isolated, access to all the content types.