Behat support

[dkarlovi]

Adding Behat support shouldn't be too difficult as it has same kind of listener support as PHPUnit does.

[dmaicher]

Should be doable indeed 👍

But only for some drivers like goutte that do not actually use any real webserver within a separate php process (like with selenium).

[dkarlovi]

Maybe supporting official Symfony2Extension would work?

[dmaicher]

yes indeed with that symfony2 driver it might work 😊

[dmaicher]

@dkarlovi I tried it and it works nicely 😄

https://github.com/dmaicher/symfony-flex-behat-test

Are you also using behat with the symfony2 driver? Maybe you can also give it a try.

I will soon add this trait to my bundle

https://github.com/dmaicher/symfony-flex-behat-test/blob/master/features/bootstrap/TransactionalDatabaseTrait.php

So all you have to do is use it like this: https://github.com/dmaicher/symfony-flex-behat-test/blob/master/features/bootstrap/FeatureContext.php#L5

[dkarlovi]

@dmaicher I'll try it out now. BTW why add it as a Trait, it's OK to have your own context.

[dkarlovi]

@dmaicher I have a weird problem which you might know a cure to: with your proposed solution, it seems the first time StaticDriver::beginTransaction() is called, there's no static connections yet and the call goes nowhere (so, no transaction is started).

I've done this and get this output:

StaticDriver::connect(keepStatic=false): 0
Database::setup()
StaticDriver::setKeepStaticConnections(true)
StaticDriver::beginTransaction(): 0 # NOTE: no static connections are available so no TX started!
Database::createUser() # actually stored to DB
StaticDriver::connect(keepStatic=true): 1
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
Database::createUser()
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
Database::createUser()
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
Database::createUser()
StaticDriver::rollBack(): 1
StaticDriver::setKeepStaticConnections(false)

This means my createUser() actually stores and fails (duplicate entry) all subsequent times.

[dmaicher]

This is my output for the repository I posted above:

StaticDriver::setKeepStaticConnections(true)
StaticDriver::beginTransaction(): 0
StaticDriver::connect(keepStatic=true): 1
Write to DB from inside Controller
StaticDriver::connect(keepStatic=true): 1
Write to DB from inside Controller
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
StaticDriver::connect(keepStatic=true): 1
Write to DB from inside Controller
StaticDriver::rollBack(): 1
StaticDriver::setKeepStaticConnections(false)

In the controller after the flush I just added:

\error_log("Write to DB from inside Controller\n", 3, 'app/behat.log');

Its actually ok if the very first call to StaticDriver::beginTransaction() does not do anything as whenever a new connection is created (and keepStatic=true) it will immediately have a transaction started already: https://gist.github.com/dkarlovi/04c322789685cc5291a7537d50f426b2#file-staticdriver-php-L50

So can you check when you create the user for the first time (Database::createUser()) what the transaction nesting level on the Doctrine connection is?

[dkarlovi]

Added

        $this->connect();

        ++$this->_transactionNestingLevel;

       // this
        \error_log('Doctrine::beginTransaction(): '.$this->_transactionNestingLevel ."\n", 3, '/app/behat.log');

to Doctrine\DBAL\Connection::beginTransaction()

(note: only the number by Doctrine::beginTransaction() is the level)

StaticDriver::connect(keepStatic=false): 0
Database::setup()
StaticDriver::setKeepStaticConnections(true)
StaticDriver::beginTransaction(): 0
Database::createUser()
Doctrine::beginTransaction(): 1
Doctrine::beginTransaction(): 1
StaticDriver::connect(keepStatic=true): 1
Doctrine::beginTransaction(): 1
Doctrine::beginTransaction(): 2
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
Database::createUser()
Doctrine::beginTransaction(): 1
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
Database::createUser()
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
Database::createUser()
StaticDriver::rollBack(): 1
StaticDriver::setKeepStaticConnections(false)

[dmaicher]

ok so for Database::createUser() it calls Doctrine::beginTransaction(): 1 before flushing it?

[dkarlovi]

It just flushes it.

    private function createUser(string $username, string $password, array $roles): User
    {
        $user = new User();
        // ..setters

        \error_log('Database::createUser()'."\n", 3, __DIR__ .'/../../behat.log');

        $manager = $this->getRegistry()->getManagerForClass(User::class);
        $manager->persist($user);
        $manager->flush();

        return $user;
    }

[dmaicher]

Ok interesting 😕

This is my output:

StaticDriver::setKeepStaticConnections(true)
StaticDriver::beginTransaction(): 0
StaticDriver::connect(keepStatic=true): 1
Doctrine::beginTransaction(): 1
Doctrine::beginTransaction(): 2
Write to DB from inside Controller
StaticDriver::connect(keepStatic=true): 1
Doctrine::beginTransaction(): 1
Doctrine::beginTransaction(): 2
Write to DB from inside Controller
StaticDriver::rollBack(): 1
StaticDriver::beginTransaction(): 1
StaticDriver::connect(keepStatic=true): 1
Doctrine::beginTransaction(): 1
Doctrine::beginTransaction(): 2
Write to DB from inside Controller
StaticDriver::rollBack(): 1
StaticDriver::setKeepStaticConnections(false)

So whats the difference between your project and the one I posted above? 😄

[dkarlovi]

Guess that's for me to find out. 🙈

I'll keep you posted if I figure anything out, thank you very much for your help here. 👍

[dkarlovi]

Just to confirm: as I see you have no DB setup step, I assume you're using a pre-setup DB instance (with the proper schema etc?) That might be it: I have a BeforeScenario step which sets up the database schema so the tests can run unsupervised on new CI servers. Probably need to move it to the bootstrap.

[dmaicher]

@dkarlovi yes indeed I did not add any bootstrapping for the DB on that project.

Hmm ok I assumed that all DB changes done within a whole behat suite (hence this hook https://github.com/dmaicher/symfony-flex-behat-test/blob/master/features/bootstrap/TransactionalDatabaseTrait.php#L12) should be transactional / rolled back.

So you would have to make sure to call StaticDriver::setKeepStaticConnections(true); after your DB bootstrapping

[dkarlovi]

@dmaicher I have not been able to get it to work, sadly. Seems the setup-vs-run is a non-trivial setup so it might make sense to try and get the whole thing running, might reveal some bugs in the implementation.

For example, you could use a custom Kernel instead of a bootstrap which sets up your schema.

[dmaicher]

@dkarlovi but does not make sense to run DB bootstrapping within @BeforeScenario? So you do that before every scenario?

To do it once before all scenarios seems more logical to me? I mean ideally all changes from scenarios are rolled back anyway.

[dkarlovi]

Yes, I've moved the setup procedure to the Kernel, as linked.

My point is, Behat integration seems to be more involved than I expected if you account for the setup step too.

[kojidev]

@dmaicher I tried your trait and it seems to be working, but after some time it explodes with exception:

  [Doctrine\DBAL\DBALException]                                                              
  An exception occured while establishing a connection to figure out your platform version.  
  You can circumvent this by setting a 'server_version' configuration value                  
                                                                                             
  For further information have a look at:                                                    
  https://github.com/doctrine/DoctrineBundle/issues/673
                               
  [Doctrine\DBAL\Exception\ConnectionException]                                                 
  An exception occurred in driver: SQLSTATE[08006] [7] FATAL:  sorry, too many clients already  
  FATAL:  sorry, too many clients already

  [Doctrine\DBAL\Driver\PDOException]                          
  SQLSTATE[08006] [7] FATAL:  sorry, too many clients already  
  FATAL:  sorry, too many clients already

  [PDOException]                                               
  SQLSTATE[08006] [7] FATAL:  sorry, too many clients already  
  FATAL:  sorry, too many clients already

Process finished with exit code 1

[dmaicher]

@kojidev hmm interesting. Seems for some reasons too many new connections are created?

Can I see your behat yml config?

[kojidev]

@dmaicher

Seems for some reasons too many new connections are created?

Well idk, as soon as I exclude bundle everything works well again.

Before I ran into your bundle I tried (and still trying) to make a workaround, like listen to onFlush and record everything that have been deleted, inserted, updated and undo after every scenario, but I have to store all these updates, deletes and inserts in static property of an event listener, because Symfony2Extension boots Kernel everytime I do a request (multiple times per scenario). May be it somehow related to this

Can I see your behat yml config?

default:
  gherkin:
    cache: ~ # https://github.com/Behat/Behat/issues/1076#issuecomment-339576159
  suites:
    default:
      contexts: [ Tests\Behat\AppContext ]
      paths:
      - '%paths.base%/features'
  extensions:
    Behat\Symfony2Extension:
      kernel:
        bootstrap: "vendor/autoload.php"
    Behat\MinkExtension:
      sessions:
        default:
          symfony2: ~

[dkarlovi]

It might make sense to create a full-blown project (with bootstraping the database schema, fixtures, etc) and have this integrated properly, fixing all these issues we're having. I'll try to find time to do it, will link back here.

[dmaicher]

@dkarlovi yes that would make sense indeed 👍 Feel free to fork https://github.com/dmaicher/symfony-flex-behat-test and add more stuff to it

[dkarlovi]

@dmaicher Sounds good. 👍

[B-Galati]

It works really well thanks ;)

[mnapoli]

Seems to work fine as well for me, here is a recap for those interested and too lazy (like me) to read the whole thread (supposing you use Symfony Flex):

• install dmaicher/doctrine-test-bundle
• create config/packages/test/dama_doctrine_test_bundle.yaml with this content
• in your FeatureContext add (full source):

    /**
     * @BeforeSuite
     */
    public static function beforeSuite()
    {
        StaticDriver::setKeepStaticConnections(true);
    }
    /**
     * @BeforeScenario
     */
    public function beforeScenario()
    {
        StaticDriver::beginTransaction();
    }
    /**
     * @AfterScenario
     */
    public function afterScenario()
    {
        StaticDriver::rollBack();
    }
    /**
     * @AfterSuite
     */
    public static function afterSuite()
    {
       StaticDriver::setKeepStaticConnections(false);
    }

See dmaicher/symfony-flex-behat-test for a complete example.

And btw thank you so much @dmaicher for this package that is incredibly useful ❤️

[dkarlovi]

Agreed, we can close here. <3

[mnapoli]

@dkarlovi I wasn't expecting this to be closed but that's actually a good thing 😄 : I opened a PR to document all of this in the README: #80