sqlalchemy.orm.exc.DetachedInstanceError in unit test with socketio.test_client

[LostInDarkMath]

Describe the bug
I just included Flask-SocketIO in my project which is based on Flask-SQLAlchemy and wrote some tests. But I in a simple test I get the following error:

sqlalchemy.orm.exc.DetachedInstanceError: Instance <User at 0x2812f970> is not bound to a Session; attribute refresh operation cannot proceed (Background on this error at: http://sqlalche.me/e/14/bhk3)

The error causing test can be reduced to the following:

def test_receive_notifications_on_new_message_via_websockets(self) -> None:
    socketio_client = self.socketio.test_client(app=self.app, flask_test_client=self.client, headers=self.header())
    self._send_message(sender=self.dummy2, receiver=self.dummy)  # it fails here

If I comment out the the first line that creates the socketio test client, the errors does not appear. This is quite strange because I have no idea why the creation of socketio test client can lead to detached instances.
Do you have any idea whats going on here?

The dummy users self.dummy and self.dummy2 are Wrappers for an ORM user class. The instances are created inside the setup() of the parent test class. I use them in almost any of my 300+ tests and I never had any problems before.

I hope you can help me here.

Additional context
I do not think that I use Flask-SocketIO wrong because if I start the server and my client everything works perfectly fine. Additionally, my other simple socketio tests also work perfectly:

    def test_establish_websocket_connection_without_credentials(self) -> None:
        socketio_client = self.socketio.test_client(app=self.app, flask_test_client=self.client)
        self.assertFalse(socketio_client.is_connected())

    def test_establish_websocket_connection_successful_and_receive_initial_data(self) -> None:
        socketio_client = self.socketio.test_client(app=self.app, flask_test_client=self.client, headers=self.header())
        self.assertTrue(socketio_client.is_connected())
        res_list = socketio_client.get_received()
        self.assertEqual(1, len(res_list))
        # more assertions here

[miguelgrinberg]

Two important things are missing:

1. the complete stack trace of the error
2. the code that fails, which I assume is the _send_message() method of your unit test class?

[LostInDarkMath]

I did some further investigation on this and I think I found out why the instances got detached. In the connect() method of test_client.py you push and pop an flask app context with these lines:

  with self.app.app_context():
            self.socketio.server._handle_eio_message(self.eio_sid,  pkt.encode())

But this calls the session.remove() in flask-SQLAlchemy:

  @app.teardown_appcontext
        def shutdown_session(response_or_exc):
            if app.config['SQLALCHEMY_COMMIT_ON_TEARDOWN']:
                warnings.warn(
                    "'COMMIT_ON_TEARDOWN' is deprecated and will be"
                    " removed in version 3.1. Call"
                    " 'db.session.commit()'` directly instead.",
                    DeprecationWarning,
                )

                if response_or_exc is None:
                    self.session.commit()

            self.session.remove()
            return response_or_exc

This leads to the DetachedInstanceError in my case, because I create a User via ORM, then I connect the SocketIO test client and after that I read some properties of the ORM user.

So my question is: How can I solve this problem?

[LostInDarkMath]

This is a bad way? But that's the way you did it in your Book Flask Web Development: https://github.com/miguelgrinberg/flasky/blob/master/tests/test_api.py#L110

I'm a bit confused now. I just did it the way you did it in your book.

[miguelgrinberg]

Well, do you understand what I'm saying? Do we agree that this is a bad idea? You haven't shown me actual code, so I can only comment on the example code you are showing me. The print(user.username) that you added in your test triggers a database query. Once the error is removed, this query is going to run under the context of a session that is not clean, the state of that session is going to be whatever the connect event handler in your application left it.

Flask is permissive in this respect, it does let you reuse sessions, and for that reason the test that you linked from my book's project works fine. Remember that I asked to see your code? This is why, depending on what you do, your usage might be okay (like my test) or not okay, and I wanted to see which is it.

As I said above, "I'm not sure how to address this problem". I can remove the error and let you share your database sessions left and right. That is going to make it easier to make mistakes though. It would be better to not reuse your database session.

[LostInDarkMath]

This is why, depending on what you do, your usage might be okay (like my test).

I'm aware of this. The thing is that the shared session never caused any problems before. Like your flasky tests. Sure, uncommitted changes or forgotten rollbacks could cause problems. But in fact, these were never problems for me. So in my point of view the call to session.remove() is the problem here. The socketIO test client closes the session during connecting. I do not know if this is really necessary. But if not, I think that this should be removed. Currently it is just not possible to reuse the the session because the test client prevent that and do not think that it is a good thing to force your users to maintain multiple sessions.

[miguelgrinberg]

I'm aware of this. The thing is that the shared session never caused any problems before.

You can say the same thing of all other bad practices. They never cause any problems until they do. :)

I'll think about how to address this problem. I hope you understand that I cannot base a solution on the fact that this has never been a problem "for you". I want to find a good solution, not something that works in your case but potentially creates issues for other users.

[LostInDarkMath]

Sure. I believe that there is a solution that does not cause any problems for others. I'm looking forward for your solution. Thanks for your time and your fast responses!

[LostInDarkMath]

One last question concerning this:

It would be better to not reuse your database session.

Do you have an example how to do this? You said this is best practice but I dont find a way how to do this. There is also an open question on StackOverflow for this: https://stackoverflow.com/questions/44744183/how-to-isolate-sqlalchemy-session-between-tests-and-tested-code

[miguelgrinberg]

All you need to do is not use model instances that you obtained in a previous logical session.

For example, this is what you are doing in the test that you posted here as an example:

• Get a user (this starts a session and leaves it open)
• Invoke a Socket.IO endpoint (the session is already dirty, Flask-SocketIO before the fix closes it at the end)
• Use the user that you obtained in item 1, which is now detached because the session was closed.

A better way to do this would be:

• Get a user, do whatever you need to do with it and close the session
• Run the Socket.IO endpoint, now with a clean session
• Get the user again, and do what you need to do with it, close the session when you are done.

Note that SQLAlchemy 1.4 (which is not fully supported by Flask-SQLAlchemy) introduced context managers for sessions. This is going to help prevent this type of mistakes, because you have more control over the life of a session without having to manually close it.