Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alt: Add a session backend to store session data in the database #21478

Merged
merged 14 commits into from Feb 15, 2022

Conversation

jedcunningham
Copy link
Member

@jedcunningham jedcunningham commented Feb 9, 2022

This is an alternate approach to #21167, the primary difference being we don't need to vendor in Flask-Session.

The downside, however, is the weirdness of having our own session model that isn't actually used other than to remove the table during db reset. Is there a better way to accomplish that?

setup.cfg Outdated Show resolved Hide resolved
airflow/utils/db.py Outdated Show resolved Hide resolved
airflow/www/extensions/init_session.py Outdated Show resolved Hide resolved
airflow/www/session.py Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Feb 9, 2022

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Feb 9, 2022
Copy link
Contributor

@blag blag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ashb
Copy link
Member

ashb commented Feb 10, 2022

https://github.com/apache/airflow/blob/ee20a03908a2f3ad37dbfcec71a19f37f1446088/airflow/utils/db.py#L1057

We can drop the table like this without needing a model


def upgrade():
"""Apply add session table to db"""
op.create_table(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also add an ORM model for this table so that it's not only through migration file that the table can be created.
I'm working on a change that would have new DBs created through the ORM rather than going through the migration files. #21462

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do it like this I think

from flask.session import SqlAlchemySessionInterface
from flask_appbuilder import SQLA
...

    db = SQLA()
    db.session = settings.Session
    SessionModel = SqlAlchemySessionInterface(db=db).sql_session_model

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(If we should or not is the next question)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what it ends up looking like, when considering #21462's changes:

--- a/airflow/utils/db.py
+++ b/airflow/utils/db.py
@@ -606,12 +606,21 @@ def initdb(session: Session = NEW_SESSION):
     from airflow.models.base import Base
     from airflow.www.app import create_app
 
+    from airflow.www.session import AirflowDatabaseSessionInterface
+    from flask_sqlalchemy import SQLAlchemy
+
     Base.metadata.create_all(settings.engine)
     # Stamp migration head with alembic
     config = _get_alembic_config()
     command.stamp(config, "head")
     # sets up the db; Sync permissions etc
-    create_app(config={'UPDATE_FAB_PERMS': True})
+    app = create_app(config={'UPDATE_FAB_PERMS': True})
+
+    # Create `session` table
+    db = SQLAlchemy(app)
+    AirflowDatabaseSessionInterface(app=app, db=db, table='session', key_prefix='')
+    db.create_all()
+
     if conf.getboolean('core', 'LOAD_DEFAULT_CONNECTIONS'):
         create_default_connections(session=session)
 

Dockerfile.ci Outdated Show resolved Hide resolved
@jedcunningham jedcunningham force-pushed the alt_db-session-backend branch 2 times, most recently from fbd11aa to b0d1e05 Compare February 15, 2022 15:01
@jedcunningham jedcunningham merged commit da9d086 into apache:main Feb 15, 2022
@jedcunningham jedcunningham deleted the alt_db-session-backend branch February 15, 2022 17:57
@jedcunningham jedcunningham added this to the Airflow 2.2.4 milestone Feb 15, 2022
@jedcunningham jedcunningham added the type:bug-fix Changelog: Bug Fixes label Feb 15, 2022
jedcunningham added a commit that referenced this pull request Feb 15, 2022
Co-authored-by: Jed Cunningham <jedcunningham@apache.org>
(cherry picked from commit da9d086)
jedcunningham added a commit that referenced this pull request Feb 17, 2022
Co-authored-by: Jed Cunningham <jedcunningham@apache.org>
(cherry picked from commit da9d086)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:API Airflow's REST/HTTP API area:webserver Webserver related Issues full tests needed We need to run full set of tests for this PR to merge kind:documentation type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants