Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSRF Warning! State not equal in request and response #376

Closed
nl2412 opened this issue Aug 18, 2021 · 25 comments
Closed

CSRF Warning! State not equal in request and response #376

nl2412 opened this issue Aug 18, 2021 · 25 comments
Assignees
Labels

Comments

@nl2412
Copy link

nl2412 commented Aug 18, 2021

Describe the bug

It happens when using authlib to configure Keycloak for Apache Superset. Everything works perfectly up until redirecting back from Keycloak to Superset.

Error Stacks

Traceback (most recent call last):
  File "/home/abc/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/abc/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/abc/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/abc/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/abc/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/abc/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/abc/.local/lib/python3.8/site-packages/flask_appbuilder/security/views.py", line 681, in oauth_authorized
    resp = self.appbuilder.sm.oauth_remotes[provider].authorize_access_token()
  File "/usr/local/lib/python3.8/site-packages/authlib/integrations/flask_client/remote_app.py", line 74, in authorize_access_token
    params = self.retrieve_access_token_params(flask_req, request_token)
  File "/usr/local/lib/python3.8/site-packages/authlib/integrations/base_client/base_app.py", line 145, in retrieve_access_token_params
    params = self._retrieve_oauth2_access_token_params(request, params)
  File "/usr/local/lib/python3.8/site-packages/authlib/integrations/base_client/base_app.py", line 126, in _retrieve_oauth2_access_token_params
    raise MismatchingStateError()
authlib.integrations.base_client.errors.MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response.

To Reproduce

A minimal example to reproduce the behavior:
This is my code:
In superset_config.py

AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Public'
CSRF_ENABLED = True
ENABLE_PROXY_FIX = True
OAUTH_PROVIDERS = [
    {
        'name': 'keycloak',
        'token_key': 'access_token',
        'icon': 'fa-icon',
        'remote_app': {
            'client_id': CLIENT_ID,
            'client_secret': CLIENT_SECRET,
            'client_kwargs': {
                'scope': 'openid email profile'
            },
            'access_token_method': 'POST',
            'api_base_url': 'https://KEYCLOAK_URL/auth/realms/REALM_NAME/protocol/openid-connect/',
            'access_token_url': 'https://KEYCLOAK_URL/auth/realms/REALM_NAME/protocol/openid-connect/token',
            'authorize_url': 'https://KEYCLOAK_URL/auth/realms/REALM_NAME/protocol/openid-connect/auth',
        },
    }
]
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager

In extended_security.py



class OIDCSecurityManager(SupersetSecurityManager):
    def get_oauth_user_info(self, provider, response=None):
        if provider == 'keycloak':
            me = self.appbuilder.sm.oauth_remotes[provider].get("userinfo")
            return {
                "first_name": me.data.get("given_name", ""),
                "last_name": me.data.get("family_name", ""),
                "email": me.data.get("email", "")
            }

Expected behavior

Suerpset redirect user to keycloak authentication site as expected. Upon finishing authenticating and getting redirected back to superset, CSRF Warning! State not equal in request and response occur.

Environment:
Docker:

  • Python Version: 3.8
  • Authlib Version: 0.15.4

Additional context

Tried on different browser (chrome, firefox, edge, etc.), clearing cookies, etc., but the error is still there.
Would very appreciate some help.

@dnskr
Copy link

dnskr commented Aug 19, 2021

I have the same issue and error stack trace, but use google oidc.

Environment

  • Airflow 2.1.2
  • Python 3.6.13
  • Authlib 0.14.3 0.15.2 0.15.3 0.15.4 1.0.0a1 1.0.0a2

@lepture
Copy link
Owner

lepture commented Sep 19, 2021

@dnskr did you get this error with Authlib 1.0.0a2 ? How to reproduce it? I should have fixed this case in 1.0.0.

@MilanRgm
Copy link

MilanRgm commented Sep 21, 2021

I am also getting oauth error mismatching_state: CSRF Warning! State not equal in request and response.

I used both 0.15.4 and 1.0.0a2 version of Authlib.

if settings.BACKEND_CORS_ORIGINS:
    app.add_middleware(SessionMiddleware, secret_key=settings.SECRET_KEY)
    app.add_middleware(
        CORSMiddleware,
        allow_origins=[str(origin) for origin in settings.BACKEND_CORS_ORIGINS],
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )

config = Config(".env")
oauth = OAuth(config)
oauth.register(
    name="gitlab",
    client_id=settings.GITLAB_CLIENT_ID,
    client_secret=settings.GITLAB_CLIENT_SECRET,
    authorize_url="https://gitlab.com/oauth/authorize",
    client_kwargs={"scope": "read_user+profile"},
)

@app.get("/auth/gitlab")
async def auth_gitlab(request: Request):
    print("###################")
    print("request", request, request.session)
    # {'_gitlab_authlib_redirect_uri_': 'http://localhost:8888/auth/gitlab'}
    gitlab = oauth.create_client("gitlab")
    try:
        token = await gitlab.authorize_access_token(request)
        print("token", token)
        user = await gitlab.parse_id_token(request, token)
        print("user", dict(user))
        return {"token": token}
    except OAuthError as error:
        print("oauth error", error, error.error)

From frontend the following url is hit once authorize button is clicked to reach to /auth/gitlab function as I have used redirect_uri as http://localhost:8888/auth/gitlab

"GET /auth/gitlab?code=asdkjfasdjfkasdjkfasjdkjlsadajk&state=_gitlab HTTP/1.1" 200 OK" (code is changed here)

my oauth url for the popup is

const oauthUrl = `${GITLAB_URL}/oauth/authorize?client_id=${client_id}&response_type=code&scope=${scope}&state=${
          state + '_gitlab'
        }&redirect_uri=${redirect_uri}&allow_signup=${allow_signup}`

UPDATE:
my request.session from /auth/gitlab looks like

{'_state_gitlab_WiBjFgSNd5BV1A7hlDHX0': {'data': {'redirect_uri': 'http://localhost:8888/auth/gitlab', 'url': 'https://gitlab.com/oauth/authorize?response_type=code&client_id=e2dc9edc72dbcf5524910eca1d0577473b6005a833c97&redirect_uri=http%3A%2F%2Flocalhost%3A8888%2Fauth%2Fgitlab&scope=read_user%2Bprofile&state=WiBjFgSNd5BV1A7hlDHX0'}, 'exp': 1632275167.3455658}, '_state_gitlab_3YUfQJ4ubbNjErkqY4dJ7ZQMzzmCqt': {'data': {'redirect_uri': 'http://localhost:8888/auth/gitlab', 'url': 'https://gitlab.com/oauth/authorize?response_type=code&client_id=e2dc9edc72dbcf5524910eca1d0577473b6005a833c97&redirect_uri=http%3A%2F%2Flocalhost%3A8888%2Fauth%2Fgitlab&scope=read_user%2Bprofile&state=3YUfQJ4ubbNjErkqY4dJ7ZQMzzmCqt'}, 'exp': 1632275280.9188702}, '_state_gitlab_S3OQ93EDvralFGYiu5HxRWxUMWZFQh': {'data': {'redirect_uri': 'http://localhost:8888/auth/gitlab', 'url': 'https://gitlab.com/oauth/authorize?response_type=code&client_id=e2dc9edc72dbcf5524910eca1d0577473b6005a833c97&redirect_uri=http%3A%2F%2Flocalhost%3A8888%2Fauth%2Fgitlab&scope=read_user%2Bprofile&state=S3OQ93EDvralFGYiu5HxRWxUMWZFQh'}, 'exp': 1632275404.760191}, '_state_gitlab_vImiUiWK4VIUL82PywWlIZ1K9yA5Ss': {'data': {'redirect_uri': 'http://localhost:8888/auth/gitlab', 'url': 'https://gitlab.com/oauth/authorize?response_type=code&client_id=e2dc9edc72dbcf5524910eca1d0577473b6005a833c97&redirect_uri=http%3A%2F%2Flocalhost%3A8888%2Fauth%2Fgitlab&scope=read_user%2Bprofile&state=vImiUiWK4VIUL82PywWlIZ1K9yA5Ss'}, 'exp': 1632275509.933466}}

When I changed oauth_url for pop up to

const oauthUrl = `${GITLAB_URL}/oauth/authorize?client_id=${client_id}&response_type=code&scope=${scope}&state=${
          state + 'WiBjFgSNd5BV1A7hlDHX0'
        }&redirect_uri=${redirect_uri}&allow_signup=${allow_signup}`

I get python TypeError: Invalid type for url. Expected str or httpx.URL, got <class 'NoneType'>: None

I am using fastapi

@MilanRgm
Copy link

@lepture I tried your fastapi example and it is not working as well. Although, I am using gitlab instead of google. I am getting python TypeError: Invalid type for url. Expected str or httpx.URL, got <class 'NoneType'>: None issue.

@lepture
Copy link
Owner

lepture commented Oct 18, 2021

@MilanRgm

For your case, I see

GET /auth/gitlab?code=asdkjfasdjfkasdjkfasjdkjlsadajk&state=_gitlab HTTP/1.1" 200 OK

The state value is _gitlab. But in request.session it is _state_gitlab_WiBjFgSNd5BV1A7hlDHX0, which means the state is WiBjFgSNd5BV1A7hlDHX0. It is certainly CSRF not matching.

And for:

TypeError: Invalid type for url. Expected str or httpx.URL, got <class 'NoneType'>: None

This is caused by not specifying access_token_url.

@jerome-poisson
Copy link

jerome-poisson commented Nov 16, 2021

Hello,

we have the same issue while calling authorize_access_token() for Google, and it was working fine a while ago.

* Python 3.6.15
* Authlib 0.15.5

edit: I've found the issue in our case: the regression was introduced in our own code (session was reset between requests). You can ignore this comment, authlib is still working as expected for us.

@nikhil003
Copy link

@jerome-poisson , I am encountering an issue similar to yours. I noticed that the session is being reset between requests which leads to this mismatch error. Do you know how I can sort out the session resetting?

In my case, the second request is the call back which has different data in the flask session, so I am not sure where authlib is updating the session values.

@jerome-poisson
Copy link

@jerome-poisson , I am encountering an issue similar to yours. I noticed that the session is being reset between requests which leads to this mismatch error. Do you know how I can sort out the session resetting?

In my case, the second request is the call back which has different data in the flask session, so I am not sure where authlib is updating the session values.

hi @nikhil003 the issue was on our side with cookie management, and at the end it was not a problem with authlib as I was initially suspecting. Thus there is little I can do here, sorry.

@kaktusss123
Copy link

Encountering the same issue. What's the solution??

@iQiexie
Copy link

iQiexie commented Jan 27, 2022

same

@rushilsrivastava
Copy link
Contributor

Previously, I would get a state issue from #419 on Starlette. This seems to be the problem I am running into now.

@rushilsrivastava
Copy link
Contributor

rushilsrivastava commented Feb 15, 2022

After some further investigation: The easiest way for me to consistently reproduce this is to use Google OIDC, and this happens after the 5th continuous re-authentication.

It seems as if during these cases, state_data is None

state_data = await self.framework.get_state_data(session, params.get('state'))

Which led me to check the session middleware in Starlette. My suspicion was that there were lots of used states in the session Cookie, and when authlib tried to set the new state in the session variable, it exceeded the maximum length of a cookie (4096 Bytes). Sure enough, when I printed the session, I could see all my previous states and the length was almost at the maximum. This would explain why state_data would be None when validating.

I can see that there is functionality for adding your own cache mechanism to the StarletteIntegration (also see #425), but I am not sure how you can configure that at the moment. Maybe @lepture can provide some insight?

I'm not too familiar with the flow, but it would seem intuitive to me to clear the state from the session once it has been authenticated via authorize_access_token, what are your thoughts on this @lepture? In the meantime, I suppose one could clear the session before sending an authentication request to guarantee a clean session (assuming you aren't using the session for anything else).

For what it is worth, this behavior already happens in the OAuth1 flow:

await self.framework.clear_state_data(request.session, state)

@rushilsrivastava
Copy link
Contributor

I've put up a PR to fix this behavior: #428

@kurian-dm
Copy link

Did the issue get fixed.

@lepture
Copy link
Owner

lepture commented Mar 17, 2022

1.0.0 was released.

@lepture lepture closed this as completed Mar 18, 2022
@sorasful
Copy link

@lepture Hi there! I've tried the 1.0.0 version just now, and it seems the issue persists with Google Oauth2.

oauth = OAuth()
oauth.register(
    name='google',
    server_metadata_url='https://accounts.google.com/.well-known/openid-configuration',
    client_kwargs={
        'scope': 'openid email profile'
    },
    client_id=GOOGLE_OAUTH2_CLIENT_ID,
    client_secret=GOOGLE_OAUTH2_CLIENT_SECRET
)



@app.route('/oauth2-login')
async def oauth2_login(request: Request):
    # absolute url for callback
    # we will define it below
    redirect_uri = request.url_for('oauth2_auth')
    return await oauth.google.authorize_redirect(request, redirect_uri)


@app.route('/oauth2-auth')
async def oauth2_auth(request: Request):
    token = await oauth.google.authorize_access_token(request)
    user = token['userinfo']
    return user

My stacktrace :

Traceback (most recent call last):
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 376, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/middleware/cors.py", line 78, in __call__
    await self.app(scope, receive, send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/middleware/sessions.py", line 75, in __call__
    await self.app(scope, receive, send_wrapper)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/asgi_correlation_id/middleware.py", line 60, in __call__
    await self.app(scope, receive, handle_outgoing_request)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 580, in __call__
    await route.handle(scope, receive, send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 241, in handle
    await self.app(scope, receive, send)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 52, in app
    response = await func(request)
  File "/home/sorasful/dev/my_api_async/my_api/__main__.py", line 168, in oauth2_auth
    token = await oauth.google.authorize_access_token(request)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/authlib/integrations/starlette_client/apps.py", line 74, in authorize_access_token
    params = self._format_state_params(state_data, params)
  File "/home/sorasful/.cache/pypoetry/virtualenvs/my-api-5UH-8tMH-py3.10/lib/python3.10/site-packages/authlib/integrations/base_client/sync_app.py", line 234, in _format_state_params
    raise MismatchingStateError()
authlib.integrations.base_client.errors.MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response.

@rushilsrivastava
Copy link
Contributor

@sorasful try clearing the session cookie and try again. If you use the session cookie for other things, that may also be causing problems.

@oribs1
Copy link

oribs1 commented Jun 22, 2022

The issue is not exactly fixed. As said before the issue happens when the session cookie is larger than 4096 bytes.
It can happen for multiple reasons, not all in the control of Authlib, but the library should do its best to minimize the size of the cookie.

The major culprit at the moment is the fact the URI to fetch the authorization code is inserted into the session cookie for no reason, I think removing it will help to reduce the number of people facing this issue

@lepture
Copy link
Owner

lepture commented Jun 30, 2022

@oribs1 the saved URI is callback/redirect. Also, you can use cache instead of session cookie.

@rmblau
Copy link

rmblau commented Jul 2, 2022

I also experience this issue with the latest version of Authlib and Google auth. This happens when I am testing on my phone and I go from mobile to desktop view as well as occasionally with my laptop, depending on how many times I re-auth.

@lorenmh
Copy link

lorenmh commented Aug 22, 2022

Some info here, I solved it for myself, leaving this here in case anyone needs it.

The issue seems to originate because the "state" the auth provider provides mismatches the "state" that authlib tries to retrieve from the session.

ie, authlib generates a "state" string, then it stores it in the Flask session:
session["_state_foo_bar"] = "some_state_string"

It puts the state string into a query string it passes to the auth provider (ie, to Google or whatever):
https://accounts.google.com/o/oauth2/v2/auth?some_params_here&state=some_state_string

After auth, the auth provider redirects the user back to your authentication endpoint (preferably with the state param):
https://example.com/auth?some_params_here&state=some_state_string

Then authlib checks the session for the state string:

# roughly the following, pseudo-code
state = request.args.get("state")
if state != session.get("_state_foo_bar"):
    raise MismatchingStateError("oopsies")

So, to fix this, there are a couple of possible areas this could break. I'm a bit rusty at how Flask works in entirety, but some avenues to look into:

  • Are your sessions set up correctly? If a user navigates away from your site and back, do they have the same session?
  • Is the auth provider actually including the state? Is the auth provider including a state=some_state_string query param when it's forwarded to your auth endpoint. Are you able to get the state query param from the provided url? ie, it's possible you could accidentally try to encode query parameters twice, ie, by providing the callback url as https://example.com/auth?my_special_param=foo maybe authlib would turn this into https://example.com/auth?my_special_param=foo?state=some_state_string (notice that there are two ? in there, which would likely break query param parsing)
  • Is your session able to adequately store data? (ie, maybe SECRET_KEY must be set if you are storing session data in cookies). I.e., even if your session persists between requests, are you able to store and retrieve data from the session?

For me personally, I had to set SESSION_COOKIE_SAMESITE='Lax' in my flask config because when the user was sent back to my auth callback endpoint (https://example.com/auth) the user was getting a completely new session, so when authlib checked the session for the state, it did not exist (because it was a new session).

@ahipp13
Copy link

ahipp13 commented Nov 29, 2022

This issue still persists using Azure OAuth 2.0 on Airflow 2.4.3 and these tool versions:

Python 3.8.15
Authlib 1.1.0
Flask 2.2.2
Flask-AppBuilder 4.1.4
Flask-Babel 2.0.0
Flask-Caching 2.0.1
Flask-JWT-Extended 4.4.4
Flask-Login 0.6.2
Flask-Session 0.4.0
Flask-SQLAlchemy 2.5.1
Flask-WTF 1.0.1

The only thing we get in our logs is:
airflow-web [2022-11-29 14:25:10,709] {views.py:659} ERROR - Error authorizing OAuth access token: mismatching_state: CSRF Warning! State not equal in request and response. ││ airflow-web [2022-11-29 14:25:10,709] {views.py:659} ERROR - Error authorizing OAuth access token: mismatching_state: CSRF Warning! State not equal in request and response.

Have used the network debug tool in Chrome to look at the url's being sent and it looks like the same state is being sent in the request and the response, but it just brings us back to the login page every time and throws this error.

@NotoriousPyro
Copy link

Encountered this issue. Our auth flow is as follows:
1: Client web application makes request to an auth server running elsewhere.
2: Auth server sends request to Okta, user logs in and is sent back to Auth server.
3: Auth server sends user back to client web application.

After hitting step 2, when the client is sent back to the Auth server, we were seeing 400: Bad request.

Viewing the logs for the service was revealing the error "CSRF Warning..."

To fix it, when I saw the 400: Bad Request, I cleared the cookies in Chrome for that URL. I also cleared the client web application's cookies.

Then the authorisation was allowed to proceed without any CSRF errors.

It seems something cookie is not cleared even when sending new authorisation requests.

@rsirres
Copy link

rsirres commented Mar 28, 2023

I encountered the CSRF mismatch error when an user is using the back button after logging in.

In the following example client the user is redirected to the /auth endpoint:
https://github.com/authlib/demo-oauth-client/blob/master/flask-google-login/app.py#L35

However, authlib is clearing the CSRF token right after authentication process:
https://github.com/lepture/authlib/blob/master/authlib/integrations/flask_client/apps.py#L99

As a result you will receive a CSRF mismatch error. To prevent this in my application, it was necessary to check whether a session exists before creating a new one;

@app.route('/auth')
def auth():
    if "user" in session:
        token = oauth.google.authorize_access_token()
        session['user'] = token['userinfo']
    return redirect('/')

@sm-Fifteen
Copy link

sm-Fifteen commented Mar 30, 2023

Like I've mentionned in #334 and encode/starlette#2019, a common cause for this bug (for me) comes from a Chrome + Starlette issue where Starlette handles session storage as more or less a JWT and will include a new Set-Cookie header for every response it returns if a session is ongoing, while Chrome acknowledges Set-Cookie in fetch() requests, including if the response arrives after the page has unloaded.

The issue is reproduced like so:

  • User is on some page with a session cookie already set.
  • The page sends a fetch request.
  • User clicks an OAuth button that causes a redirect to an external login page, altering the session cookie with callback data.
  • The fetch request returns, updating the session cookie so that it no longer contains any callback data.
  • User completes the login process, which sends them back to the authorize_url.
  • The authorize_url route fails because the session cookie contains none of the initial callback data.

There seems to be several other ways in which this error message can be triggered, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests