auth for worker log serve app

[vazmeee]

Description

To use Authentication for Flask app: airflow.utils.serve_logs while serving logs for Celery Workers

Airflow UI does not allow users to view task logs if they don't have access to the dag.
But the Flask server is still open and user can send simple request without any authentication method to fetch the log file with the path.

Use case / motivation

To fix this vulnerability via a suitable methodology as log files shouldn't be open to access freely.

Are you willing to submit a PR?

Yes

@mik-laj

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[mik-laj]

How do you propose to secure this server? We previously did not have any additional protection for communication between components, so it is worth considering the best way to do this.

[vazmeee]

Hi.
At the most basic level for communication I think a key in the config can be set so as to allow interactions between systems.
Webserver UI-> Scheduler
Webserver UI-> Worker

ex:
log_serve_secret_key = <alphanumericstring>

I've thought of 2 ways for this. please suggest

1. use the same secret key and encode/fernet encrypt it and store as a cookie for a session. requests include a header. Worker/Scheduler decode/decrypt and send response if matched.
2. Use a login based session cookie itself to send request and worker/scheduler authenticate it by verifying in the storage backend. This would end up using some IO time and would require SQLA engine on the scheduler and worker side as well to verify.

I've noticed systems use method 1 for connecting services, but also have their own individual auth backend builtin each component service by decoupling with the rest. But this is if each is a component service that can be used without the other.

If Airflow Webserver UI is tied with scheduler,worker and we can go with 1 by making the log api use a global key.
Otherwise to provide users these APIs and integrate with their own external usecase, authentication would be required at the Access Control level.

[uranusjr]

Sessions may be suboptimal since it means restarting a service requires re-establishing authentication. Having a fernet key is a more popular solution for microservices, from what I know.

[vazmeee]

or maybe send an encrypted response on a request without any headers but can only be decrypted via fernet key on the client side ?

[uranusjr]

The client still needs to somehow send a key for the server to encode things, so that’s basically the same idea implemented a bit differently.

[mik-laj]

We can also use the itsdangerous library, which is already used to generate tokens in API

airflow/airflow/api_connexion/endpoints/dag_source_endpoint.py

Lines 32 to 34 in db10c68

	auth_s = URLSafeSerializer(secret_key)
	try:
	path = auth_s.loads(file_token)

In our case, we can use TimedSigner/TimedJSONWebSignatureSerializer so that even a key leak does not pose a security risk https://github.com/pallets/itsdangerous/blob/b1e32ca0a77a819dc992e0b9ab9e70929b9224aa/src/itsdangerous/timed.py#L169.
https://github.com/pallets/itsdangerous/blob/b1e32ca0a77a819dc992e0b9ab9e70929b9224aa/src/itsdangerous/jws.py#L186-L196

[mik-laj]

Done: #16754