Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add startup/shutdown dependencies and dependency caching lifespan control #3516

Closed
wants to merge 37 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
cf7462d
Add support for app lifetime dependencies
adriangb Jul 11, 2021
2590de6
Test overrides
adriangb Jul 11, 2021
ed4b9d6
Add test for generator consumption
adriangb Jul 11, 2021
e2abb83
Update docs/en/docs/tutorial/dependencies/lifetime-dependencies.md
adriangb Jul 12, 2021
6bfbfa6
Update docs/en/docs/tutorial/dependencies/lifetime-dependencies.md
adriangb Jul 12, 2021
480062c
Update docs/en/docs/tutorial/dependencies/lifetime-dependencies.md
adriangb Jul 12, 2021
d8b6d14
Add type aliases & rename lifetime -> lifespan
adriangb Jul 12, 2021
fdea40f
fix typo
adriangb Jul 12, 2021
ea5f247
rename DataBase -> Database
adriangb Jul 12, 2021
cecd77f
Working lifespan management
adriangb Jul 12, 2021
c4cd269
Add renamed file
adriangb Jul 12, 2021
629e465
Use Starlette(lifetime=...)
adriangb Jul 19, 2021
675110f
Update docs_src/dependencies/tutorial013.py
adriangb Jul 19, 2021
5964bf0
Fix suggestion
adriangb Jul 19, 2021
78b1276
Support dependencies in startup/shutdown events
adriangb Jul 21, 2021
03c38db
Add files
adriangb Jul 21, 2021
16b0fa8
Merge branch 'master' into lifetime-dependencies
adriangb Jul 21, 2021
2d67655
Update examples
adriangb Jul 21, 2021
029e96b
wip
adriangb Jul 22, 2021
ef9caed
add test file
adriangb Jul 22, 2021
ab26a2f
Use enums
adriangb Jul 22, 2021
243180f
Working implementation for use_cache={app,request,False} and lifetime…
adriangb Jul 23, 2021
c8f492b
Add handling of startup/shutdown dependencies that depend on connecti…
adriangb Jul 23, 2021
7d6a11f
remove stray file
adriangb Jul 23, 2021
220d716
Add test
adriangb Jul 23, 2021
1e17804
Docs files
adriangb Jul 23, 2021
fe709c4
Add docs to index
adriangb Jul 23, 2021
c721807
fix linting
adriangb Jul 23, 2021
be55014
Update db.py
adriangb Jul 23, 2021
b6670dd
Update main.py
adriangb Jul 23, 2021
d530491
Add test for nested app cached deps
adriangb Jul 23, 2021
5a2c9d8
Add Dependency Cache Scopes docs
adriangb Jul 23, 2021
c165bf5
basic dep lifetime docs
adriangb Jul 23, 2021
45ba5ea
formatting
adriangb Jul 23, 2021
234dea2
Rework docs & examples
adriangb Jul 23, 2021
f1d561e
fix link
adriangb Jul 23, 2021
5641c0d
re-add existig docs
adriangb Jul 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
96 changes: 71 additions & 25 deletions docs/en/docs/advanced/async-sql-databases.md
Expand Up @@ -17,44 +17,56 @@ Later, for your production application, you might want to use a database server

This section doesn't apply those ideas, to be equivalent to the counterpart in <a href="https://www.starlette.io/database/" class="external-link" target="_blank">Starlette</a>.

## Import and set up `SQLAlchemy`
## Create a configuration for our database

* Import `SQLAlchemy`.
* Create a `metadata` object.
* Create a table `notes` using the `metadata` object.
We start by creating [Pydantic Settings](../../advanced/settings.md#pydantic-settings){.internal-link target=_blank} for our app:

```Python hl_lines="4 14 16-22"
```Python hl_lines="6 9-10 13-14"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

!!! tip
Notice that all this code is pure SQLAlchemy Core.
You'll see that in our demo and tests we use **SQLite**, because it uses a single file and Python has integrated support.
For your production application, you might want to use a database server like **PostgreSQL**.

`databases` is not doing anything here yet.
!!! tip
Since we are loading our config using dependency injection, we can use [Dependency overrides](testing-dependencies.md){.internal-link target=_blank} in our tests to set the database URL, without touching enviroment variables!

## Import and set up `databases`
## Import and set up `SQLAlchemy`

* Import `databases`.
* Create a `DATABASE_URL`.
* Create a `database` object.
* Import `SQLAlchemy`.
* Create a `metadata` object.
* Create a table `notes` using the `metadata` object.

```Python hl_lines="3 9 12"
```Python hl_lines="4 17 19-25"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

!!! tip
If you were connecting to a different database (e.g. PostgreSQL), you would need to change the `DATABASE_URL`.

## Create the tables

In this case, we are creating the tables in the same Python file, but in production, you would probably want to create them with Alembic, integrated with migrations, etc.

Here, this section would run directly, right before starting your **FastAPI** application.
Here, this section will run directly, right before starting your **FastAPI** application.

* Create a **FastAPI** dependency that depends on our app's settings.
* Create an `engine`.
* Create all the tables from the `metadata` object.

```Python hl_lines="25-28"
```Python hl_lines="28 29-31 32"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

!!! tip
Notice that all this code is pure SQLAlchemy Core.

`databases` is not doing anything here yet.

## Import and set up `databases`

* Import `databases`.
* Create a `database` dependency called `get_db` that depends on our `setup_schema` dependency.

```Python hl_lines="3 35-39"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

Expand All @@ -65,28 +77,28 @@ Create Pydantic models for:
* Notes to be created (`NoteIn`).
* Notes to be returned (`Note`).

```Python hl_lines="31-33 36-39"
```Python hl_lines="42-44 47-50"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

By creating these Pydantic models, the input data will be validated, serialized (converted), and annotated (documented).

So, you will be able to see it all in the interactive API docs.

## Connect and disconnect
## Create a startup handler

* Create your `FastAPI` application.
* Create event handlers to connect and disconnect from the database.
* Create a startup event handler that connects to the database, applies the migrations and checks the connection.
* Create your `FastAPI` application and bind the startup event handler.

```Python hl_lines="42 45-47 50-52"
```Python hl_lines="53-54 57"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

## Read notes

Create the *path operation function* to read notes:

```Python hl_lines="55-58"
```Python hl_lines="60-63"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

Expand All @@ -103,7 +115,7 @@ That documents (and validates, serializes, filters) the output data, as a `list`

Create the *path operation function* to create notes:

```Python hl_lines="61-65"
```Python hl_lines="66-70"
{!../../../docs_src/async_sql_databases/tutorial001.py!}
```

Expand Down Expand Up @@ -151,12 +163,46 @@ So, the final result returned would be something like:

## Check it

You can copy this code as is, and see the docs at <a href="http://127.0.0.1:8000/docs" class="external-link" target="_blank">http://127.0.0.1:8000/docs</a>.
To run this code, set an enviroment variable named `DB_URL` to point to your database.
If you are running PostgreSQL locally, you might do: `export DB_URL=postgresql://user:password@postgresserver/db`.
To test with SQLite, you can set `export DB_URL=sqlite:///./test.db`.

For more information on enviroment variables, see [Environment Variables](settings.md#environment-variables){.internal-link target=_blank}.

Then you can copy this code as is, run it using Uvicorn and see the docs at <a href="http://127.0.0.1:8000/docs" class="external-link" target="_blank">http://127.0.0.1:8000/docs</a>.

There you can see all your API documented and interact with it:

<img src="/img/tutorial/async-sql-databases/image01.png">

## Testing

To test this app, we'll first import the app and it's dependencies into a test file:

```Python hl_lines="6"
{!../../../docs_src/async_sql_databases/test_tutorial_001.py!}
```

Then create a dependency override for our config that points to a local SQLite database:

```Python hl_lines="2 9-10"
{!../../../docs_src/async_sql_databases/test_tutorial_001.py!}
```

!!! tip
We create a new database for every test run by using `uuid4()` instead of a fixed name for our test database.
This saves us from worrying about side effects between tests.

Finally, in our tests we use `app.dependency_overrides` to inject our test config:

```Python hl_lines="1 14"
{!../../../docs_src/async_sql_databases/test_tutorial_001.py!}
```

!!! tip
We use `unittest.mock.patch` to *patch* the dictionary instead of assigning to it directly.
This saves us the trouble of having to clean up our dependency override, even if our test fails, which otherwise might have created a confusing cascade of failing tests.

## More info

You can read more about <a href="https://github.com/encode/databases" class="external-link" target="_blank">`encode/databases` at its GitHub page</a>.
85 changes: 6 additions & 79 deletions docs/en/docs/advanced/settings.md
Expand Up @@ -220,11 +220,6 @@ Now we create a dependency that returns a new `config.Settings()`.
{!../../../docs_src/settings/app02/main.py!}
```

!!! tip
We'll discuss the `@lru_cache()` in a bit.

For now you can assume `get_settings()` is a normal function.

And then we can require it from the *path operation function* as a dependency and use it anywhere we need it.

```Python hl_lines="16 18-20"
Expand Down Expand Up @@ -281,7 +276,7 @@ Here we create a class `Config` inside of your Pydantic `Settings` class, and se
!!! tip
The `Config` class is used just for Pydantic configuration. You can read more at <a href="https://pydantic-docs.helpmanual.io/usage/model_config/" class="external-link" target="_blank">Pydantic Model Config</a>

### Creating the `Settings` only once with `lru_cache`
### Creating the `Settings` only once with `use_cache="app"`

Reading a file from disk is normally a costly (slow) operation, so you probably want to do it only once and then re-use the same settings object, instead of reading it for each request.

Expand All @@ -293,90 +288,22 @@ config.Settings()

a new `Settings` object would be created, and at creation it would read the `.env` file again.

If the dependency function was just like:
If we declared our dependency without `Depends(..., use_cache="app")`, we would create that object for each request, and we would be reading the `.env` file for each request. ⚠️

```Python
def get_settings():
return config.Settings()
```

we would create that object for each request, and we would be reading the `.env` file for each request. ⚠️

But as we are using the `@lru_cache()` decorator on top, the `Settings` object will be created only once, the first time it's called. ✔️
But by setting `use_cache="app"`, the `Settings` object will be created only once, the first time it's called. ✔️

```Python hl_lines="1 10"
{!../../../docs_src/settings/app03/main.py!}
```

Then for any subsequent calls of `get_settings()` in the dependencies for the next requests, instead of executing the internal code of `get_settings()` and creating a new `Settings` object, it will return the same object that was returned on the first call, again and again.

#### `lru_cache` Technical Details

`@lru_cache()` modifies the function it decorates to return the same value that was returned the first time, instead of computing it again, executing the code of the function every time.

So, the function below it will be executed once for each combination of arguments. And then the values returned by each of those combinations of arguments will be used again and again whenever the function is called with exactly the same combination of arguments.

For example, if you have a function:

```Python
@lru_cache()
def say_hi(name: str, salutation: str = "Ms."):
return f"Hello {salutation} {name}"
```

your program could execute like this:

```mermaid
sequenceDiagram

participant code as Code
participant function as say_hi()
participant execute as Execute function

rect rgba(0, 255, 0, .1)
code ->> function: say_hi(name="Camila")
function ->> execute: execute function code
execute ->> code: return the result
end

rect rgba(0, 255, 255, .1)
code ->> function: say_hi(name="Camila")
function ->> code: return stored result
end

rect rgba(0, 255, 0, .1)
code ->> function: say_hi(name="Rick")
function ->> execute: execute function code
execute ->> code: return the result
end

rect rgba(0, 255, 0, .1)
code ->> function: say_hi(name="Rick", salutation="Mr.")
function ->> execute: execute function code
execute ->> code: return the result
end

rect rgba(0, 255, 255, .1)
code ->> function: say_hi(name="Rick")
function ->> code: return stored result
end

rect rgba(0, 255, 255, .1)
code ->> function: say_hi(name="Camila")
function ->> code: return stored result
end
```

In the case of our dependency `get_settings()`, the function doesn't even take any arguments, so it always returns the same value.

That way, it behaves almost as if it was just a global variable. But as it uses a dependency function, then we can override it easily for testing.
Then for any subsequent requests to `/info`, instead of executing the internal code of `get_settings()` and creating a new `Settings` object, the same object will be returned again and again.

`@lru_cache()` is part of `functools` which is part of Python's standard library, you can read more about it in the <a href="https://docs.python.org/3/library/functools.html#functools.lru_cache" class="external-link" target="_blank">Python docs for `@lru_cache()`</a>.
For more details on dependency caching scopes, see [Dependency Cache Scopes](../tutorial/dependencies/dependency-cache-scopes.md){.internal-link target=_blank}

## Recap

You can use Pydantic Settings to handle the settings or configurations for your application, with all the power of Pydantic models.

* By using a dependency you can simplify testing.
* You can use `.env` files with it.
* Using `@lru_cache()` lets you avoid reading the dotenv file again and again for each request, while allowing you to override it during testing.
* Using `use_cache="app"` lets you avoid reading the dotenv file again and again for each request, while allowing you to override it during testing.