Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add windows compatibility for tests #454

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
.vscode/
.idea/
*.pyc
test.db
.coverage
Expand Down
83 changes: 83 additions & 0 deletions docs/contributing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Contibuting

All contributions to *databases* are welcomed!

## Issues

To make it as simple as possible for us to help you, please include the following:

* OS
* python version
* databases version
* database backend (mysql, sqlite or postgresql)
* database driver (aiopg, aiomysql etc.)

Please try to always include the above unless you're unable to install *databases* or **know** it's not relevant
to your question or feature request.

## Pull Requests

It should be quite straight forward to get started and create a Pull Request.

!!! note
Unless your change is trivial (typo, docs tweak etc.), please create an issue to discuss the change before
creating a pull request.

To make contributing as easy and fast as possible, you'll want to run tests and linting locally.

You'll need to have **python >= 3.6 (recommended 3.7+)** and **git** installed.

## Getting started

1. Clone your fork and cd into the repo directory
```bash
git clone git@github.com:<your username>/databases.git
cd databases
```

2. Create and activate virtual env
```bash
virtualenv -p `which python3.6` env
source env/bin/activate
```

3. Install databases, dependencies and test dependencies
```bash
pip install -r requirements.txt
```

4. Checkout a new branch and make your changes
```bash
git checkout -b my-new-feature-branch
```

## Make your changes...

## Contribute

1. Formatting and linting - databases uses black for formatting, autoflake for linting and mypy for type hints check
run all of those with lint script
```bash
./scripts/lint
```

2. Prepare tests (basic)
1. Set-up `TEST_DATABASE_URLS` env variable where you can comma separate urls for several backends
2. The simples one is for sqlite alone: `sqlite:///test.db`

3. Prepare tests (all backends)
1. In order to run all backends you need a docker instalation on your system [from here](https://docs.docker.com/get-docker/)
2. You need to set-up `TEST_IN_DOCKER` env variable (to any non-null value `YES` or `1`)

4. Run tests
```bash
./scripts/test
```

5. Build documentation
1. If you have changed the documentation make sure it runs successfully
```bash
./scripts/test
```

6. Commit, push, and create your pull request
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ nav:
- Database Queries: 'database_queries.md'
- Connections & Transactions: 'connections_and_transactions.md'
- Tests & Migrations: 'tests_and_migrations.md'
- Contributing: 'contributing.md'

markdown_extensions:
- mkautodoc
Expand Down
16 changes: 15 additions & 1 deletion scripts/check
Original file line number Diff line number Diff line change
@@ -1,9 +1,23 @@
#!/bin/sh -e

export PREFIX=""
if [ -d 'venv' ] ; then
UNAME=$(uname)
case "$UNAME" in
"Linux") export SYSTEM="Linux" ;;
"Darwin") export SYSTEM="Linux" ;;
CYGWIN*) export SYSTEM="Windows" ;;
MINGW*) export SYSTEM="Windows" ;;
*) export SYSTEM="Linux" ;;
esac

if [ -d 'venv' ]; then
if [ $SYSTEM = "Linux" ] ; then
export PREFIX="venv/bin/"
else
export PREFIX="venv/Scripts/"
fi
fi

export SOURCE_FILES="databases tests"

set -x
Expand Down
15 changes: 14 additions & 1 deletion scripts/coverage
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
#!/bin/sh -e

export PREFIX=""
if [ -d 'venv' ] ; then
UNAME=$(uname)
case "$UNAME" in
"Linux") export SYSTEM="Linux" ;;
"Darwin") export SYSTEM="Linux" ;;
CYGWIN*) export SYSTEM="Windows" ;;
MINGW*) export SYSTEM="Windows" ;;
*) export SYSTEM="Linux" ;;
esac

if [ -d 'venv' ]; then
if [ $SYSTEM = "Linux" ] ; then
export PREFIX="venv/bin/"
else
export PREFIX="venv/Scripts/"
fi
fi

set -x
Expand Down
20 changes: 20 additions & 0 deletions scripts/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
version: '2.1'
services:
postgres:
image: postgres:10.8
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: password
POSTGRES_DB: testsuite
ports:
- 5432:5432

mysql:
image: mysql:5.7
environment:
MYSQL_USER: username
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: testsuite
ports:
- 3306:3306
15 changes: 14 additions & 1 deletion scripts/docs
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
#!/bin/sh -e

export PREFIX=""
if [ -d 'venv' ] ; then
UNAME=$(uname)
case "$UNAME" in
"Linux") export SYSTEM="Linux" ;;
"Darwin") export SYSTEM="Linux" ;;
CYGWIN*) export SYSTEM="Windows" ;;
MINGW*) export SYSTEM="Windows" ;;
*) export SYSTEM="Linux" ;;
esac

if [ -d 'venv' ]; then
if [ $SYSTEM = "Linux" ] ; then
export PREFIX="venv/bin/"
else
export PREFIX="venv/Scripts/"
fi
fi

set -x
Expand Down
15 changes: 14 additions & 1 deletion scripts/lint
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
#!/bin/sh -e

export PREFIX=""
if [ -d 'venv' ] ; then
UNAME=$(uname)
case "$UNAME" in
"Linux") export SYSTEM="Linux" ;;
"Darwin") export SYSTEM="Linux" ;;
CYGWIN*) export SYSTEM="Windows" ;;
MINGW*) export SYSTEM="Windows" ;;
*) export SYSTEM="Linux" ;;
esac

if [ -d 'venv' ]; then
if [ $SYSTEM = "Linux" ]; then
export PREFIX="venv/bin/"
else
export PREFIX="venv/Scripts/"
fi
fi

set -x
Expand Down
40 changes: 34 additions & 6 deletions scripts/test
Original file line number Diff line number Diff line change
@@ -1,23 +1,51 @@
#!/bin/sh

export PREFIX=""
if [ -d 'venv' ] ; then
UNAME=$(uname)
case "$UNAME" in
"Linux") export SYSTEM="Linux" ;;
"Darwin") export SYSTEM="Linux" ;;
CYGWIN*) export SYSTEM="Windows" ;;
MINGW*) export SYSTEM="Windows" ;;
*) export SYSTEM="Linux" ;;
esac

if [ -d 'venv' ]; then
if [ $SYSTEM = "Linux" ]; then
export PREFIX="venv/bin/"
else
export PREFIX="venv/Scripts/"
fi
fi

if [ -z "$TEST_DATABASE_URLS" ] && [ -z "${TEST_IN_DOCKER+x}" ]; then
echo "Variable TEST_DATABASE_URLS or TEST_IN_DOCKER must be set."
exit 1
fi

if [ -z "$TEST_DATABASE_URLS" ] ; then
echo "Variable TEST_DATABASE_URLS must be set."
exit 1
if [ -n "${TEST_IN_DOCKER+x}" ]; then
docker-compose -f ./scripts/docker-compose.yml up -d
export TEST_DATABASE_URLS="sqlite:///test.db,
sqlite+aiosqlite:///test.db,
mysql+aiomysql://username:password@localhost:3306/testsuite,
mysql+asyncmy://username:password@localhost:3306/testsuite,
postgresql+aiopg://username:password@127.0.0.1:5432/testsuite,
postgresql+asyncpg://username:password@localhost:5432/testsuite
"
fi

set -ex

if [ -z $GITHUB_ACTIONS ]; then
scripts/check
scripts/check
fi

${PREFIX}coverage run -m pytest $@

if [ -z $GITHUB_ACTIONS ]; then
scripts/coverage
scripts/coverage
fi

if [ -n "${TEST_IN_DOCKER+x}" ]; then
docker-compose -f ./scripts/docker-compose.yml down
fi
7 changes: 6 additions & 1 deletion tests/test_databases.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@

from databases import Database, DatabaseURL

if sys.version_info >= (3, 8) and sys.platform.lower().startswith(
"win"
): # pragma: no cover
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

assert "TEST_DATABASE_URLS" in os.environ, "TEST_DATABASE_URLS is not set."

DATABASE_URLS = [url.strip() for url in os.environ["TEST_DATABASE_URLS"].split(",")]
Expand All @@ -23,7 +28,7 @@ def mysql_versions(wrapped_func):
"""

@functools.wraps(wrapped_func)
def check(*args, **kwargs):
def check(*args, **kwargs): # pragma: no cover
url = DatabaseURL(kwargs["database_url"])
if url.scheme in ["mysql", "mysql+aiomysql"] and sys.version_info >= (3, 10):
pytest.skip("aiomysql supports python 3.9 and lower")
Expand Down