Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add connectivity sample for Python #201

Merged
merged 22 commits into from
Dec 13, 2022
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/workflows/sample-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ jobs:
# Set job outputs to values from filter step
outputs:
java: ${{ steps.filter.outputs.java }}
python: ${{ steps.filter.outputs.python }}
steps:
- name: Remove PR label
if: "${{ github.event.action == 'labeled' && github.event.label.name == 'tests: run' }}"
Expand All @@ -59,6 +60,8 @@ jobs:
filters: |
java:
- 'examples/java/**'
python:
- 'examples/python/**'
java:
# This ensures that the java job execute after the changes job, since it's dependent on
# that job's output.
Expand Down Expand Up @@ -111,3 +114,52 @@ jobs:
run: |
./examples/java/run_tests.sh

python:
# This ensures that the python job executes after the changes job, since it's dependent on
# that job's output.
needs: changes
if: ${{ needs.changes.outputs.python == 'true' || github.event_name == 'schedule' }}
runs-on: [self-hosted, linux, x64]
permissions:
contents: 'read'
id-token: 'write'
steps:
- name: Checkout code
uses: 'actions/checkout@v3'
with:
ref: ${{ github.event.pull_request.head.sha }}
repository: ${{ github.event.pull_request.head.repo.full_name }}

- name: 'Authenticate to Google Cloud'
id: 'auth'
uses: 'google-github-actions/auth@v1.0.0'
with:
workload_identity_provider: ${{ secrets.PROVIDER_NAME }}
service_account: ${{ secrets.SERVICE_ACCOUNT }}

- name: 'Set up Cloud SDK'
uses: 'google-github-actions/setup-gcloud@v1.0.1'
- name: Get Secrets
id: 'secrets'
uses: 'google-github-actions/get-secretmanager-secrets@v0'
with:
secrets: |-
ALLOYDB_CONN_NAME:alloydb-connector-testing/ALLOYDB_CONN_NAME
ALLOYDB_CLUSTER_PASS:alloydb-connector-testing/ALLOYDB_CLUSTER_PASS

- name: Run lint
run: |
pip install --upgrade pip
pip install flake8
cd examples/python
python -m flake8 .

- name: Run tests
env:
DB_NAME: 'postgres'
DB_USER: 'postgres'
DB_PASS: '${{ steps.secrets.outputs.ALLOYDB_CLUSTER_PASS }}'
ALLOYDB_CONNECTION_NAME: '${{ steps.secrets.outputs.ALLOYDB_CONN_NAME }}'
run: |
./examples/python/run_tests.sh

2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,5 @@ out/
.gradle/
.vscode/
.DS_Store
__pycache__
.pytest_cache
12 changes: 5 additions & 7 deletions examples/java/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,13 @@
1. Create an AlloyDB cluster and its primary instance by following these [instructions](https://cloud.google.com/alloydb/docs/cluster-create). Make note of the Cluster ID, Instance ID, IP Address and Password

1. Create a database for your application by following these
[instructions](https://cloud.google.com/sql/docs/postgres/create-manage-databases). Note the database
[instructions](https://cloud.google.com/alloydb/docs/database-create). Note the database
name.

1. Create a user in your database by following these
[instructions](https://cloud.google.com/alloydb/docs/database-users/about). Note the username.

1. Create a service account with the 'AlloyDB Client' permissions by following these
[instructions](https://cloud.google.com/sql/docs/postgres/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account).
Download a JSON key to use to authenticate your connection.
1. Create a [service account](https://cloud.google.com/iam/docs/understanding-service-accounts) with the 'AlloyDB Client' permissions.

1. Use the information noted in the previous steps:
```bash
Expand All @@ -26,7 +24,7 @@ export DB_USER='my-db-user'
export DB_PASS='my-db-pass'
export DB_NAME='my_db'
export DB_HOST='<IP Address of Cluster or 127.0.0.1 if using auth proxy>'
export DB_POST=5432
export DB_PORT=5432
export ALLOYDB_CONNECTION_NAME='projects/<PROJECT>/locations/<REGION>/clusters/<CLUSTER>/instances/<INSTANCE>'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
Expand Down Expand Up @@ -67,7 +65,7 @@ mvn clean package com.google.cloud.tools:jib-maven-plugin:2.8.0:build \

```sh
gcloud run deploy run-postgres \
--image gcr.io/[YOUR_PROJECT_ID]/run-postgres \
--image gcr.io/[YOUR_PROJECT_ID]/run-alloydb \
--platform managed \
--allow-unauthenticated \
--region [REGION] \
Expand All @@ -82,7 +80,7 @@ mvn clean package com.google.cloud.tools:jib-maven-plugin:2.8.0:build \
Take note of the URL output at the end of the deployment process.

It is recommended to use the [Secret Manager integration](https://cloud.google.com/run/docs/configuring/secrets) for Cloud Run instead
of using environment variables for the SQL configuration. The service injects the Alloy credentials from
of using environment variables for the AlloyDB configuration. The service injects the Alloy credentials from
Secret Manager at runtime via an environment variable.

Create secrets via the command line:
Expand Down
4 changes: 4 additions & 0 deletions examples/python/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Dockerfile
.dockerignore
__pycache__
.pytest_cache
38 changes: 38 additions & 0 deletions examples/python/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Copyright 2022 Google, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Use the official Python image.
# https://hub.docker.com/_/python
FROM python:3

# Copy application dependency manifests to the container image.
# Copying this separately prevents re-running pip install on every code change.
COPY requirements.txt ./

# Install production dependencies.
RUN set -ex; \
pip install -r requirements.txt; \
pip install gunicorn

# Copy local code to the container image.
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./


# Run the web service on container startup. Here we use the gunicorn
# webserver, with one worker process and 8 threads.
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 app:app
168 changes: 168 additions & 0 deletions examples/python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Connecting to AlloyDB

## Before you begin

shubha-rajan marked this conversation as resolved.
Show resolved Hide resolved
1. Enable access to AlloyDB in your project by following these [instructions](https://cloud.google.com/alloydb/docs/project-enable-access)

1. Create a VPC network and [configure Private Services Access for AlloyDB](https://cloud.google.com/alloydb/docs/configure-connectivity)

1. Create an AlloyDB cluster and its primary instance by following these [instructions](https://cloud.google.com/alloydb/docs/cluster-create). Make note of the Cluster ID, Instance ID, IP Address and Password

1. Create a database for your application by following these
[instructions](https://cloud.google.com/alloydb/docs/database-create). Note the database
name.

1. Create a user in your database by following these
[instructions](https://cloud.google.com/alloydb/docs/database-users/about). Note the username.

1. Create a [service account](https://cloud.google.com/iam/docs/understanding-service-accounts) with the 'AlloyDB Client' permissions.


1. Use the information noted in the previous steps:
jackwotherspoon marked this conversation as resolved.
Show resolved Hide resolved
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
export DB_USER='<YOUR_DB_USER_NAME>'
export DB_PASS='<YOUR_DB_PASSWORD>'
export DB_NAME='<YOUR_DB_NAME>'
export DB_HOST='<IP Address of Cluster or 127.0.0.1 if using auth proxy>'
export DB_POST=5432
export ALLOYDB_CONNECTION_NAME='projects/<PROJECT>/locations/<REGION>/clusters/<CLUSTER>/instances/<INSTANCE>'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/) to help keep secrets safe.


## Deploy to App Engine Standard

To run on GAE-Standard, create an App Engine project by following the setup for these
[instructions](https://cloud.google.com/appengine/docs/standard/python3/quickstart#before-you-begin).

First, update `app.standard.yaml` with the correct values to pass the environment
variables into the runtime. Your `app.standard.yaml` file should look like this:

```yaml
runtime: python37
entrypoint: gunicorn -b :$PORT app:app

env_variables:
DB_HOST: '<IP Address of Cluster>'
DB_PORT: 5432
DB_USER: <YOUR_DB_USER_NAME>
DB_PASS: <YOUR_DB_PASSWORD>
DB_NAME: <YOUR_DB_NAME>
```

Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Next, the following command will deploy the application to your Google Cloud project:

```bash
gcloud app deploy app.standard.yaml
```

## Deploy to App Engine Flexible

To run on GAE-Flexible, create an App Engine project by following the setup for these
[instructions](https://cloud.google.com/appengine/docs/flexible/python/quickstart#before-you-begin).

First, update `app.flexible.yaml` with the correct values to pass the environment
variables into the runtime. Your `app.flexible.yaml` file should look like this:

```yaml
runtime: custom
env: flex
entrypoint: gunicorn -b :$PORT app:app

env_variables:
DB_HOST: '<IP Address of Cluster>'
DB_PORT: 5432
DB_USER: <YOUR_DB_USER_NAME>
DB_PASS: <YOUR_DB_PASSWORD>
DB_NAME: <YOUR_DB_NAME>

```

Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Next, the following command will deploy the application to your Google Cloud project:

```bash
gcloud app deploy app.flexible.yaml
```

## Deploy to Cloud Run

Before deploying the application, you will need to [configure a Serverless VPC Connector](https://cloud.google.com/vpc/docs/configure-serverless-vpc-access) to be able to connect to the VPC in which your AlloyDB cluster is running.
1. Build the container image:

```sh
gcloud builds submit --tag gcr.io/[YOUR_PROJECT_ID]/run-alloydb
```

2. Deploy the service to Cloud Run:

```sh
gcloud run deploy run-alloydb \
--image gcr.io/[YOUR_PROJECT_ID]/run-alloydb \
--platform managed \
--allow-unauthenticated \
--region [REGION] \
--update-env-vars DB_HOST=[DB_HOST] \
--update-env-vars DB_PORT=[DB_PORT] \
--update-env-vars DB_USER=[MY_DB_USER] \
--update-env-vars DB_PASS=[MY_DB_PASS] \
--update-env-vars DB_NAME=[MY_DB]
```

Take note of the URL output at the end of the deployment process.

Replace environment variables with the correct values for your AlloyDB
instance configuration.

It is recommended to use the [Secret Manager integration](https://cloud.google.com/run/docs/configuring/secrets) for Cloud Run instead
of using environment variables for the AlloyDB configuration. The service injects the AlloyDB credentials from
Secret Manager at runtime via an environment variable.

Create secrets via the command line:
```sh
echo -n "projects/<PROJECT>/locations/<REGION>/clusters/<CLUSTER>/instances/<INSTANCE>" | \
gcloud secrets versions add ALLOYDB_CONNECTION_NAME_SECRET --data-file=-
```

Deploy the service to Cloud Run specifying the env var name and secret name:
```sh
gcloud beta run deploy SERVICE --image gcr.io/[YOUR_PROJECT_ID]/run-alloydb \
--update-secrets --update-secrets DB_HOST=[DB_HOST_SECRET]:latest,\
DB_PORT=[DB_PORT_SECRET]:latest, \
DB_USER=[DB_USER_SECRET]:latest, \
DB_PASS=[DB_PASS_SECRET]:latest, \
DB_NAME=[DB_NAME_SECRET]:latest
```

3. Navigate your browser to the URL noted in step 2.

For more details about using Cloud Run see http://cloud.run.
Review other [Python on Cloud Run samples](../../../run/).

## Deploy to Cloud Functions

To deploy the service to [Cloud Functions](https://cloud.google.com/functions/docs) run the following command:

```sh
gcloud functions deploy votes --runtime python39 --trigger-http --allow-unauthenticated \
--set-env-vars DB_HOST=$DB_HOST \
--set-env-vars DB_PORT=$DB_PORT \
--set-env-vars DB_USER=$DB_USER \
--set-env-vars DB_PASS=$DB_PASS \
--set-env-vars DB_NAME=$DB_NAME
```

Take note of the URL output at the end of the deployment process or run the following to view your function:

```sh
gcloud app browse
```
28 changes: 28 additions & 0 deletions examples/python/app.flexible.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

runtime: custom
env: flex
entrypoint: gunicorn -b :$PORT app:app

# Note: Saving credentials in environment variables is convenient, but not
# secure - consider a more secure solution such as
# Cloud Secret Manager (https://cloud.google.com/secret-manager) to help
# keep secrets safe.
env_variables:
DB_HOST: <YOUR_DB_HOST_IP>
DB_PORT: <YOUR_DB_PORT>
DB_USER: <YOUR_DB_USER_NAME>
DB_PASS: <YOUR_DB_PASSWORD>
DB_NAME: <YOUR_DB_NAME>