Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: provide a tool to generate examples from code #618

Merged
merged 49 commits into from Nov 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
0fcd7bd
chore: convert pulsar example in executable code
mdelapenya Nov 2, 2022
9d5fbfe
chore: convert nginx example in executable code
mdelapenya Nov 2, 2022
30171e2
fix: handle error in tests
mdelapenya Nov 2, 2022
51a0690
chore: convert redis example in executable code
mdelapenya Nov 2, 2022
9cd249f
chore: convert cockroachDB example in executable code
mdelapenya Nov 2, 2022
9be5620
chore: simplify including the entire file as example
mdelapenya Nov 2, 2022
1f396e5
chore: bump dependencies in the e2e module
mdelapenya Nov 2, 2022
44434df
chore: add a common Make goal for running Go tests
mdelapenya Nov 2, 2022
2e37ff3
chore: push go mod goals to the common file
mdelapenya Nov 2, 2022
d8bf2e8
chore: add gotestsum as tools to the examples
mdelapenya Nov 2, 2022
7c48ee0
chore: support running the example files with Make
mdelapenya Nov 2, 2022
f6272d4
chore: use new goal in CI pipelines
mdelapenya Nov 2, 2022
5f0fcc2
feat: add a tool to bootstrap an example
mdelapenya Nov 2, 2022
f110af4
chore: extract code generation to a function
mdelapenya Nov 2, 2022
bfa9581
chore: pass the examples dir to the generate function
mdelapenya Nov 2, 2022
2328361
chore: pass the examples docs dir to the generate function
mdelapenya Nov 2, 2022
f32d9bb
fix: prepend examples dir when creating the examples files
mdelapenya Nov 3, 2022
b3ab1b9
chore: extract templates to a constant
mdelapenya Nov 3, 2022
e66df3f
chore: save toolsgo under its own package
mdelapenya Nov 3, 2022
7ab47d5
chore: add unit tests for the code generation tool
mdelapenya Nov 3, 2022
9eba684
chore: run unit tests for the examples
mdelapenya Nov 3, 2022
70e2ed5
chore: simplify how to compare tasks
mdelapenya Nov 3, 2022
d801fde
fix: remove useless assertion
mdelapenya Nov 3, 2022
ed0121e
Merge branch 'main' into examples
mdelapenya Nov 9, 2022
f55c0fb
chore: support setting the Docker image for the example
mdelapenya Nov 9, 2022
6305f98
docs: document the command line flags
mdelapenya Nov 9, 2022
47f2a43
fix: simplify path generation for examples
mdelapenya Nov 9, 2022
a618362
fix: remove unused package in template
mdelapenya Nov 9, 2022
fead327
fix: use a title for the example in functions
mdelapenya Nov 9, 2022
e07d6c6
chore: add comments to functions
mdelapenya Nov 9, 2022
df051de
chore: simplify passing the example to the generator
mdelapenya Nov 9, 2022
e06ab17
Merge branch 'main' into examples-work
mdelapenya Nov 10, 2022
4210a10
chore: standardize how the container is cleaned in tests
mdelapenya Nov 10, 2022
853375f
fix: use a timeout in pulsar example
mdelapenya Nov 10, 2022
aa4529f
Revert "fix: use a timeout in pulsar example"
mdelapenya Nov 10, 2022
f4c4ce9
fix: do not termiante pulsar container, as it's controlled by the Reaper
mdelapenya Nov 10, 2022
bc0ec00
Merge branch 'main' into examples-work
mdelapenya Nov 17, 2022
7f06d72
chore: add tests to verify that the examples are added to the docs
mdelapenya Nov 17, 2022
cce7f35
chore: automatically add examples to mkdocs
mdelapenya Nov 17, 2022
c4add3c
docs: update docs
mdelapenya Nov 18, 2022
58c00fa
Merge branch 'main' into examples-work
mdelapenya Nov 21, 2022
bf5f4f0
Merge branch 'main' into examples-work
mdelapenya Nov 23, 2022
7f975ef
chore: run examples in dedicated pipelines each
mdelapenya Nov 23, 2022
f62f646
chore: add CI pipelines for the examples
mdelapenya Nov 23, 2022
c270aa1
docs: move new tool docs to the website
mdelapenya Nov 23, 2022
f9ee2f8
docs: link to the examples dir on Github
mdelapenya Nov 23, 2022
9489fef
fix: update tests
mdelapenya Nov 23, 2022
0ea14c0
chore: include generator tests in the main pipeline
mdelapenya Nov 23, 2022
53fd6ef
chore: simplify example pipeline removing go build
mdelapenya Nov 23, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/ci.yml
Expand Up @@ -24,7 +24,7 @@ jobs:
run: go mod verify

- name: modTidy
run: go mod tidy
run: make tools-tidy

- name: ensure compilation
env:
Expand All @@ -43,6 +43,10 @@ jobs:
if: ${{ matrix.platform == 'ubuntu-latest' }}
run: make -C e2e tools-tidy

- name: Run example generator tests
if: ${{ matrix.platform == 'ubuntu-latest' }}
run: make -C examples test-unit

- name: Run e2e tests
if: ${{ matrix.platform == 'ubuntu-latest' }}
run: make test-e2e
Expand Down
42 changes: 42 additions & 0 deletions .github/workflows/cockroachdb-example.yml
@@ -0,0 +1,42 @@
name: Cockroachdb example pipeline

on: [push, pull_request]

jobs:
test-cockroachdb:
strategy:
matrix:
go-version: [1.18.x, 1.x]
runs-on: "ubuntu-latest"
steps:

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
working-directory: ./examples/cockroachdb
run: go mod verify

- name: modTidy
working-directory: ./examples/cockroachdb
run: make tools-tidy

- name: gotestsum
working-directory: ./examples/cockroachdb
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-cockroachdb*.xml"
if: always()
42 changes: 42 additions & 0 deletions .github/workflows/nginx-example.yml
@@ -0,0 +1,42 @@
name: Nginx example pipeline

on: [push, pull_request]

jobs:
test-nginx:
strategy:
matrix:
go-version: [1.18.x, 1.x]
runs-on: "ubuntu-latest"
steps:

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
working-directory: ./examples/nginx
run: go mod verify

- name: modTidy
working-directory: ./examples/nginx
run: make tools-tidy

- name: gotestsum
working-directory: ./examples/nginx
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-nginx*.xml"
if: always()
42 changes: 42 additions & 0 deletions .github/workflows/pulsar-example.yml
@@ -0,0 +1,42 @@
name: Pulsar example pipeline

on: [push, pull_request]

jobs:
test-pulsar:
strategy:
matrix:
go-version: [1.18.x, 1.x]
runs-on: "ubuntu-latest"
steps:

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
working-directory: ./examples/pulsar
run: go mod verify

- name: modTidy
working-directory: ./examples/pulsar
run: make tools-tidy

- name: gotestsum
working-directory: ./examples/pulsar
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-pulsar*.xml"
if: always()
42 changes: 42 additions & 0 deletions .github/workflows/redis-example.yml
@@ -0,0 +1,42 @@
name: Redis example pipeline

on: [push, pull_request]

jobs:
test-redis:
strategy:
matrix:
go-version: [1.18.x, 1.x]
runs-on: "ubuntu-latest"
steps:

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
working-directory: ./examples/redis
run: go mod verify

- name: modTidy
working-directory: ./examples/redis
run: make tools-tidy

- name: gotestsum
working-directory: ./examples/redis
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-redis*.xml"
if: always()
17 changes: 5 additions & 12 deletions Makefile
@@ -1,21 +1,14 @@
include ./commons-test.mk

.PHONY: test-all
test-all: tools test-unit test-e2e

.PHONY: test-unit
test-unit:
@echo "Running unit tests..."
go run gotest.tools/gotestsum \
--format short-verbose \
--rerun-fails=5 \
--packages="./..." \
--junitfile TEST-unit.xml

.PHONY: test-e2e
test-e2e:
@echo "Running end-to-end tests..."
make -C e2e test

.PHONY: tools
tools:
go mod download
.PHONY: test-examples
test-examples:
@echo "Running example tests..."
make -C examples test
16 changes: 16 additions & 0 deletions commons-test.mk
@@ -0,0 +1,16 @@
.PHONY: test-%
test-%:
@echo "Running $* tests..."
go run gotest.tools/gotestsum \
--format short-verbose \
--rerun-fails=5 \
--packages="./..." \
--junitfile TEST-$*.xml

.PHONY: tools
tools:
go mod download

.PHONY: tools-tidy
tools-tidy:
go mod tidy
151 changes: 6 additions & 145 deletions docs/examples/cockroachdb.md
@@ -1,148 +1,9 @@
# CockroachDB

```go
package main
<!--codeinclude-->
[Creating a CockroachDB container](../../examples/cockroachdb/cockroachdb.go)
<!--/codeinclude-->

import (
"context"
"database/sql"
"fmt"
"testing"
"time"

"github.com/google/go-cmp/cmp"
"github.com/google/uuid"
_ "github.com/jackc/pgx/v4/stdlib"
"github.com/testcontainers/testcontainers-go"
"github.com/testcontainers/testcontainers-go/wait"
)

// Task represents a unit of work to complete. We're going to be using this in
// our example as a way to organize data that is being manipulated in
// the database.
type task struct {
ID string `json:"id"`
Description string `json:"description"`
DateDue *time.Time `json:"date_due,string"`
DateCreated time.Time `json:"date_created,string"`
DateUpdated time.Time `json:"date_updated"`
}

type cockroachDBContainer struct {
testcontainers.Container
URI string
}

func setupCockroachDB(ctx context.Context) (*cockroachDBContainer, error) {
req := testcontainers.ContainerRequest{
Image: "cockroachdb/cockroach:latest-v21.1",
ExposedPorts: []string{"26257/tcp", "8080/tcp"},
WaitingFor: wait.ForHTTP("/health").WithPort("8080"),
Cmd: []string{"start-single-node", "--insecure"},
}
container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
return nil, err
}

mappedPort, err := container.MappedPort(ctx, "26257")
if err != nil {
return nil, err
}

hostIP, err := container.Host(ctx)
if err != nil {
return nil, err
}

uri := fmt.Sprintf("postgres://root@%s:%s", hostIP, mappedPort.Port())

return &cockroachDBContainer{Container: container, URI: uri}, nil
}

func initCockroachDB(ctx context.Context, db sql.DB) error {
// Actual SQL for initializing the database should probably live elsewhere
const query = `CREATE DATABASE projectmanagement;
CREATE TABLE projectmanagement.task(
id uuid primary key not null,
description varchar(255) not null,
date_due timestamp with time zone,
date_created timestamp with time zone not null,
date_updated timestamp with time zone not null);`
_, err := db.ExecContext(ctx, query)

return err
}

func truncateCockroachDB(ctx context.Context, db sql.DB) error {
const query = `TRUNCATE projectmanagement.task`
_, err := db.ExecContext(ctx, query)
return err
}

func TestIntegrationDBInsertSelect(t *testing.T) {
if testing.Short() {
t.Skip("Skipping integration test")
}

ctx := context.Background()

cdbContainer, err := setupCockroachDB(ctx)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
if err := cdbContainer.Terminate(ctx); err != nil {
t.Fatalf("failed to terminate container: %s", err)
}
})

db, err := sql.Open("pgx", cdbContainer.URI+"/projectmanagement")
if err != nil {
t.Fatal(err)
}
defer db.Close()

err = initCockroachDB(ctx, *db)
if err != nil {
t.Fatal(err)
}
defer truncateCockroachDB(ctx, *db)

now := time.Now()

// Insert data
tsk := task{ID: uuid.NewString(), Description: "Update resum茅", DateCreated: now, DateUpdated: now}
const insertQuery = `insert into "task" (id, description, date_due, date_created, date_updated)
values ($1, $2, $3, $4, $5)`
_, err = db.ExecContext(
ctx,
insertQuery,
tsk.ID,
tsk.Description,
tsk.DateDue,
tsk.DateCreated,
tsk.DateUpdated)
if err != nil {
t.Fatal(err)
}

// Select data
savedTsk := task{ID: tsk.ID}
const findQuery = `select description, date_due, date_created, date_updated
from task
where id = $1`
row := db.QueryRowContext(ctx, findQuery, tsk.ID)
err = row.Scan(&savedTsk.Description, &savedTsk.DateDue, &savedTsk.DateCreated, &savedTsk.DateUpdated)
if err != nil {
t.Fatal(err)
}

if !cmp.Equal(tsk, savedTsk) {
t.Fatalf("Saved task is not the same:\n%s", cmp.Diff(tsk, savedTsk))
}
}
```
<!--codeinclude-->
[Test for a CockroachDB container](../../examples/cockroachdb/cockroachdb_test.go)
<!--/codeinclude-->