Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for libsql and Turso #1042

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
@@ -1,5 +1,5 @@
SOURCE ?= file go_bindata github github_ee bitbucket aws_s3 google_cloud_storage godoc_vfs gitlab
DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb yugabytedb clickhouse mongodb sqlserver firebird neo4j pgx pgx5 rqlite
DATABASE ?= postgres mysql redshift cassandra spanner cockroachdb yugabytedb clickhouse mongodb sqlserver firebird neo4j pgx pgx5 rqlite libsql
DATABASE_TEST ?= $(DATABASE) sqlite sqlite3 sqlcipher
VERSION ?= $(shell git describe --tags 2>/dev/null | cut -c 2-)
TEST_FLAGS ?=
Expand Down
113 changes: 57 additions & 56 deletions README.md
Expand Up @@ -9,41 +9,42 @@

# migrate

__Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__
**Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).**

* Migrate reads migrations from [sources](#migration-sources)
and applies them in correct order to a [database](#databases).
* Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof.
(Keeps the drivers lightweight, too.)
* Database drivers don't assume things or try to correct user input. When in doubt, fail.
- Migrate reads migrations from [sources](#migration-sources)
and applies them in correct order to a [database](#databases).
- Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof.
(Keeps the drivers lightweight, too.)
- Database drivers don't assume things or try to correct user input. When in doubt, fail.

Forked from [mattes/migrate](https://github.com/mattes/migrate)

## Databases

Database drivers run migrations. [Add a new database?](database/driver.go)

* [PostgreSQL](database/postgres)
* [PGX v4](database/pgx)
* [PGX v5](database/pgx/v5)
* [Redshift](database/redshift)
* [Ql](database/ql)
* [Cassandra / ScyllaDB](database/cassandra)
* [SQLite](database/sqlite)
* [SQLite3](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165))
* [SQLCipher](database/sqlcipher)
* [MySQL / MariaDB](database/mysql)
* [Neo4j](database/neo4j)
* [MongoDB](database/mongodb)
* [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170))
* [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171))
* [Google Cloud Spanner](database/spanner)
* [CockroachDB](database/cockroachdb)
* [YugabyteDB](database/yugabytedb)
* [ClickHouse](database/clickhouse)
* [Firebird](database/firebird)
* [MS SQL Server](database/sqlserver)
* [rqlite](database/rqlite)
- [PostgreSQL](database/postgres)
- [PGX v4](database/pgx)
- [PGX v5](database/pgx/v5)
- [Redshift](database/redshift)
- [Ql](database/ql)
- [Cassandra / ScyllaDB](database/cassandra)
- [SQLite](database/sqlite)
- [SQLite3](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165))
- [SQLCipher](database/sqlcipher)
- [MySQL / MariaDB](database/mysql)
- [Neo4j](database/neo4j)
- [MongoDB](database/mongodb)
- [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170))
- [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171))
- [Google Cloud Spanner](database/spanner)
- [CockroachDB](database/cockroachdb)
- [YugabyteDB](database/yugabytedb)
- [ClickHouse](database/clickhouse)
- [Firebird](database/firebird)
- [MS SQL Server](database/sqlserver)
- [rqlite](database/rqlite)
- [libsql (Turso)](database/libsql)

### Database URLs

Expand All @@ -70,24 +71,24 @@ $

Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go)

* [Filesystem](source/file) - read from filesystem
* [io/fs](source/iofs) - read from a Go [io/fs](https://pkg.go.dev/io/fs#FS)
* [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata))
* [pkger](source/pkger) - read from embedded binary data ([markbates/pkger](https://github.com/markbates/pkger))
* [GitHub](source/github) - read from remote GitHub repositories
* [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories
* [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories
* [Gitlab](source/gitlab) - read from remote Gitlab repositories
* [AWS S3](source/aws_s3) - read from Amazon Web Services S3
* [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage
- [Filesystem](source/file) - read from filesystem
- [io/fs](source/iofs) - read from a Go [io/fs](https://pkg.go.dev/io/fs#FS)
- [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata))
- [pkger](source/pkger) - read from embedded binary data ([markbates/pkger](https://github.com/markbates/pkger))
- [GitHub](source/github) - read from remote GitHub repositories
- [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories
- [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories
- [Gitlab](source/gitlab) - read from remote Gitlab repositories
- [AWS S3](source/aws_s3) - read from Amazon Web Services S3
- [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage

## CLI usage

* Simple wrapper around this library.
* Handles ctrl+c (SIGINT) gracefully.
* No config search paths, no config files, no magic ENV var injections.
- Simple wrapper around this library.
- Handles ctrl+c (SIGINT) gracefully.
- No config search paths, no config files, no magic ENV var injections.

__[CLI Documentation](cmd/migrate)__
**[CLI Documentation](cmd/migrate)**

### Basic usage

Expand All @@ -104,14 +105,14 @@ $ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate

## Use in your Go project

* API is stable and frozen for this release (v3 & v4).
* Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies.
* To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`.
* Bring your own logger.
* Uses `io.Reader` streams internally for low memory overhead.
* Thread-safe and no goroutine leaks.
- API is stable and frozen for this release (v3 & v4).
- Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies.
- To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`.
- Bring your own logger.
- Uses `io.Reader` streams internally for low memory overhead.
- Thread-safe and no goroutine leaks.

__[Go Documentation](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)__
**[Go Documentation](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)**

```go
import (
Expand Down Expand Up @@ -155,8 +156,8 @@ Go to [getting started](GETTING_STARTED.md)

## Tutorials

* [CockroachDB](database/cockroachdb/TUTORIAL.md)
* [PostgreSQL](database/postgres/TUTORIAL.md)
- [CockroachDB](database/cockroachdb/TUTORIAL.md)
- [PostgreSQL](database/postgres/TUTORIAL.md)

(more tutorials to come)

Expand All @@ -174,15 +175,15 @@ Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-file
## Coming from another db migration tool?

Check out [migradaptor](https://github.com/musinit/migradaptor/).
*Note: migradaptor is not affliated or supported by this project*
_Note: migradaptor is not affliated or supported by this project_

## Versions

Version | Supported? | Import | Notes
--------|------------|--------|------
**master** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first |
**v4** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | Used for stable releases |
**v3** | :x: | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | **DO NOT USE** - No longer supported |
| Version | Supported? | Import | Notes |
| ---------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| **master** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first |
| **v4** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | Used for stable releases |
| **v3** | :x: | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | **DO NOT USE** - No longer supported |

## Development and Contributing

Expand Down
18 changes: 18 additions & 0 deletions database/libsql/README.md
@@ -0,0 +1,18 @@
# libsql

To use with Turso

`libsql://[DATABASE].turso.io?authToken=[TOKEN]&query`

Unlike other migrate database drivers, the libsql driver will automatically wrap each migration in an implicit transaction by default. Migrations must not contain explicit `BEGIN` or `COMMIT` statements. This behavior may change in a future major release. (See below for a workaround.)

Refer to [upstream documentation](https://github.com/mattn/go-sqlite3/blob/master/README.md#connection-string) for a complete list of query parameters supported by the sqlite3 database driver. The auxiliary query parameters listed below may be supplied to tailor migrate behavior. All auxiliary query parameters are optional.

| URL Query | WithInstance Config | Description |
| -------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table. Defaults to `schema_migrations`. |
| `x-no-tx-wrap` | `NoTxWrap` | Disable implicit transactions when `true`. Migrations may, and should, contain explicit `BEGIN` and `COMMIT` statements. |

## Notes

- Uses the `https://github.com/libsql/go-libsql` libsql db driver (cgo)
@@ -0,0 +1 @@
DROP TABLE IF EXISTS pets;
3 changes: 3 additions & 0 deletions database/libsql/examples/migrations/33_create_table.up.sql
@@ -0,0 +1,3 @@
CREATE TABLE pets (
name string
);
@@ -0,0 +1 @@
DROP TABLE IF EXISTS pets;
1 change: 1 addition & 0 deletions database/libsql/examples/migrations/44_alter_table.up.sql
@@ -0,0 +1 @@
ALTER TABLE pets ADD predator bool;