error parsing database password in database.yml

[rezan83]

Description

my database pass was something like this: longname#?
when running dev it failed to make transaction giving the error: could not create new transaction: dial tcp: address tcp/longname: unknown port

Expected Behavior

one should be able to make whatever password he wants for the database

Actual Behavior

because there was no actual tcp with the name tcp/longname I thought that its a parsing problem with the #? part of the password. I have no idea why and how!

Info

for me, I simply solved the problem with changing the password and delete the #? part and using only ordinary characters and that's how I made sure that that was the problem

[stanislas-m]

Hi @rezan83, can you provide more info about your setup? At least your OS and the database used. Thanks!

[rezan83]

ubuntu 19.04, postgres 11.4

[rezan83]

the output of buffalo info:
`### Buffalo Version
v0.14.2

App Information

Pwd=/home/rezan/go/src/github.com/buffalo_project/first
Root=/home/rezan/go/src/github.com/buffalo_project/first
GoPath=/home/rezan/go
PackagePkg=github.com/buffalo_project/first
ActionsPkg=github.com/buffalo_project/first/actions
ModelsPkg=github.com/buffalo_project/first/models
GriftsPkg=github.com/buffalo_project/first/grifts
WithModules=false
Name=first
Bin=bin/first
VCS=git
WithPop=true
WithSQLite=false
WithDep=false
WithWebpack=true
WithNodeJs=true
WithYarn=true
WithDocker=true
WithGrifts=true
AsWeb=true
AsAPI=false
PackageJSON={map[]}

Go Version

go version go1.12.6 linux/amd64

Go Env

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rezan/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rezan/go"
GOPROXY=""
GORACE=""
GOROOT="/snap/go/3947"
GOTMPDIR=""
GOTOOLDIR="/snap/go/3947/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build958201183=/tmp/go-build -gno-record-gcc-switches"

Node Version

v10.15.2

NPM Version

6.9.0

Yarn Version

1.12.3

PostgreSQL Version

PostgreSQL Not Found

MySQL Version

MySQL Not Found

SQLite Version

3.27.2 2019-02-25 16:06:06 bd49a8271d650fa89e446b42e513b595a717b9212c91dd384aab871fc1d0alt1

Dep Version

could not find a Gopkg.toml file

Dep Status

could not find a Gopkg.toml file

config/buffalo-app.toml

name = "first"
bin = "bin/first"
vcs = "git"
with_pop = true
with_sqlite = false
with_dep = false
with_webpack = true
with_nodejs = true
with_yarn = true
with_docker = true
with_grifts = true
as_web = true
as_api = false

config/buffalo-plugins.toml

[[plugin]]
binary = "buffalo-plugins"
go_get = "github.com/gobuffalo/buffalo-plugins"

[[plugin]]
binary = "buffalo-pop"
go_get = "github.com/gobuffalo/buffalo-pop"
`

[andrewmelis]

This issue bit me as well while using postgres. Turns out it was an "issue" with lib/pq's use of net/url to parse connection strings that contain reserved characters (see RFC 3986). Reserved characters in a connection string need to be URL encoded for things to work properly.

This comment in lib/pq pointed me in the right direction: lib/pq#787 (comment)

Also, here's a go playground illustrating the issue in case it helps: https://play.golang.org/p/bqArco_cMyP

---

Changing pass to pass# in Test_ConnectionDetails_Finalize is sufficient to reproduce the issue

pop/connection_details_test.go

Lines 13 to 21 in aab726f

	URL: "postgres://user:pass@host:port/database",
	}
	err := cd.Finalize()
	r.NoError(err)
	r.Equal("database", cd.Database)
	r.Equal("postgres", cd.Dialect)
	r.Equal("host", cd.Host)
	r.Equal("pass", cd.Password)

following the breadcrumbs...

pop/connection_details.go

Lines 78 to 80 in aab726f

	if up, ok := urlParser[cd.Dialect]; ok {
	return up(cd)
	}

pop/dialect_postgresql.go

Line 214 in aab726f

name, err = pg.ParseURL(name)

pop/dialect_postgresql.go

Line 18 in aab726f

pg "github.com/lib/pq"

then, over in lib/pq, ParseUrl calls net/url
https://github.com/lib/pq/blob/78223426e7c66d631117c0a9da1b7f3fde4d23a5/url.go#L6-L33

---

The next question is what to do about the situation. I can understand the use of the underlying url parsing library, but it seems clear that some users (myself included) expect to be able to use an unaltered password when specifying connection details.

Is a note in the documentation visible enough as a solution? Should something be responsible for encoding any passwords that include reserved characters so that things just work?

[onesick]

@andrewmelis

Hey Andrew, you are on the right track. I just had to solve the same problem where my password used a reserved character. The solution is you put all the address string information into url.URL struct. You can put userinfo into url.URL.User = url.UserPassword(user, password). You can print out the final constructed url string and can see that it is encoded to verify.