"Failed to init storage factory" error when trying to start jaeger-all-in-one with Badger

[lowzzz]

Hello,

I'm trying to run jaeger-all-in-one with the Badger non-ephemeral storage using command from docs

pyrron@pyrron-20T2002VRT:~/Documents/job$ docker run -e SPAN_STORAGE_TYPE=badger -e BADGER_EPHEMERAL=false -e BADGER_DIRECTORY_VALUE=/badger/data -e BADGER_DIRECTORY_KEY=/badger/key -v /home/pyrron/Documents/job/badger:/badger -p 16687:16686 jaegertracing/all-in-one:1.53.0

I receive the "Failed to init storage factory" error when using jaeger version >=1.50.0

{"level":"fatal","ts":1705237596.280394,"caller":"all-in-one/main.go:113","msg":"Failed to init storage factory","error":"cannot open directory "/badger/key" error: open /badger/key: permission denied","stacktrace":"main.main.func1\n\tgithub.com/jaegertracing/jaeger/cmd/all-in-one/main.go:113\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.8.0/command.go:983\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.8.0/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.8.0/command.go:1039\nmain.main\n\tgithub.com/jaegertracing/jaeger/cmd/all-in-one/main.go:245\nruntime.main\n\truntime/proc.go:267"}

However, it works just fine with versions up to 1.49.0.

The problem seems to be related with this change (quote from the release notes)

Use non-root user in built containers (@nikzayn in #4783) - this change may cause issues with existing installations using Badger storage, because the existing files would be owned by a different user and would not be writeable after Jaeger upgrade. The workaround is to manually chown the files to the new user (uid=10001).

But I can't figure out how to get it working. Changing folder group to docker didn't help:

pyrron@pyrron-20T2002VRT:~/Documents/job$ ls -tahl | grep badger
drwxrwxr-x 4 pyrron docker 4.0K Jan 14 12:24 badger

The folder itself is empty:

pyrron@pyrron-20T2002VRT:~/Documents/job$ ls -tahl badger/
total 8.0K
drwxrwxr-x 2 pyrron docker 4.0K Jan 14 14:29 .
drwxr-xr-x 8 pyrron root 4.0K Jan 14 12:20 ..

i'd appreciate any advice on this. Thank you

[yurishkuro]

If the folder is empty just delete it

[Toudahl]

Hi @yurishkuro

Im having the same permission issue.
I have an nfs volume on a remote storage. i.e jaeger-data:/badger

jaeger-data is of course a volume on a remote host. The folder on the host is empty, since i deleted all the data in an attempt to avoid the issue. The data was just from a hobby project, so it was an easy choice.

Now despite the folder being empty on the remote host, jaeger-all-in-one cannot write to it.

Am i to understand that i need to chown the folder on the remote host to a userid from a container to be able to run the container?
That seems ... wrong...