Math DB connection issue

[raulsperoni]

Expected behavior:
I'm trying to run the DEV deployment as instructed in the docs but I'm getting this on the math container. Any ideas on why this could be happening are appreciated!

polis-dev-math-1         | org.postgresql.util.PSQLException: Connection to localhost:80 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
polis-dev-math-1         | 	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:342)
polis-dev-math-1         | 	at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54)
polis-dev-math-1         | 	at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:253)
polis-dev-math-1         | 	at org.postgresql.Driver.makeConnection(Driver.java:434)
polis-dev-math-1         | 	at org.postgresql.Driver.connect(Driver.java:291)
polis-dev-math-1         | 	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:683)
polis-dev-math-1         | 	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:191)
polis-dev-math-1         | 	at clojure.java.jdbc$get_connection.invokeStatic(jdbc.clj:272)
polis-dev-math-1         | 	at clojure.java.jdbc$get_connection.invoke(jdbc.clj:193)
polis-dev-math-1         | 	at clojure.java.jdbc$db_query_with_resultset.invokeStatic(jdbc.clj:848)
polis-dev-math-1         | 	at clojure.java.jdbc$db_query_with_resultset.invoke(jdbc.clj:818)
polis-dev-math-1         | 	at clojure.java.jdbc$query.invokeStatic(jdbc.clj:874)
polis-dev-math-1         | 	at clojure.java.jdbc$query.invoke(jdbc.clj:854)
polis-dev-math-1         | 	at clojure.java.jdbc$query.invokeStatic(jdbc.clj:867)
polis-dev-math-1         | 	at clojure.java.jdbc$query.invoke(jdbc.clj:854)
polis-dev-math-1         | 	at polismath.components.postgres$query.invokeStatic(postgres.clj:53)
polis-dev-math-1         | 	at polismath.components.postgres$query.invoke(postgres.clj:47)
polis-dev-math-1         | 	at polismath.components.postgres$query.invokeStatic(postgres.clj:52)
polis-dev-math-1         | 	at polismath.components.postgres$query.invoke(postgres.clj:47)
polis-dev-math-1         | 	at polismath.components.postgres$poll.invokeStatic(postgres.clj:77)
polis-dev-math-1         | 	at polismath.components.postgres$poll.invoke(postgres.clj:72)
polis-dev-math-1         | 	at polismath.poller$poll$fn__58731$state_machine__55416__auto____58766$fn__58771.invoke(poller.clj:22)
polis-dev-math-1         | 	at polismath.poller$poll$fn__58731$state_machine__55416__auto____58766.invoke(poller.clj:22)
polis-dev-math-1         | 	at clojure.core.async.impl.runtime$run_state_machine.invokeStatic(runtime.clj:62)
polis-dev-math-1         | 	at clojure.core.async.impl.runtime$run_state_machine.invoke(runtime.clj:61)
polis-dev-math-1         | 	at clojure.core.async.impl.runtime$run_state_machine_wrapped.invokeStatic(runtime.clj:66)
polis-dev-math-1         | 	at clojure.core.async.impl.runtime$run_state_machine_wrapped.invoke(runtime.clj:64)
polis-dev-math-1         | 	at clojure.core.async.impl.runtime$take_BANG_$fn__50152.invoke(runtime.clj:75)
polis-dev-math-1         | 	at clojure.core.async.impl.channels.ManyToManyChannel$fn__50011.invoke(channels.clj:269)
polis-dev-math-1         | 	at clojure.lang.AFn.run(AFn.java:22)
polis-dev-math-1         | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
polis-dev-math-1         | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
polis-dev-math-1         | 	at clojure.core.async.impl.concurrent$counted_thread_factory$reify__49815$fn__49816.invoke(concurrent.clj:29)
polis-dev-math-1         | 	at clojure.lang.AFn.run(AFn.java:22)
polis-dev-math-1         | 	at java.base/java.lang.Thread.run(Thread.java:1583)
polis-dev-math-1         | Caused by: java.net.ConnectException: Connection refused
polis-dev-math-1         | 	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
polis-dev-math-1         | 	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
polis-dev-math-1         | 	at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
polis-dev-math-1         | 	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
polis-dev-math-1         | 	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
polis-dev-math-1         | 	at java.base/java.net.Socket.connect(Socket.java:751)
polis-dev-math-1         | 	at org.postgresql.core.PGStream.createSocket(PGStream.java:243)
polis-dev-math-1         | 	at org.postgresql.core.PGStream.<init>(PGStream.java:98)
polis-dev-math-1         | 	at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:132)
polis-dev-math-1         | 	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258)
polis-dev-math-1         | 	... 34 more

[metasoarous]

Hey @raulsperoni. Thanks for posting here.

I'm a little suspicious that the PSQLException is trying to connect to localhost:80. The standard port would be 5432. If you already have postgres installed on your system, you may have tried to change this from the default in your .env file. However, there was a bug with the way that we were trying to customize this, in which case it's not just the math worker that's failing to connect to the db, but also the server (if you look further up the logs). This has now been resolved in #1749. Please take a look at that and let me know if it fixes your issue (and please feel free to review that PR if you have a minute).

All that having been said, port 80 would probably not work for the postgres port, since this is the port the HTTP server is exposed on (see HTTP_port in example.env). So it's possible that this is just a red herring and there is something else going on. Either way, let us know what you figure out, and we'll try to assist if you're still having issues.

Thanks again!

[raulsperoni]

Hey @metasoarous thanks for your reply! I haven't tried to use 80 for Postgres which is why this error was so odd. In the past, I was able to run polis without issues. I will try again and look into #1749 because being able to use an external Postgres instance (which I can share with other parts of what I'm trying to do) is something I wanted to explore.

Thanks!

[metasoarous]

Hey @raulsperoni. Sorry for the late reply. I did read your message, but wasn't immediately sure what to say in response. Looking back, I still think it's weird that it's trying to connect on 80, so I'd just suggest tinkering with your configuration files a bit and making sure they're properly loaded. I know different systems come with different constraints around port mapping on the host, so just be on the lookout for other things that are consuming various ports, and ensuring that ports on blocked by the host operating system. However, the fact that the log message in this case is coming from inside a container (rather than from docker itself) probably means that the port being referred to is as it stands inside the network docker-compose puts together to wire up the containers, in which case system level constraints or conflicts shouldn't be an issue, unless it's a config that is being used for both internal and host configuration. Otherwise, I'd say just make sure the database itself is actually running. Those are the things I'd check. Sorry for not having a clearer answer, but continue to post back with status updates if you're still having issues here, and if you figure out what's going on please leave some breadcrumbs here for the next person. Thanks!

[raulsperoni]

Hey @metasoarous, just now I had time to dive in again. I can confirm that this is the error the math container gives when DATABASE_URL is not correctly set. I don't know why because for me clojure is pretty unreadable 😢 . I could finally solve it and now I have new problems! which is always good. So feel free to close this ticket. Or maybe repurpose it to improve the error message. Thanks for your help!