Error 502 when running local CHT instance

Technical Support
1

Hallo,

I am getting error 502 Bad Gateway when I try to follow this guide on how to install CHT local instance. I had run localhost earlier today and it worked fine. I am not sure what the issue could be.

2

Hi @Esther_Moturi

Can you please run docker ps and share the output?
I’m looking to get logs from some of the services, and want to see what is actually running on your machine. Thanks!

3

image
image1920×303 65.6 KB

Here is the output

4

Thanks for sharing.

Can you please run docker logs cht_nginx_1 -n 50 and share the output?

5 
esthermoturi@Esthers-MacBook-Pro cht-local-setup % docker logs cht_nginx_1 -n 50
2023/01/19 09:12:11 [error] 70#70: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.19.0.6:5988/favicon.ico", host: "localhost", referrer: "https://localhost/"
172.19.0.1 - - [19/Jan/2023:09:12:11 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023/01/19 09:12:15 [error] 70#70: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://172.19.0.6:5988/", host: "localhost", referrer: "https://docs.communityhealthtoolkit.org/"
172.19.0.1 - - [19/Jan/2023:09:12:15 +0000] "GET / HTTP/1.1" 502 559 "https://docs.communityhealthtoolkit.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023/01/19 09:12:15 [error] 70#70: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.19.0.6:5988/favicon.ico", host: "localhost", referrer: "https://localhost/"
172.19.0.1 - - [19/Jan/2023:09:12:15 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/server.conf.template to /etc/nginx/conf.d/server.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/ssl-install.sh
Running SSL certificate checks
CERTIFICATE MODE = SELF_SIGNED
self signed SSL cert already exists.
Launching Nginx
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/01/19 09:14:25 [emerg] 67#67: io_setup() failed (38: Function not implemented)
2023/01/19 09:14:25 [emerg] 69#69: io_setup() failed (38: Function not implemented)
2023/01/19 09:14:25 [emerg] 73#73: io_setup() failed (38: Function not implemented)
2023/01/19 09:14:25 [emerg] 71#71: io_setup() failed (38: Function not implemented)
2023/01/19 09:15:49 [error] 67#67: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://172.19.0.3:5988/", host: "localhost", referrer: "https://docs.communityhealthtoolkit.org/"
172.19.0.1 - - [19/Jan/2023:09:15:49 +0000] "GET / HTTP/1.1" 502 559 "https://docs.communityhealthtoolkit.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023/01/19 09:15:49 [error] 67#67: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.19.0.3:5988/favicon.ico", host: "localhost", referrer: "https://localhost/"
172.19.0.1 - - [19/Jan/2023:09:15:49 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023/01/19 09:29:38 [error] 69#69: *8 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://172.19.0.3:5988/", host: "localhost", referrer: "https://docs.communityhealthtoolkit.org/"
172.19.0.1 - - [19/Jan/2023:09:29:38 +0000] "GET / HTTP/1.1" 502 559 "https://docs.communityhealthtoolkit.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023/01/19 09:29:38 [error] 69#69: *8 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.19.0.3:5988/favicon.ico", host: "localhost", referrer: "https://localhost/"
172.19.0.1 - - [19/Jan/2023:09:29:38 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/server.conf.template to /etc/nginx/conf.d/server.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/ssl-install.sh
Running SSL certificate checks
CERTIFICATE MODE = SELF_SIGNED
self signed SSL cert already exists.
Launching Nginx
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/01/19 09:49:20 [emerg] 71#71: io_setup() failed (38: Function not implemented)
2023/01/19 09:49:20 [emerg] 67#67: io_setup() failed (38: Function not implemented)
2023/01/19 09:49:20 [emerg] 69#69: io_setup() failed (38: Function not implemented)
2023/01/19 09:49:20 [emerg] 70#70: io_setup() failed (38: Function not implemented)
2023/01/19 09:49:57 [error] 69#69: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://172.19.0.2:5988/", host: "localhost", referrer: "https://docs.communityhealthtoolkit.org/"
172.19.0.1 - - [19/Jan/2023:09:49:57 +0000] "GET / HTTP/1.1" 502 559 "https://docs.communityhealthtoolkit.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023/01/19 09:49:57 [error] 69#69: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.19.0.1, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://172.19.0.2:5988/favicon.ico", host: "localhost", referrer: "https://localhost/"
172.19.0.1 - - [19/Jan/2023:09:49:57 +0000] "GET /favicon.ico HTTP/1.1" 502 559 "https://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
esthermoturi@Esthers-MacBook-Pro cht-local-setup %
6

Thanks @Esther_Moturi

It looks like your API service is not responding.
Can you please run docker logs cht_api_1 -n 50 and share the output?

7 
'</body></html>\n'

}
StatusCodeError: 503 - “

503 Service Unavailable

\nNo server is available to handle this request.\n\n”
at new StatusCodeError (/api/node_modules/request-promise-core/lib/errors.js:32:15)
at Request.plumbing.callback (/api/node_modules/request-promise-core/lib/plumbing.js:104:33)
at Request.RP$callback [as _callback] (/api/node_modules/request-promise-core/lib/plumbing.js:46:31)
at Request.self.callback (/api/node_modules/request/request.js:185:22)
at Request.emit (node:events:513:28)
at Request. (/api/node_modules/request/request.js:1154:10)
at Request.emit (node:events:513:28)
at IncomingMessage. (/api/node_modules/request/request.js:1076:12)
at Object.onceWrapper (node:events:627:28)
at IncomingMessage.emit (node:events:525:35) {
statusCode: 503,
error: ‘

503 Service Unavailable

\n’ +
‘No server is available to handle this request.\n’ +
‘\n’
}
StatusCodeError: 503 - “

503 Service Unavailable

\nNo server is available to handle this request.\n\n”
at new StatusCodeError (/api/node_modules/request-promise-core/lib/errors.js:32:15)
at Request.plumbing.callback (/api/node_modules/request-promise-core/lib/plumbing.js:104:33)
at Request.RP$callback [as _callback] (/api/node_modules/request-promise-core/lib/plumbing.js:46:31)
at Request.self.callback (/api/node_modules/request/request.js:185:22)
at Request.emit (node:events:513:28)
at Request. (/api/node_modules/request/request.js:1154:10)
at Request.emit (node:events:513:28)
at IncomingMessage. (/api/node_modules/request/request.js:1076:12)
at Object.onceWrapper (node:events:627:28)
at IncomingMessage.emit (node:events:525:35) {
statusCode: 503,
error: ‘

503 Service Unavailable

\n’ +
‘No server is available to handle this request.\n’ +
‘\n’
}
StatusCodeError: 503 - “

503 Service Unavailable

\nNo server is available to handle this request.\n\n”
at new StatusCodeError (/api/node_modules/request-promise-core/lib/errors.js:32:15)
at Request.plumbing.callback (/api/node_modules/request-promise-core/lib/plumbing.js:104:33)
at Request.RP$callback [as _callback] (/api/node_modules/request-promise-core/lib/plumbing.js:46:31)
at Request.self.callback (/api/node_modules/request/request.js:185:22)
at Request.emit (node:events:513:28)
at Request. (/api/node_modules/request/request.js:1154:10)
at Request.emit (node:events:513:28)
at IncomingMessage. (/api/node_modules/request/request.js:1076:12)
at Object.onceWrapper (node:events:627:28)
at IncomingMessage.emit (node:events:525:35) {
statusCode: 503,
error: ‘

503 Service Unavailable

\n’ +
‘No server is available to handle this request.\n’ +
‘\n’
}

8

@diana This is the error I get when I go to the couch_url indicated in docker

image
image1873×593 22.6 KB

9

Hi @Esther_Moturi

It’s not expected that CouchDb is directly accessible from the host, so getting that error in the browser is normal.
According to the logs you shared above, it looks like haproxy is unable to make a connection with Couchdb.

Can you please check the logs of both the haproxy container and the couchdb container and share if you see any errors?

Thanks!

10

Hi @diana I am getting this for haproxy logs : Unrecognized input header: 99 and this for couchdb: No couchdb end point Found at http://medic:password@couchdb:5984
Waiting for cht couchdb

11

Cool, so we’ve likely narrowed it down to CouchDb not responding, although I am seeing the container running in your docker ps call.
Would you mind repeating the docker logs command, but this time for cht_couchdb_1 please?

12

@diana

image
image571×764 7.25 KB

13

@Esther_Moturi - we were privately chatting about how to restart your containers in case that might help. Here’s what I shared based on the set up in the local-setup docs you’re using:

  1. ctrl + c in the terminal window where you first ran docker. It should have a bunch of upgrade-cht-upgrade-service-1 log entries. Since we don’t start the service detached (-d ), you need stop the upgrade service as a first step
  2. docker kill $(docker ps --all --filter "name=cht_" --quiet) - This forcibly kills all containers that start with cht_, which are all the ones we just started
  3. cd ~/cht-local-setup/upgrade/ - make sure you’re still in the right directory
  4. docker-compose up - restart the services we just stopped

Side note - docker-compose down won’t work to stop all related service. This because of the way CHT Upgrade Service is architected. It is intentionally meant to be autonomous from the CHT instances it is managing upgrades for.

15

A post was split to a new topic: Error 502 on local development instance