Running local CHT Sync setup with couch2pg, PostgreSQL, pgAdmin, dbt, and CouchDB

Hi there!

I’m trying to run the local set up of CHT Sync per the docs. My assumption is that the values in env.template file are valid as is for a happy path of running the full stack - but maybe this is not the case?

My steps so far have been:

  1. clone the repo locally
  2. copy the env.template to .env
  3. call docker: docker compose -f docker-compose.postgres.yml -f docker-compose.yml up -d

What I expect to happen is CHT Sync to startup and work. What happens instead is that while dbt, postgres and pgadmin seem to start OK, couchdb and couch2pg have errors.

couch2pg has 404 errors when trying to get /medic/_changes?limit=0&since=0 from couch:

node:internal/process/promises:391
    triggerUncaughtException(err, true /* fromPromise */);
    ^
{
  error: 'not_found',
  reason: 'Database does not exist.',
  status: 404,
  name: 'not_found',
  message: 'Database does not exist.',
  stack: 'Error\n' +
    '    at Object.generateErrorFromResponse (/node_modules/pouchdb-errors/lib/index.js:100:18)\n' +
    '    at fetchJSON (/node_modules/pouchdb-adapter-http/lib/index.js:251:31)\n' +
    '    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' +
    '    at async fetchData (/node_modules/pouchdb-adapter-http/lib/index.js:1029:24)'
}

Couch, unsurprisingly has errors that /medic is 404, but more so has other errors too:

[notice] 2024-09-04T21:02:52.687694Z couchdb@127.0.0.1 <0.1445.0> fe807c6ed5 couchdb:5984 172.20.0.2 medic GET /medic/_changes?limit=0&since=0 404 ok 1
[notice] 2024-09-04T21:02:52.743711Z couchdb@127.0.0.1 <0.1445.0> f2e8a0fdbd couchdb:5984 172.20.0.2 medic POST /_session 200 ok 1
[notice] 2024-09-04T21:02:52.757588Z couchdb@127.0.0.1 <0.1446.0> 7927fb9064 couchdb:5984 172.20.0.2 medic GET /medic/_changes?seq_interval=1000&since=0&limit=25 404 ok 1
[notice] 2024-09-04T21:02:53.058841Z couchdb@127.0.0.1 <0.442.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.
[error] 2024-09-04T21:02:53.059110Z couchdb@127.0.0.1 emulator -------- Error in process <0.1463.0> on node 'couchdb@127.0.0.1' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,[<<"_users">>],[{file,"src/mem3_shards.erl"},{line,430}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,405}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,434}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,100}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,214}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,160}]}]}

[error] 2024-09-04T21:02:53.059188Z couchdb@127.0.0.1 emulator -------- Error in process <0.1463.0> on node 'couchdb@127.0.0.1' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,[<<"_users">>],[{file,"src/mem3_shards.erl"},{line,430}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,405}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,434}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,100}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,214}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,160}]}]}
1 Like

oh - update! the next section in the docs “Separate CouchDB instance” seems to work well!

Here’s the steps I took for this:

  1. set up a docker helper instance at https://192-168-68-26.local-ip.medicmobile.org:10445/
  2. edit my .env bottom #coucdb section to be:
    # couchdb
    COUCHDB_USER=medic
    COUCHDB_PASSWORD=password
    COUCHDB_DBS="medic" # space separated list of databases you want to sync e.g "medic medic_sentinel"
    COUCHDB_HOST=192-168-68-26.local-ip.medicmobile.org
    COUCHDB_PORT=10445
    COUCHDB_SECURE=true
    
  3. ran the docker call: docker compose -f docker-compose.postgres.yml -f docker-compose.yml up
  4. logged into pgadmin (after setting up a new connection to postgres host) and was able to browse all the imported rows in v1 database

So it’s just the all in one with couchdb that has the errors, while using an external couchdb seems to work OK!

3 Likes

5 posts were split to a new topic: Errors running CHT Sync in k8s with helm chart

@njuguna tracked the issue down to the medic database not existing in couch and created an issue to track the fix for it.

1 Like

@mrjones an update on this. After some discussion we decided that running CouchDB as part of the CHT Sync setup is not very beneficial. Connecting to an existing CouchDB instance has the added benefit that users can see they sync process as they add docs to their CouchDB instance rather than having static docs. With that in mind we updated the documentation to remove the Run all CHT Sync services locally section.

Gotcha - makes sense! I agree - given most folks will want to see their data from the CHT Core instance.

Thanks for the followup!