CouchDB installation issue on Apple Silicon chip

Esther_Moturi · December 5, 2023, 9:58am

Hey team,

I am running npm run local-images within cht-core folder and I am getting the output below:

medic@4.5.0 local-images
export VERSION=$(node ./scripts/build/get-version.js) && ./scripts/build/build-service-images.sh && node scripts/build/cli localDockerComposeFiles
build-service-images: running
copy-static-files: copying
cp: api/node_modules/bowser/bundled.js: No such file or directory

Am I missing something?

cc @Ben_Kiarie

mrjones · December 5, 2023, 3:57pm

Hi @Esther_Moturi - thanks for reporting the issue with our npm build scripts.

What is your end goal you’re looking to achieve? Happy to help either way, but sometimes it’s helpful to know where someone is heading so maybe we can find an alternate path to their destination.

Checking on my Ubuntu x86 system, I can confrim this script works. Here’s what you would see if it were able to continue past your error:

npm run local-images

> medic@4.5.0 local-images
> export VERSION=$(node ./scripts/build/get-version.js) && ./scripts/build/build-service-images.sh && node scripts/build/cli localDockerComposeFiles

build-service-images: running
copy-static-files: copying
copy-static-files: done
cleancss-api: cleaning
cleancss-api: done

Can you confirm the following:

what version of node? you can check with node --version
what version of npm? you can check with npm --version
that you have latest master with git checkout master;git pull orgin in cht-core directory

diana · December 6, 2023, 9:47am

hi @Esther_Moturi

To build the images correctly, you must first locally build webapp. From the looks of the error, some code tries to copy webapp compiled source code and complains it can’t find it.
Please try to run the following commands in the root folder:

npm ci
npm run build-dev
npm run local-images

Esther_Moturi · December 7, 2023, 4:11am

Hey @mrjones , the goal is to setup couchDB in CHT 4.x, I am using node: ‘v18.3.0’, npm: ‘8.11.0’. I will try and downgrade my node version and see if it works.

Esther_Moturi · December 7, 2023, 4:33am

Hey @diana , I downgraded my node version, npm ci ran without any issues, I ran npm run build-dev and got the error below:

medic@4.5.0 build-ddocs
mkdir -p build/ddocs && cp -r ddocs/* build/ddocs/ && node scripts/build/cli setDdocsVersion && node scripts/build/cli setBuildInfo && node ./scripts/build/ddoc-compile.js primary && mkdir -p api/build/ddocs && cp build/ddocs/*.json api/build/ddocs/
node:fs:585
handleErrorFromBinding(ctx);
^
Error: ENOTDIR: not a directory, open ‘/Users/esthermoturi/cht-core/build/ddocs/medic-db/.DS_Store/version’
at Object.openSync (node:fs:585:3)
at Object.writeFileSync (node:fs:2157:35)
at /Users/esthermoturi/cht-core/scripts/build/index.js:188:10
at Array.forEach ()
at /Users/esthermoturi/cht-core/scripts/build/index.js:187:11
at Array.forEach ()
at setDdocsVersion (/Users/esthermoturi/cht-core/scripts/build/index.js:181:13)
at /Users/esthermoturi/cht-core/scripts/build/cli.js:13:21
at Object. (/Users/esthermoturi/cht-core/scripts/build/cli.js:14:3)
at Module._compile (node:internal/modules/cjs/loader:1103:14) {
errno: -20,
syscall: ‘open’,
code: ‘ENOTDIR’,
path: ‘/Users/esthermoturi/cht-core/build/ddocs/medic-db/.DS_Store/version’
}

mrjones · December 7, 2023, 5:51am

@Esther_Moturi

To clarify, if this is you’re only goal, then the easier way to do this is to following the developer setup steps under CouchDB Setup.

If you’d like still get the npm commands to run, then you’re in with @diana in this thread helping out!

diana · December 7, 2023, 8:39am

Hi @Esther_Moturi

Can you please try deleting the ‘/Users/esthermoturi/cht-core/build/ddocs/medic-db/.DS_Store’ folder? Our build script expects every folder in ddocs to be a CouchDb view descriptor, and MacOS is adding these additional folders every time you navigate to it (source).

Esther_Moturi · December 7, 2023, 9:14am

Alright, I deleted it, saw the screenshot below

Then the error below appeared:

diana · December 7, 2023, 11:37am

Hi @Esther_Moturi

It seems that nodejs doesn’t have access to the docker daemon. Can you please share which version of docker you have installed? Thanks!

Kymoraa · January 22, 2024, 6:34am

Hello @diana

I initially had the same error as @Esther_Moturi but upon running the commands you suggested npm ci, npm run build-dev then npm run local-images, I am now getting the error below.

=> [internal] load build definition from Dockerfile                          0.0s
 => => transferring dockerfile: 460B                                  0.0s
 => [internal] load .dockerignore                                   0.0s
 => => transferring context: 266B                                   0.0s
 => ERROR [internal] load metadata for docker.io/library/alpine:3.15                  1.7s
------
 > [internal] load metadata for docker.io/library/alpine:3.15:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: rpc error: code = Unknown desc = open /Users/jackiemoraa/.docker/.token_seed: permission denied
/Users/jackiemoraa/Documents/Work/cht-core/scripts/build/index.js:201
   return reject(new Error(`${command} exited with ${code}`));
          ^

Error: docker exited with 1
  at ChildProcess.<anonymous> (/Users/jackiemoraa/Documents/Work/cht-core/scripts/build/index.js:201:21)
  at ChildProcess.emit (node:events:527:28)
  at maybeClose (node:internal/child_process:1092:16)
  at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)

What might I be overlooking?
cc. @jkuester

diana · January 22, 2024, 8:39am

Hi @Kymoraa

From the error, it seems that the user you are running this command from doesn’t have sufficient permissions:

open /Users/jackiemoraa/.docker/.token_seed: permission denied

Can you either:

try adding more permissions to your user
trying to run the command with super user to check if it fixes things

Kymoraa · January 22, 2024, 9:39am

Hey,
Thanks for your response @diana
The user is an admin, so they have all the permissions.

Interestingly, re-running the command finally completed. (Without any changes to the user)

Successfully built 80355cea81ce
Successfully tagged medicmobile/cht-nginx:4.5.0-dev.1705914674948

So the next part in the set-up CHT Core dev environment setup | Community Health Toolkit

Confirm you have these four files in the local-build folder: docker-compose.yml , cht-core.yml , cht-couchdb.yml and cht-couchdb-clustered.yml
Use ‘docker scan’ to run Snyk tests against images to find vulnerabilities and learn how to fix them
build-service-images: done

I only have the cht-x files. I am missing the docker-compose.yml file.
Should I manually add it?

diana · January 22, 2024, 6:10pm

Hi @Kymoraa

Those are the correct files. I believe the docs are incorrect where they say a docker-compose.yml file is required.

diana · January 22, 2024, 6:12pm

I believe the docker-compose.yml file you were indeed supposed to create manually, to add the overrides: CHT Core dev environment setup | Community Health Toolkit

Esther_Moturi · January 23, 2024, 8:35am

Hey @diana ,

I can work on updating the docs. What should be included in the manually created docker-compose.yml file?

Kymoraa · January 23, 2024, 10:35am

Hey @diana

This is well noted.

However, I am a bit lost. If I manually create the docker-compose.yml file, I will be going back up in the documentation process, right?
So, what goes into the docker-compose.yml file and couchdb-override.yml file?

Would it be this?

mkdir -p ~/cht-docker
curl -s -o ~/cht-docker/docker-compose.yml https://staging.dev.medicmobile.org/_couch/builds_4/medic:medic:master/docker-compose/cht-couchdb.yml
cat > ~/cht-docker/couchdb-override.yml << EOF
version: '3.9'
services:
    couchdb:
        ports:
          - "5984:5984"
          - "5986:5986"
EOF

Then do I proceed to start couch with this:

cd ~/cht-docker 
COUCHDB_USER=medic COUCHDB_PASSWORD=password docker-compose -f docker-compose.yml -f couchdb-override.yml up -d

or do I need to jump back down and execute this:

Set environment variables required by npm and node

echo "export COUCH_NODE_NAME=nonode@nohost">> ~/.$(basename $SHELL)rc
echo "export COUCH_URL=http://medic:password@localhost:5984/medic">> ~/.$(basename $SHELL)rc
. ~/.$(basename $SHELL)rc
echo "export COUCH_DB_USER=medic">> ~/.$(basename $SHELL)rc
echo "export COUCH_DB_PASSWORD=password">> ~/.$(basename $SHELL)rc

Then followed by: In your terminal, navigate inside the `local-build` folder, stop any running Docker containers and start the CHT Docker containers:

cd ~/cht-core/local-build
docker kill $(docker ps -q)
docker-compose -f cht-couchdb.yml up -d

Looking forward to some guidance.
Thanks

jkuester · January 23, 2024, 11:33pm

Okay, yeah, I see that the docs get a bit confused here.

I do not have a Mac to be able to confirm this, but I think the most simple way to get things running is to do the steps in “CouchDB Setup in CHT 4.x”, but stop before running the docker-compose -f docker-compose.yml -f couchdb-override.yml up -d command. Then:

Run the npm run local-images script in the cht-core project as noted in the “CouchDB Setup in 4.x for Apple Silicon Chip” instructions.
Then copy the contents of cht-core/local-build/cht-couchdb.yml and overwrite the contents of your cht-docker/docker-compose.yml file with it.

Once that is done, you can go back to the “CouchDB Setup in CHT 4.x” and run the Couch container with:

cd ~/cht-docker 
COUCHDB_USER=medic COUCHDB_PASSWORD=password docker-compose -f docker-compose.yml -f couchdb-override.yml up -d

Please let me know if this works and we can be sure to update the docs to be more clear!

Kymoraa · January 24, 2024, 12:13pm

Hey @jkuester
I had already run npm run local-images and got the cht-xxx.yml files generated.
So now, I have gone back and manually created the docker-compose.yml and couchdb-override.yml in a cht-docker directory and I have pasted the code from cht-couchdb.yml to both these files as advised.

Running the command from cht-docker, I don’t get errors but warnings since these variables are not set.

WARNING: The COUCHDB_SECRET variable is not set. Defaulting to a blank string.
WARNING: The COUCHDB_UUID variable is not set. Defaulting to a blank string.

Then back to cht-core directory, I have run npm run build-dev-watch. I get to the waiting ... point. On the second terminal, running npm run dev-api, I get this:

Work/cht-core [master●] » npm run dev-api

> medic@4.5.0 dev-api
> ./scripts/build/copy-static-files.sh && TZ=UTC nodemon --inspect=0.0.0.0:9229 --ignore 'api/build/static' --ignore 'api/build/public' --watch api --watch 'shared-libs/**/src/**' api/server.js -- --allow-cors

copy-static-files: copying
copy-static-files: done
[nodemon] 3.0.1
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): api/**/* shared-libs/**/src/**
[nodemon] watching extensions: js,mjs,cjs,json
[nodemon] starting `node --inspect=0.0.0.0:9229 api/server.js --allow-cors`
Debugger listening on ws://0.0.0.0:9229/6db01c11-7ce3-471e-b59f-5f86c4b8a036
For help, see: https://nodejs.org/en/docs/inspector
2024-01-24T12:10:59.698 INFO: Running server checks… 
Node Environment Options: 'undefined'
Node Version: 16.15.0 in development mode
CouchDB Version: 3.3.3
Error: System databases do not exist
    at checkCluster (/Users/jackiemoraa/Documents/Work/cht-core/shared-libs/server-checks/src/checks.js:102:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async couchDbCheck (/Users/jackiemoraa/Documents/Work/cht-core/shared-libs/server-checks/src/checks.js:135:7)
    at async /Users/jackiemoraa/Documents/Work/cht-core/api/server.js:23:5

We’re making progress, but I am still not successful.

jkuester · January 24, 2024, 2:23pm

For a local dev instance, I believe this is fine. (I do not set values for either of these when running locally.)

Error: System databases do not exist

This is much more puzzling. Seems to be some kind of corruption in Couch, or in-completion in the the DB setup… I think the next thing to try is tearing down Couch and re-running it. So from the cht-docker directory:

docker-compose down -v
sudo rm -rf srv/
COUCHDB_USER=medic COUCHDB_PASSWORD=password docker-compose -f docker-compose.yml -f couchdb-override.yml up -d

(Make sure to include both the docker-compose.yml and the couchdb-override.yml in the docker-compose up command. I have seem behavior similar to what you describe when I forgot to include the couchdb-override.yml…)

Once the Couch container spins up successfully, you should see something like this in the logs for that container:

[notice] 2024-01-24T14:18:45.265049Z couchdb@127.0.0.1 <0.357.0> -------- rexi_buffer : cluster stable

Also, you should be able to navigate to http://localhost:5984/_utils/ and login to the Fauxton interface. Before starting the CHT api server, the only DBs you should see in Fauxton are _global_changes, _replicator, and _users. Finally, if all of that looks good, then you can try going back to cht-core again and running npm run dev-api again to start the api server (no need to re-run build-dev-watch until you make actual changes to the webapp code).

My calendar is wide open today, so please feel free to DM if you want to have a quick call and work through any of these issues!

Kymoraa · January 25, 2024, 8:36am

Running docker-compose down -v from the cht-docker directory I immediately get this error

Work/cht-docker » docker-compose down -v
ERROR: Missing mandatory value for "environment" option interpolating
['COUCHDB_USER=${COUCHDB_USER:-admin}',
'COUCHDB_PASSWORD=${COUCHDB_PASSWORD:?COUCHDB_PASSWORD must be set}',
'COUCHDB_SECRET=${COUCHDB_SECRET}', 'COUCHDB_UUID=${COUCHDB_UUID}',
'SVC_NAME=${SVC_NAME:-couchdb}', 'COUCHDB_LOG_LEVEL=${COUCHDB_LOG_LEVEL:
info}'] in service "couchdb": COUCHDB_PASSWORD must be set

This is what I have in the environment as per the docs

export COUCH_NODE_NAME=nonode@nohost
export COUCH_URL=http://medic:password@localhost:5984/medic
export COUCH_DB_USER=medic
export COUCH_DB_PASSWORD=password

So might the difference in the variable name as in the docker-compose.yml file vs the environment e.g., COUCHDB_USER vs COUCH_DB_USER respectively have anything to do with the error? I am not sure…

Also, as it is, I am able to login to couch: http://localhost:5984/_utils/#/_all_dbs. But there are no DBs.