Describe the Bug
For context, This is what I am hoping to achieve.
I need to run a command to bulk update certain users’ couchDB documents with an additional flag so that they are able to have access to specific forms while those who don’t have the flag aren’t able to access the forms. This is because, Uganda is currently training their VHT members on family planning sequentially. Thus, when the training is completed, only those users need to view those forms and targets. @derick shared this helpful documentation on how to go about editing the contacts: GitHub - medic/cht-conf: A command-line interface for configuring Community Health Toolkit applications
After following the steps detailed, I am running into the error below.
How To Reproduce The Issue
1.Create a contact.csv file with your columns in the csv folder in your current path.
(I am not able to attach the contacts.csv file. But here is a screenshot. I just have the mandatory column ‘documentID’ and I am testing with 1 UUID)
Use the following command to download and edit the documents:
Thanks for your reply
I can see that you have changed directories to the path of the csv file. Is this the case? I am running the command from the root folder and appending the path to the csv.
If I run the command from within the csv folder, I get the expected error that we don’t have the package.json …
Unless, I am not supposed to have the csv file in a csv folder as stated here: 1. Create a contact.csv file with your columns in the csv folder in your current path.?
cht edit-contacts --url=https://medic:password@clone-smarthealth-ke-prod-3-13.lg-apps.com
WARN No project package.json file found at /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/package.json
WARN Project has no dependency on cht-conf.
INFO Checking that https://medic:****@clone-smarthealth-ke-prod-3-13.lg-apps.com/medic/ is available...
INFO Processing config in csv.
INFO Actions:
- edit-contacts
INFO Starting action: edit-contacts…
WARN No csv directory found at /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/csv.
INFO edit-contacts complete.
However, as much as i received the above error, something interesting happened. An empty folder json_docs was created in the csv folder.
And when I ran the same command from the root, the same empty file was created. But here I got the error ERROR missing "documentID" column.
Ok I think we’re getting closer. You’ll see in your output this line:
WARN No csv directory found at /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/csv.
Notice the directory has two /csv paths at the end. If you go up one level in the directory I think it’ll find the right file.
How can I check this?
In your original console output there’s a line that starts with “INFO: Processing CSV file: …”. It’d be worth checking that the file at that path to make sure it’s the csv you expect, eg:
less /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/contact.csv
WARN No csv directory found at /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/csv.
This error above is if I run the command when I cd into the csv folder… (which I saw you did in the example you shared)
But I’m not sure if we are supposed to cd into that csv folder to run the command
So if I run the command at the root, the error I get is instead the one below. Where it finds the contact.csv file but the column is missing
config-lg-forked/config-lg-kenya [test-update-users●] » cht edit-contacts --url=https://medic:668dd65770d21ffe@clone-smarthealth-ke-prod-3-13.lg-apps.com
INFO Checking that https://medic:****@clone-smarthealth-ke-prod-3-13.lg-apps.com/medic/ is available...
INFO Processing config in config-lg-kenya.
INFO Actions:
- edit-contacts
INFO Starting action: edit-contacts…
INFO Processing CSV file: /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/contact.csv …
INFO Error: missing "documentID" column.
at getIDs (/Users/jackiemoraa/.nvm/versions/node/v16.15.0/lib/node_modules/cht-conf/src/fn/edit-contacts.js:101:10)
at /Users/jackiemoraa/.nvm/versions/node/v16.15.0/lib/node_modules/cht-conf/src/fn/edit-contacts.js:71:68
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async module.exports (/Users/jackiemoraa/.nvm/versions/node/v16.15.0/lib/node_modules/cht-conf/src/lib/main.js:210:5)
at async /Users/jackiemoraa/.nvm/versions/node/v16.15.0/lib/node_modules/cht-conf/src/bin/index.js:16:5
ERROR missing "documentID" column.
I can confirm that this is indeed the path to the csv file
But I’m not sure if we are supposed to cd into that csv folder to run the command
I used that in my testing to be sure it was getting to the right file and to simplify the command to eliminate other possible errors. Please run this test again, one directory higher, eg:
cd /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya
cht edit-contacts --url=https://<admin_user>:<password>@<instance>
If you get the same error that will confirm that it’s attempting to read the same file.
Once you’ve done that, then it suggests that cht-conf cannot read the file. Try replacing the csv with a simple example, like the two line one I used:
documentID
123
This should error because the contact with ID 123 is not found but if you get that far then that proves the CSV is being parsed correctly. To create this new CSV I’d recommend just using a simple text editor, typing the contents, and saving as contact.csv.
Hello @gareth
a) If I run it one directory higher, I will be outside the folder containing the package.json and also, another error is not finding the .csv as well
WARN No project package.json file found at /Users/jackiemoraa/Documents/Work/config-lg-forked/package.json
WARN Project has no dependency on cht-conf.
INFO Checking that https://medic:****@clone-smarthealth-ke-prod-3-13.lg-apps.com/medic/ is available...
INFO Processing config in config-lg-forked.
INFO Actions:
- edit-contacts
INFO Starting action: edit-contacts…
WARN No csv directory found at /Users/jackiemoraa/Documents/Work/config-lg-forked/csv.
INFO edit-contacts complete.
b) I have created another file and saved it as suggested.
Running that then I get the error you anticipated
INFO Checking that https://medic:****@clone-smarthealth-ke-prod-3-13.lg-apps.com/medic/ is available...
INFO Processing config in config-lg-kenya.
INFO Actions:
- edit-contacts
INFO Starting action: edit-contacts…
INFO Processing CSV file: /Users/jackiemoraa/Documents/Work/config-lg-forked/config-lg-kenya/csv/contact.csv …
INFO Downloading doc(s)...
WARN No columns specified, the script will add all the columns in the CSV!
INFO edit-contacts complete.
I can see a json file added in the json_docs folder containing all the fields for the user with the specified uuid. So this route works
What I am looking to achieve: given the a user’s uuid, I want to add a new flag to the docs to be able to restrict forms. In this way, if I have e.g. 100 UUID, it will download all the 100 user couch docs then I’ll need to edit all of them to add the flag then upload them?
I have been able to successfully do it for the one user using the upload-docs command, but I am just wondering about the scenario where I am anticipating hundreds…
Given that the simple version works I think there’s something wrong with the original CSV which means it’s not parsing properly. To automatically add your column create a CSV like…
documentID, is_in_fp_pilot
123, true
456, true
Then when you run edit-contacts you should see two JSON files with the new property added. Verify that these are correct, then you can use upload-docs to upload them to the server.
Note that the upload-docs command cannot be undone so be sure to test it on a test server before running against production.
I have implemented it with the flag while downloading and later uploading the docs, and now I can confirm that the flags have been updated correctly for 3 test uuids. Thank you for this @gareth
I am still having challenges with having it as a CSV file and I am just editing it as a text file, which also runs some errors at times. But I think now I can just work around this.
ERROR Invalid Record Length: expect 2, got 3 on line 4
Sure, we’ll run it against a dev instance and confirm before going live with it.
Thank you very much!
