While testing this feature to write the upcoming 5.3 release notes, I had some key points I wanted to share!
I started off by creating a local dev instance with docker helper. I then used the easy mode in the test data generator to flood the instance with new data.
Note - there used to be a section here about the importance of clearing your sentinel backlog. This turned out to be inaccurate, so it has been removed and an update was posted below.
I then used the /api/v2/export/reports API to download many report IDs. Specifically I picked an arbitrary start time and then converted it to seconds since the epoch. So Monday, August 24, 2026 at 2:35:15 PM UTC-07:00 DST became 1787607315000. I then looked the query format to use against the API - this forum post covers the topic well. Since I wanted the raw IDs, with out any other data, I cleansed the output with cut and sed like so:
curl -qs -g \
"https://medic:password@192-168-68-26.local-ip.medicmobile.org:10473/api/v2/export/reports?filters[search]=&filters[date][from]=0&filters[date][to]=1787255664000" | \
cut -f1 -d, | sed "s/\"//g" > test.IDs.to.archive
Be patient! I found this command can take some time to run - for me it took over an hour. Consider a very narrow query (short date period, specific reports etc) to test with before letting it run.
I wanted to slowly test, so i first cut off 10 IDs from the list: head -n10 test.IDs.to.archive > 10.ids.test.archive and ran this against my dev instance with a curl call:
curl -qs -X POST \
-H "Content-Type: text/csv" \
--data-binary @10.ids.test.archive \
https://medic:password@192-168-68-26.local-ip.medicmobile.org:10473/api/v1/archive | jq .
This immediately returned some JSON:
{
"jobs": [
{
"id": "archive:01a035b8-5477-7aa1-9d08-b3e740df9162",
"count": 10
}
]
}
As my dev instance was still busy indexing new documents sentinel only runs every 5 minutes, at first I was confused because the archive docs say:
Each archive job writes a log document to the medic-logs database with the same ID as the job.
However, I didn’t see any entries in the medic-logs database. I realized that this can take a bit, possibly many minutes for a busy instance until sentinel runs on its regular interval. It can be helpful to tail the logs of your Sentinel container. For me, I used this call:
docker logs -f cht-sentinel-1 --since 100m 2>&1 | grep -E "Archiving: processing job archive|skipped"
When you see activity here, you can then go to into Fauxton and look at the medic-logs database and look for documents that start with archive::
I also had a watchdog instance running so I could monitor the size of the medic database both for test data generate calls increasing it (see #1) and the archive feature decreasing (see #2):
Once I gained confidence in how the system worked, I uploaded file with thousands of lines to the archiving API and used the above techniques to monitor its progress. I was able to successfully archive over 100,000 reports!
!