Recommendations for testing a purge function

@Marina and I are configuring a purge function that:

  1. Purges a set of reports differently for two different roles (chw/supervisor).
  2. Purges very aggressively for one role (some reports after just a few days).
  3. Has some complex logic for how/when to purge one report type.

I unit tested the function extensively, but now I want to test it on production data to make sure that it works, and evaluate how these purge rules will affect production experiences.

I started by Replicating some production data locally. I waited for a purgelog in medic-sentinel. But when I login, I’m still seeing reports for the specific user that I would expect to be purged.

I’m wondering if there are some recommended steps to go through when testing a purge function? If I change the function, how can I restart testing completely?

In general, changing the purging configuration feels a bit scary. It feels like it will be difficult to undo if there is a bug in the function - so it would be great to have clear steps to verify before releasing - particularly when I want to iterate on the rules and experiment with multiple different rules.

1 Like

Hi!

If I change the function, how can I restart testing completely?

Purged docs are stored in separate databases, one for each set of unique roles. If you delete all the databases and re-sync your user (from 0), you’ve restarted completely.

In general, changing the purging configuration feels a bit scary.

Completely understandable. If your function would incorrectly purge a doc, changing the purge function and running purge
again will not touch the doc in medic, so in order for the devices to re-download this doc, it has to be touched manually
(so it’d be queued at the end of the changes feed) or devices would need to re-sync (from 0).

I’m wondering if there are some recommended steps to go through when testing a purge function?

I don’t think there are recommended steps. See medic-docs/configuration/purging.md at master · medic/medic-docs · GitHub, although this was written at the time when purging was done client-side.
I would recommend either cloning production data or generating mock data to resemble production data (and use medic-conf to import the data into a test instance).

2 Likes

Hi @kenn , wondering how did you end up testing in your production environment. We are also working on one of our instance and I think that information will be helpful.