Offline synchronization support

I recently started an improvement issue on github and @gareth kindly suggested to also bring the topic here so that we might have an interesting discussion.

I tried to remove ambiguity from my main post according to @gareth questions, hopefully it’s clearer like this, so with the git issue template:

What feature do you want to improve?
Have a way of synchronize data offline with other devices. E.g. Health worker A going to somewhere rural (being offline) would be able to synchronize data from other offline devices (health worker B, C, D, F etc…). Health worker A then could go to the main health facility, go online and synchronize with the main server.

Describe the improvement you’d like
Offline synchronization of data between two devices. Does not have to be bidirectional flow of data.

Describe alternatives you’ve considered
Download the data locally and the Health worker willing to collect the data would aggregate the data from every device in an USB key. He would then go to the main health facility and synchronize the collected data with the server.

Additional context
There would be an area with the main server (online) and then multiple areas offline where someone (health worker A from the previous example) would go, collect the data, come back, and synchronize with the main server

1 Like

@magp18 thanks for giving this topic its own dedicated conversation here! This sounds very similar to the concept of “peer-to-peer” syncing that was briefly discussed here. I just wanted to link in that conversation to add some additional context here. Is your desired functionality similar to that “peer-to-peer” sync?

It does not necessarily need to be p2p but it would certainly be an option and solve the issue.

One way could also be having a button to download the data locally to a folder and then use the device’s bluetooth file sharing service to share the data (even if not as elegant as a smooth option with e.g. wifi direct).

Is it possible to download the data from the db whilst offline? I am looking into develop something on top of the cht-android, still looking how to get my hands on the data offline to then try to transport it

This is pretty far outside my range of experience so please consider all of this to be educated guesses + wild speculation. :slight_smile:

For indirect syncning of data between two PouchDB instances, I would start by looking at this pouchdb-replication-stream util (it has not been updated in awhile, but it might still work or at least its source code would have some good hints on how to do things). Just remember that it is not enough to just get the doc JSON. For the CHT, you would also want to sync all the attachments too.

The big unknown for me is how hard it will be to connect some custom code to the PouchDB instance running locally on the device. Perhaps it is just a matter of having a custom version of cht-android where you would include your import/export code that connects to the Pouch running in the webview within the app…?

Thanks for your input, ideas are always highly appreciated! I think the link helps actually, I will take a look and let you know if I manage to get some progress. The minimum requirement would be to dump the database, but of course having sync of data automatically would be a lot fancier.
I was first looking into doing this at the level of cht-android, but then I wouldn’t have access to the db right? Unless I could inject JS but even then :thinking:
I think it might be easier to modify cht-core but probably cooler to have it at cht-android level.

Once again I have not actually tried this, but I would think that injecting JS from cht-android to communicate with Pouch would be at least theoretically possible. Here is an example of where we actually inject some external JS code already.

Hi Thanks,

I was looking into the same thing, so I guess I would have to start at android → inject JS to get the db and dump the json → back to android stream the file to the other device.
Probably something like this right :thinking: I will look into this and let you know. If someone has some other ideas, I would love to hear them!

This is a fun thread to see being discussed! There’s some interesting problems to solve.

I was reading over the github thread where we talk about securing the data in transit by encrypting it. I realize however that if we use a publicly accessible accessible key, then anyone can send any data in to the CHT and we skip authentication. I haven’t fully thought this through, but I think both encryption and authentication likely need to both be part of the solution.

Yes indeed I agree that both will probably necessary. I haven’t thought this through either, I am still trying to download the data that is being stored locally. I have tried it from the android application as discussed with @jkuester but although we can inject JS, I can’t access the db from the android still, unless I am missing something. I have read about JS interface with android to access android elements from JS but not the other way around.
I might be wrong (not android dev here ) but I feel like it would be easier to create a button in JS access android widgets for a Toast when the download is over.

Not sure how helpful (or relevant) this actually is, but I had to satisfy my curiosity on this… :slight_smile:

I tried to see if I could load data from PouchDB into the Android context. In a really simple example I was able to load the settings doc with the following code:

I added this method to MedicAndroidJavascript as a convenient callback for displaying info returned from JS:

public void toastResult(String result) {
	Toast.makeText(parent, result, Toast.LENGTH_LONG).show();

Then, to actually get the document, I just had to run:

String script =
	"window.PouchDB('medic-user-chw')" +
		".get('settings')" +
		".then(result => medicmobile_android.toastResult(JSON.stringify(result)));";
container.evaluateJavascript(script, null);

Note that chw is my username and container is the WebView container in EmbeddedBrowserActivity.

1 Like

This is really cool and incredibly useful many thanks! Will try this out to download the data locally first (with a given user first and then trying to do it dynamically) and will keep you updated!

1 Like

Just a short update:
It works now! Got a button to fetch all the documents on the db, your line PouchDB(‘db…’) alone clarified what I was struggling to find! I am now downloading the data to a folder in the phone (still hardcoded user)
So now I will try to update it to have dynamic user injected & look into how to import it.

1 Like

Does anyone know how to find out which user is logged in to have access to the db dynamically?
I assume this is stored somewhere even offline right? I just can’t find where. I have been trying to read from cache but haven’t found anything yet, or to have the jsonclient pass the variable as we have url.getUserInfo() but didn’t work either. Also tried to access the SharedPreferences but it appears as if the username is not saved here either. Does anyone have any idea where I can grab the username from to have it in the Pouchdb()?