CHT application not working offline

We recently upgraded to CHT v3.13 in Uganda Region 1 which covers about 1925 VHT Members distributed across 6 branches. It was however noted that, for users with poor network/slow internet, the application stops working.
These are users who have already logged in and synced their data, so the expectation is the application will work in the offline mode while the network is poor. This has been observed for approx 35 VHT Members in different branches.

What is observed: A loading screen as shown below

Environment

  • CHT Version: 3.13
  • APK version: 0.10.0
  • Android versions: 8, 9, 10
  • Android devices: pop2, pop3, spark

Are you able to share device or browser console logs? What kind of connectivity are the users experiencing? 2G/3G?

If this is happening after an upgrade, there’s a chance that views have changed and need reindexing on the client.

Hello @derick. We are currently trying to see how we can have the UG team get the logs from the devices. They have not noted the issue when logging in from the browser yet.
Is there a simple way you can suggest which will enable them to fetch these logs from their devices?

Hi @Kymoraa

Getting logs will depend on which version of cht-android the device has installed.

If using a webview version, these instructions should cover getting logs.
If using an xwalk version, I think some level of browser logs will be available through adb, these instructions should cover getting Android logs.

Additionally, if the users have some connectivity, they could have uploaded feedback documents with the eventual errros.

@derick, please correct me if other protocols are usually followed to get logs.

The version of cht-android should be visible in the About page of the device.

Some follow-up questions:

  • are other pages of the app loading
  • have any of the devices started working after being stuck in a loading screen for a while?

Thanks!

2 Likes

Thank you @diana
I have shared the instructions on fetching the logs with our UG team. I will share them with you as soon as I get them.

For the follow up questions:

  • No, no pages are loading. Only the spinner is visible
  • The devices start working once the VHT members return the phones back to HQ where the internet connection is good. But as soon as they head back, and lose connectivity, the issue re-emerges

Thanks for the update @Kymoraa

I have one suspicion: User stuck on spinner on app load on a poor connection · Issue #7492 · medic/cht-core · GitHub. Is it possible to disconnect one of these devices from the internet entirely (airplane mode?) and try to see it this changes anything?

1 Like

Hi @Kymoraa

I tried replicating your issue locally, on 3.13, simulating a very slow network, and I’m not seeing the infinite spinners in contacts. The long loading spinner occurs before the actual page is loaded, just like the issue above describes.

The issue I linked above has instructions about how to throttle network in Chrome desktop browser. Can you try to follow them, while being logged in with one of the users experiencing the loading issue and see if you can replicate it?

Thank you @diana
I am following up with the UG team to simulate the issue as well. Meanwhile, let me try it on my end too.

Hi @Kymoraa, do you have any updates on this issue. Thanks.

Hello @antony and @diana
The QA team has not been able to replicate the issue on their end. We are still following up with the stakeholders to try and simulate the conditions that lead to the issues, after which we shall be able to share the logs.

I have also not able to replicate the issue above on my end, as per the suggestions shared above. This is because, when I try to log in to production with a user they shared (internet is okay and I am able to log in the user in a dev instances), I get this error instead:

I am not sure if the issue is related though.

Hi @Kymoraa

I see a bunch of errors there, the first one seems to be caused by a browser extension. Could you please use a clean browser when trying to connect?
The second errors are server errors, further investigations should start with checking API logs on the server and seeing the source of the crashes.

Hi @diana

I have also not able to replicate the issue above on my end, as per the suggestions shared.
When I login a fresh, the behavior changes and operation stabilizes for those reported with error fetching tasks, this is the same work around that is being used in the field to get relief, but not feasible for the large numbers.

I have 2 accounts that have been logged in for 2 weeks now, and tried observing with throttled connections, but still this doesn’t seem to reproduce the offline issue.

Also the solution from User stuck on spinner on app load on a poor connection was tried out and still no change

Stakeholders (@Moses_Obulei @Aguti_Sandra ) have been able to share some Logs from the field occurrences of this issue.
Some of the Logs shared from different scenarios with some affected users from the field




Thanks a lot for sharing @raymondskkb .

It looks like PouchDb is erroring out. This error is visible in the first and third images that you shared: Database has a global failure..
The specific cause for this error is very hard to pin down. In one instance, it was caused by large attachments (Database errors on some devices when forms have large attachments · Issue #6699 · medic/cht-core · GitHub), in others by some PouchDb internal cleanup (compaction), in others because the devices ran out of storage.

A bunch of work was done related to this across many years: DOMException crash leaves users stuck at loading spinner · Issue #5045 · medic/cht-core · GitHub, Handle Database has a global failure error during replication · Issue #5348 · medic/cht-core · GitHub.
One of these issues added a modal when the database becomes corrupted. The modal looks like this:

The modal can be cancelled, and it does not force a reload (reloading will not necessarily resolve the problem).

Have your affected users seen this modal? If not, I’ll submit an issue to check which PouchDb errors are not caught. To be completely clear, the modal does not have any functionality besides warning the user that something has gone wrong, as opposed to infinite spinners and no information.

2 Likes

Hello Diana,

We are experiencing a similar Modal, It could be related to the one you have above.
But this usually comes when the the app has been offline and one tries to access it when online.

Hi @Aguti_Sandra

The modal text can be changed with translations, so given the similarities, I think it’s safe to assume you’re also encountering PouchDb errors.
Coming online will trigger replication, which will involve reads and writes from the local PouchDb database. I suggest you check browser logs to see if you can get more insight into what is happening.

Hi Diana,

Have a look at these browser logs that we have managed to replicate from the Modal text.
hope the errors generated are able to give more insight to further the investigation on the issue.

Hi @Aguti_Sandra

Thanks for sharing the image.
This confirms that:

  • PouchDb is crashing, because of an IDB error (this is a very low level crash)
  • the crash happens in Chrome desktop (is this correct?)
  • the crash happens when the user is online (connected to the internet)

I have some follow-up questions:

  • does reloading the app resolve the issue?
  • does this happen only when loading targets or does it happen on any tab? (fe: navigate to the tab and reload on that tab).
  • does throttling the internet connection to offline (from browser network settings tab) and reloading change the behavior
  • can you please share the whole browser log, from the reload up until the crash?
  • does this happen for a few select users or is it a common occurrence for all users?

Thanks!

Hi Diana,

To answer your questions.

  • the crash happens in Chrome desktop (is this correct?) - Yes that is chrome desktop

  • does reloading the app resolve the issue?
    On the browser- yes this is resolved
    On the CHT app - No Reloading the app doesnot fix this

  • does this happen only when loading targets or does it happen on any tab? (fe: navigate to the tab and reload on that tab).
    This can happen when one attempts to load any tab

  • does throttling the internet connection to offline (from browser network settings tab) and reloading change the behavior.
    Throttling internet connection and reloading the app does change the behaviour on the browser but not on the CHT app.

  • can you please share the whole browser log, from the reload up until the crash?
    (As of now, i havenot yet succeeded in replicating this crash again from the browser)

  • does this happen for a few select users or is it a common occurrence for all users?
    Yeah this is a common occurence, initially few users were affected but the number has kept growing.

Thanks for the update @Aguti_Sandra

Please keep us updated on the developments.
It would be most helpful if you could find one user that you can consistently replicate this for, and if possible collect all browser logs and share them, also making note of every action that you make from the point you log in until you see the error triggered.

Thanks!