Storage pressure indicator

Development
, ,
1

Proposal: Storage Pressure Indicator for Offline Users

Would there be interest in including a storage pressure indicator for offline users? This could be especially useful for devices operating in kiosk mode, where users don’t have direct access to system storage information.

While troubleshooting replication issues and unexpected browser crashes on lower-end devices, we developed a simple “storage pressure bookmark” to better understand what was happening. However, this approach wasn’t user-friendly.

To make it more accessible, we built an in-app display that allows users to see, at a glance, whether storage limitations might be causing performance issues or if purging isn’t working as intended.

image
image720×287 38.8 KB

How It Works

We integrated this functionality into the header component by periodically retrieving storage estimates using the navigator.storage.estimate() API:

storageInfo: string = 'Calculating...';
storageUsagePercentage: number = 0;
intervalId: any;
isUpdating: boolean = false;

...

ngOnInit(): void {
    ...
    this.updateStorageInfo();
    this.intervalId = setInterval(() => this.updateStorageInfo(), 30000);
}

ngOnDestroy() {
    ...
    if (this.intervalId) {
        clearInterval(this.intervalId);
    }
}

...

async updateStorageInfo(): Promise<void> {
    if (this.isUpdating) return;
    
    this.isUpdating = true;
    try {
        const estimate = await navigator.storage.estimate();
        const availableGB = ((estimate.quota! - estimate.usage!) / 1024 / 1024 / 1024).toFixed(2);
        this.storageUsagePercentage = (estimate.usage! / estimate.quota!) * 100;
        this.storageInfo = `Available Space: ${availableGB} GB`;
    } catch (error) {
        this.storageInfo = 'Error retrieving storage information.';
        console.error('Storage estimate failed:', error);
    } finally {
        this.isUpdating = false;
    }
}

Why Is This Important?

While a device may appear to have sufficient storage, what the browser allocates to the app is not the same as total device storage. This distinction might be important for web apps running in offline mode, particularly when using PouchDB for local storage and replication.

In kiosk environments, where users can’t manually check storage, this indicator could be a useful troubleshooting tool for explaining slow performance, failed replications, or unexpected app terminations.

Browser Support & Considerations

The navigator.storage.estimate() API is widely supported, but there are some differences between browsers:

Browser Minimum Version
Chrome 61
Firefox 76
Safari 16 (macOS Ventura, iOS 16)

:link: Full Browser Compatibility Table

Accounting for Browser Differences

This API, as far as I know, is designed to handle storage quota variations per browser, but it only works in secure contexts (HTTPS or localhost). If the API is unsupported, perhaps there’s a fallback that we could explore?

Would This Be Valuable for CHT?

Would other teams find this useful? Should we refine the UI/UX further?

I’m still coming to grips with browser APIs and storage, so please let me know if I’ve missed anything.

Additional info:
Storage requirement on devices for CHT - Technical Support - Community Health Toolkit
Explore and document limitations using WebView version · Issue #170 · medic/cht-android
Origin - HTTP | MDN

10 Likes
2

I LOVE this!
Could you please link to the repo where this code is shared?

1 Like
3

Awesome :grin:! The code can be found here:
fix: storage pressure indicator · ChinHairSaintClair/cht-core@b6e7525.

In the repo, the solution is only implemented in the new sidebar-menu.

image
image467×332 11.7 KB

2 Likes
4

Good progress. I really appreciate it.

1 Like
5

This is brilliant @Anro !

1 Like
6

We’ve created a storage-info.service, which is now used by both the old nav header.component and sidebar-menu.component:

image
image260×166 5.5 KB

Code has been pushed to the repo :slightly_smiling_face: .

2 Likes
7

Thanks, this is a very useful feature. I believe it will help our users identify and understand the issue, which in turn can facilitate discussions with their supervisors.
A quick question though - Can this user-level information be captured and viewed on the supervisor-level dashboard?

1 Like
8

Awesome. Awesome. Since the first line trouble shooting/responders and the end users are non-techy, the problem could be multifaceted and are clueless on what it the problem. Giving them this information at an ease to access place is great clue to accelerate troubleshooting.

Could we also have a connectivity status indicator (internet & network connectivity) as a cue to users for syncing.

1 Like
9

Great proposal! I think this is really important!

My feedback is that – even though it is very important, having it as the first thing on the hamburger dropdown (even above sync status and “sync now”) seems a bit too prominent from a UI perspective.

Could I suggest something like:

  • Move the “progress bar showing storage pressure” (currently in hamburger dropdown) into the “about page”. (eg. "Available Space 320 GB)
  • Add an “alert” in the hamburger dropdown. Something like “!! Insufficient Disk Space” which fires only if there is <500 MB (?) of space available. Clicking it takes you to a simple guide on simple steps to fix it.

Cant wait to see it live!

2 Likes
10

In addition to this, there is telemetry on device storage that is recorded daily.
It is possible to query telemetry documents in rdbms to graph users that are close to storage limits.

11
  • Move the “progress bar showing storage pressure” (currently in hamburger dropdown) into the “about page”. (eg. "Available Space 320 GB)
  • Add an “alert” in the hamburger dropdown. Something like “!! Insufficient Disk Space” which fires only if there is <500 MB (?) of space available. Clicking it takes you to a simple guide on simple steps to fix it.

I love this suggestion! I think it offers the same level of ease of access initially intended by Anro, but puts the data right next to a lot of other helpful info that’s on the About page. Here’s a screenshot from my test instance of the current About page:

image
image436×1776 63.7 KB

In addition to this, there is telemetry on device storage that is recorded daily.
It is possible to query telemetry documents in rdbms to graph users that are close to storage limits.

Yes! I was thinking of this too. Looking at that doc, we get the two fields of deviceInfo.storage.free and deviceInfo.storage.total. Given these are, as you said, pushed up daily in telemetry reports, I was reminded that couch2pg has an materialized view in place to get user statistics on this. So when you “query telemetry documents in rdbms” with a query like this:

SELECT 
  DISTINCT ON (user_name) user_name,
  storage_free::bigint / 1000000 as free_storage_mb
FROM useview_telemetry_devices 
WHERE storage_free::bigint / 1000000 <= 250
ORDER BY
  user_name ASC,
  period_start DESC ;

your tabular output is thus:

 user_name  | free_storage_mb 
------------+-----------------
 mrjones    |              71
 diana      |             233
 kenn       |              59
 anro       |             157

Further, there’s another layer of analysis that was done with the Data Ingest project. This allows you to easily hook all these telemetry metrics, including end users’ disk usage, in your Watchdog instance.

@elijah did a ton of work on the Data Ingest effort - feel free to add any relevant comments if you have a spare sec!

Update! I just remembered there’s a an API at /api/v2/export/user-devices that returns software versions (APK, Android, Browser, etc) for each user. It would be easy to extend this to include storage.free and storage.total. @Jane_Katanu , @nitin @Anro - would having the storage information available to deployment administrators be helpful? Or is the original feature in this thread making the data available at the CHW device level of more interest? (Or they’re both interesting! \o/)

1 Like
12

@mrjones, both, but I would prioritize having it on the CHW phone as most of the first line troubleshooting attempts are done at that level by the users themselves and their immediate supervisors. For this level of users, the visual indicator bar which may/may not be colour coded to provide at a glance actionable information/interpretation of the storage space will be very useful. Alternatively, we can consider saying 200 out of 520GB used, instead of just sayin 200GB used.

2 Likes
13

Thanks for the data point about what would be most helpful @Jane_Katanu !

I’ll let this community effort move forward, but @Anro please feel emboldened to submit a PR with this feature - the community sounds like they’re very interested!

For the second feature of an easy way for a deployment administrator to get a list of users who might be running low on storage space, I’ve opened the “Add user storage usage to user-devices API” ticket.

1 Like
14

This feature has now been released in 4.20.0! :rocket:

3 Likes
15

@Anro, considering the point that:

“While a device may appear to have sufficient storage, what the browser allocates to the app is not the same as total device storage.”

Wouldn’t it be more useful to show the amount of storage actually allocated to the app by the browser, instead of just the total device storage? I can imagine cases where a user sees enough free space on the device, but the app is slow or crashes. This could be related to browser-allocated storage limits.

Also, in such cases, what steps can a user take to diagnose or resolve the issue?

1 Like
16

I may be misunderstanding things here, but I think you are looking at @Anro’s comment backwards. The implementation of the storage pressure indicator is using the navigator.storage.estimate data reported by the browser/webview. (calculating the difference between the quota and the usage). I think this means that it is tuned to the “storage actually allocated to the app” and NOT to the “total device storage”. :+1:

If the storage pressure indicator shows that the app is out of space I think it could mean one of two things:

  1. The device, itself, is low on storage.
  2. The device has plenty of free storage, but the CHT app has used all the storage allocated to it.

I think that case #1 is the most typical and any user seeing a full storage pressure indicator should take steps to clean up any unnecessary data on their device (remove unnecessary apps, delete old pictures/videos, etc).

For #2, AFIK the storage quota rules around Android webviews are somewhat opaque (and might vary between Android versions and devices). But if a user confirms their device has plenty of free storage, but the indicator is still showing as full, then it is probably time to consider leveraging some of the normal ways of limiting user data in the CHT (replication depth, purging, etc).

1 Like
17

Well said, @jkuester.

After playing around with this feature - especially with our OTL users replicating a lot (over 80k) of records - I found that it was lacking in one of the areas where it could be most useful: during bootstrapping, i.e., during replication.

As you pointed out in item 2, while we were debugging an OTL replication failure, we discovered that the affected user had logged in with multiple online users on the same device and in the same browser.

This meant that while the device had plenty of total storage, the space allocated to that OTL user was insufficient to complete replication (55k / 88k). And the only feedback we saw was a retry button - with no explanation as to why.

We haven’t had time to upstream this yet, but we’ve crudely implemented some additional messaging during the initial replication phase to help track available storage space.

We tweaked the following files:

bootstrap/initial-replication.js:

const downloadDocs = async (remoteDb, localDb) => {
  setUiStatus('FETCH_INFO', { 
    count: remoteDocCount - docIdsRevs.length, 
    total: remoteDocCount, 
    storage: await utils.getAvailableStorage()
  });
  do {
    await getDocsBatch(remoteDb, localDb);
    setUiStatus('FETCH_INFO', { 
      count: remoteDocCount - docIdsRevs.length, 
      total: remoteDocCount,
      storage: await utils.getAvailableStorage()
    });
  } while (docIdsRevs.length > 0);
};

bootstrap/translator,js:

const storageString = (storage) => storage ? `${storage.space || 0} ${storage.unit || 0}` : 'Unavailable';
...
en: {
    FETCH_INFO: ({ count, total, storage }) => `Fetching info (${count} of ${total} docs )…` + 
      `\n Available storage space: ${storageString(storage)}`,

bootstrap/utils.js:

const getAvailableStorage = async() => {
  const estimate = await navigator.storage.estimate();
  if (typeof estimate.quota !== 'number' || typeof estimate.usage !== 'number') {
    return;
  }

  const bytes = estimate.quota - estimate.usage;
  const mb = bytes / Math.pow(1024, 2);
  const gb = bytes / Math.pow(1024, 3);

  let space;
  let unit;

  if (gb >= 1) {
    space = gb.toFixed(2);
    unit = 'gb';
  } else if (mb >= 1) {
    space = mb.toFixed(2);
    unit = 'mb';
  } else {
    space = bytes;
    unit = 'bytes';
  }
  return { space, unit };
}
1 Like