I am not sure where to share this but wanted to make a post on the impact of the recently released 4.6 on two of our CHT running algorithms!
The categories we benchmarked are :
Loading screen : How long does it take to open the form
Next page: How long does it take to go to the next page of the questionnaire
Submit: How long does it take to submit the form
Tasks: How long does it take to load the task
Asymptomatic consultation: How long does it take to go through the consultation without any finding
Symptomatic consultation : How long does it take to go through a consultation where there was a finding. Note: Here the Algorithms were different so not the same finding was taken
I had three values, per category per algorithm and I plotted the fastest since I might be slower because of different reasons in one run but I can’t be faster than what’s possible. But here are all the values so that you can have all of it if interested:
Bottom line is, it improved by a lot, with loading screen time being cut by half! Even the “next page” seemed to be faster, being almost immediately in the 4.6 which made it hard to time it.
4.5:
Alg 1
Loading screen
~7.44 s
~8 s
~7.66 s
Next page
0.8 s
0.7 s
0.6 s
Submit
3.4 s
3.55 s
3.3 s
Tasks:
5.5 s
5.6 s
5.4 s
asymptomatic consultation
58.61 s
58.45 s
56.26 s
symptomatic consultation
1:07.71 min
1:09.20 min
1:07.10 min
Alg 2
Loading screen
~7.55 s
~7.21 s
~6.93 s
Next page
~0.66 s
~1 s
~1.1 s
Submit
-2.31 s
-2.21 s
-2.3 s
Tasks:
-4.3 s
-4.06 s
-4.34 s
asymptomatic consultation
-1:07.1 m
-57.80 s
-57.90 s
symptomatic consultation
-1:18 min
-1:13.9 min
-1:12.43 min
@magp18 - wow! Thanks so much for taking the time to share all these results, that’s really great to hear! This echos our earlier testing efforts we made with a Feature Release of the CHT. Very happy to see some real world results around this - thanks again!
Thanks for sharing @magp18 ! I’d love to know more about how you measured these, as we are currently working on improvements to telemetry that will give us a more comprehensive look at client performance across the entire CHT (the video hasn’t dropped yet, but we presented about APDEX in the CHT on the March 2024 CHT Round-up call).
You mentioned that you “plotted the fastest”… so I assume you are looking at telemetry, and specifically the min property, but I wanted to confirm a few things. Would you mind sharing some more details on these specifically ?
Loading screen : How long does it take to open the form
I assume you are using → enketo:contacts:<form>:add:render
Submit: How long does it take to submit the form
I assume you are using → enketo:contacts:<form>:add:submit
Next page: How long does it take to go to the next page of the questionnaire
Not sure on this one…
Tasks: How long does it take to load the task
Are you looking at enketo:tasks:<form>:add:render? Or something else?
The telemetry system has been recording Enketo (ie form specific) related telemetry data since 3.4.0. So you can already know the fastest (min) and slowest (max) times for things like loading a form (render) or submitting a form (save), but you didn’t really have visibility into any of the measurements between the min and max.
The apdex work in 4.7.0 will give you another level of granularity, disaggregating those times (in seconds) based on whether they were…
?