I have a select_one question type with a lot of choices (16k plus) to select (with choice_filter of course : ). cht-conf times out while trying to validate the contact form (works on my local container but takes like 13 minutes, fails completely on my server enviroment). Is there any workaround around this.
Also does the core framework support attaching an external choices csv of xml file like ODK does using the command select_one_from_file choices.csv? I’ve tried with no joy.
Running your cht-conf commands with the --skip-validate should keep it from trying to run the validation.
Also does the core framework support attaching an external choices csv of xml file like ODK does using the command select_one_from_file choices.csv?
Unfortunately, no. This is not currently supported in the CHT.
@morlig - Let us know how it goes with @jkuester’s suggestion to skip validation.
As well, it’d fun to have a copy of the form itself it it’s appropriate to share.
Finally, let us know how the CHT performs with this form on the device itself!
The workaround allows the form to be uploaded without a hitch, but the form does not open for a while even when sync reports it’s done. But after like 5 minutes it loads the choices…
The field itself accepts a list of health facilities in the country, filtered by county.
@morlig - Thanks for getting back to us with an update!
Even with the choice_filter, I’m not too surprised that it takes so long to load 16k choices.
What version of CHT are you running? If you’re back on CHT 3.x, there’s some amazing speed improvements done in CHT 4.x - and when CHT 4.6 comes out, it will ship with an updated Enketo version which should specifically make your use case faster.
That said, assuming we see a 2x speed improvement, I suspect 2.5min is still too long for a form to load. I would suggest breaking up your workflow so that the user is never selecting from more than 100s of choices.
If you do the work to break up your workflow like this, it would be great to hear back on your results! I’m sure there’s some interesting thresholds the CHT will hit where it starts to really slow down.
Finally - If it is appropriate to share, what is the workflow you’re using the CHT for that has 16k choices?
Hi @mrjones, I am currently on version 4.5.1.
Just to clarify.
I upload the forms to the system via cht-conf (this works okay if I use --skip-validate)
The user synchronizes their device or refreshes their browser (this works fine as well)
The user attempts to load the form (with many choices). The system responds with an error message (Error loading form. Please try again or check with an administrator.) during the first 5 - 10 minutes after which it loads and can be used just fine; with a load speed of approximately 5-10 seconds, I haven’t measured this accurately. Update: I have measure the consequent load times, it’s less that two seconds!
If I update the forms again and upload, the issue repeats.
As for the workflow, the questionnaire attempts to pinpoint a health facility. There are about 16k health facilities in Kenya; I use a choices filter in the preceding fields (county and subcounty) to narrow the choices down to less than 100 select from.
I’ll share more details of this project in due course, it’s very important to the Kenyan people in the domain of outbreak preparedness 
Thanks for the details! The steps you outlined are what I’d expect in that the slowdown happens on the client specifically when they try to load the form the first time.
The error in step #3 is of concern though. Could you check the developer console to see if there’s client side Javascript errors?
As well - is there any way you could share an XLSX copy of this form?
I figured the delay in loading the form occurs after updating the form. If we move all updates to a time when users aren’t using the system (at night or weekends), they will not notice this problem.
