Hosting Total Cost of Ownership Squad

Product Squads
1

Intro

Medic is pursuing reducing the total cost of ownership (TCO) of the CHT by trying to reduce disk space. This wiki is an experiment in using Discourse to greatly increase visibility and participation outside of Medic in this effort, and future efforts we’re calling “squads”.

Goal

As research has already been concluded, the current goal of the TCO Squad is now to ship a disk reduction feature in the form of CouchDB Nouveau.

Meetings

There’s a weekly meeting on Tuesday 7.00AM Pacific/6.00pm East Africa/8.45pm Nepal - anyone interested in attending is free to join the video call -

NB 1: the first meeting on Jan 21st will start at 7.30AM Pacific/6.30pm East Africa/9.15pm Nepal - This note will be deleted after this date passes to avoid confusion

NB 2: This time is currently pinned to 7AM Pacific which in the US changes with daylight savings time - please check here for updates on when the call is! (and maybe it shouldn’t be pinned to Pacific?!)

Meeting notes and recordings of the call will be added as comments below.

Related

For more information, see:

5 Likes
2

@antony @jkuester @sugat - feel free to add any info that’s missing above!

2 Likes
3

21 Jan 2025 Call

Attending

Notes

  • review parent ticket and sub ticket status
  • delegate outstanding tickets to the two engineers
  • there’s some concern that master keeps moving and the TCO effort is taking a while to get merged - need to keep master merged back to multiple branches as we develop. Development is branch of branch off master

Follow up

  • @mrjones - move this wiki to new forum section
  • @jkuester - log new issue to update CHT Core to use new library for free-text search instead directly using Couch as it is currently
  • @mrjones reach out to stewardship about helping out tickets to speed up delivery (too late for community membership support )

Call recording

4

28 Jan 2025 Call

Attending

Notes

  • review parent ticket and sub ticket status
  • mrjones - confirmed 1 Nouveau instance will work N instances of couch nodes - in the case of CHT Multi-node couch
  • Josh not concerned about seeing a new Couch release to test Nouveau upgrade to ensure it works
  • Possibly test a CouchDB upgrade as CouchDB master now has #5336 and #5335 - thinking is that we could test upgrading from Nouveau 3.4.2 → Nouveau Master to test upgrade in CHT
  • possible happy path for k8s would be using CHT Deploy Script? then others can extrapolate from there
  • Discuss on going PR

Follow up

Call recording

5

4 Feb 2025 Call

Attending

Notes

  • review parent ticket and sub ticket status
  • discussing this comment: I was reviewing the view query code and realized that the keyed freetext values are actually emitted whole and un-split
  • how to get current branch running in Docker? (eg npm run local-images...)

Follow up

Call recording

6

11 Feb 2025 Call

Attending

Notes

  • review parent ticket and sub ticket status
  • @sugat starting on Refactor medic code
    • this will remove a couchdb endpoint that User Man. Tool and cht-conf is using - the freetext view
  • still need to figure out how run mokhtar’s code in central branch to start it in docker compose
  • let’s release CHT Datasource ASAP (eg 4.18) with the backing of existing CouchDB pre-nouveau Views
  • discuss calling REST APIs or call CHT Datasource?

Follow up

  • @mrjones - Move Meeting time to 15 min later so it’s not as close to earlier meetings
  • @jkuester - figure out how run mokhtar’s code in central branch to start it in docker compose

Call recording

1 Like
7

18 Feb 2025 Call

Attending

Notes

Follow up

  • @mrjones - Move Meeting time to 15 min later so it’s not as close to earlier meetings
  • @jkuester - figure out how run mokhtar’s code in central branch to start it in docker compose

Call recording

1 Like
8

25 Feb 2025 Call

Attending

Notes

Follow up

Call recording

9

04 March 2025 Call

Attending

Notes

Follow up

  • NA

Call recording

  • Calls are no longer being recorded - please ask questions in this forum or show up on the call! We’ll strive to take good notes. Next call will not have a “call recording” section
10

11 March 2025 Call

Attending

Notes

  • Progress updates:
    • Sugat - cht-conf PR ready for review - cannot merge PR until the cht-core release.
    • Sugat - started on refactoring shared-libs/search. Ticket.
    • Josh - couchdb-nouveau

Follow up

  • NA
13

18 March 2025 Call

Attending

Notes

  • Progress updates:
    • New Couch version dropped
    • jkuester - couchdb-nouveau PR
      • Building, unit tests passing. Integration tests passing.
      • k3s integration tests still failing - will spin off new ticket for this
      • e2e tests failing - looking into it

Follow up

  • NA
1 Like
14

25 March 2025 Call

Attending

Notes

Follow up

  • NA
15

1 Apr 2025 Call

Attending

Notes

  • Review main ticket
  • “implement CouchDB Nouveau” PR ready for review \o/
  • noting that Nouveau communication is not authenticated or encrypted (TLS)
    • we’re not going to fix this per MVP effort
    • all containers don’t use TLS right now, so this is no worse
    • if this encryption/authentication feature is important, partners can highlight this
  • discuss authentication in “refactor shared libssearch to call cht datasource freetext search APIs”
    • solution: hack the types and force TS to work as we want
    • be great to have small suite of integration tests for search library workflow

Follow up

  • NA
1 Like
16

8 Apr 2025 Call

Attending

Notes

1 Like
17

15 Apr 2025 Call

Attending

Notes

  • Review main ticket
  • Josh is very busy on all the things, but Tom’s PR and Sugat’s PR are high priorities
  • Sugat updates - making slow progress with the search lib e2e tests with WDIO
  • Tom updates - frustrated with sonar - move from 5 to 7?
  • Tom take on more issues as he rolls of prior ticket.
  • What extra docs should we add if any? Release docs will cover extra container and such, but what else?
    • architecture diagram - ticket
    • docker helper (eg list of containers) and other docker docs which list containers
    • other?
1 Like
18

22 Apr 2025 Call

Attending

Notes

  • Review main ticket
  • Josh short week this week, working on reviewing PRs to unblock others
  • Sugat was off last week, so just catching up. adding some tests to the issue he’s working on
    • WDIO tests are somewhat limited b/c there’s just the one search input text field
    • likely existing tests that we’ve already improved on, are good as is
    • will add key/value searches
    • images from this will allow us to do TCO performance testing on production data
    • Sugat to reach out to mrjones when branch has all the commits added to remove DDocs and then we can commence testing on EC2 per last time we tested
    • Branch for search lib changes - 9751-refactor-shared-libssearch-to-call-cht-datasource-freetext-search-apis
  • Tom
    • waiting for Sugat’s review on PR
    • looking at helm charts
  • mrjones
1 Like
19

29 Apr 2025 Call

Attending

Notes

  • Review main ticket
  • Josh to step aside as he’s busy with other PRs - merge without him as needed!
  • Sugat working on disk use testing
    • MoH Zanz test in progress
    • mrjones to check on MoH Zanz upgrade that Sugat started
    • mrjones to get Niger and Kailali (sp?) snapshots mounted to ec2 instance
  • tom working on helm charts
    • test passing locally with k3d \o/
    • wondering how helm upgrades will work in EKS/k8s
    • Tom reach out to Hareet/Diana as needed to see about happy paths for MoH deployments
    • how to deal with volumes (PV) and volume claims (PVC) - simplest would be nouveau be it’s own pod with it’s own data
    • how to handle nouveau volume in general? bind mount or named volume in docker? reuse couch1 volume always for both k8s and docker?
  • discuss Hareet’s comments?
1 Like
20

6 May 2025 Call

Attending

Notes

  • Review main ticket
  • Josh and Derick talking about testing current branch against 3TB MoH instance
    • how helpful is hosting TCO’s improvements for a deployment with say 30 instances that has purchased 100TB of disk for all 30? Can they actually get rid of 20TB of disk? Probably not because of spike in disk use to re-index views - which is outside of Nouveau’s per view
    • how exactly does a view upgrade work? does one view change force the entire design doc it’s in to be re-indexed? Again, this is outside Nouveau’s view
  • Tom has concerns about k8s upgrade path for our helm charts
    • upgrade button in admin won’t work: CHT Core will upgrade fine, but online users won’t have search
    • there will almost certainly be manual steps involving helm or kubectl
    • if you’re on helm chart happy path - it will just work via helm or deploy script. In app admin section upgrade GUI will not work
    • if you’re outside helm chart use using DIY k8s, the upgrade path will be the most unhappy path, needing to document a few more steps
  • testing upgrade of MoH Z from 4.5.2 to 4.18 so we can test a 4.18 upgrade Nouveau
    • it’s stuck not being able upgrade
    • decided to delete volume, recreate the volume from the snapshot, start fresh and hope it works
  • Josh still stuck on other efforts, let him know how he can unblock or unassigned as needed
21

13 May 2025 Call

Attending

Notes

22

20 May 2025 Call

Attending

Notes