Preparing for 5.0 document outlines the following information.
Deployments should only use the “Stage” button when upgrading from 4.x to 5.x in Kubernetes. Once staging completes, it is safe to upgrade through helm upgrade!
This got me confused, if this is necessary (to stage before running 5.x upgrade) or not. What happens if I take my instance down without staging and initiate 5.x upgrade? Does that upgrade path also work?
Hi @yuv - thanks for the question!
This got me confused, if this is necessary (to stage before running 5.x upgrade) or not.
As with any CHT upgrade, it is not required to stage before upgrading.
What happens if I take my instance down without staging and initiate 5.x upgrade? Does that upgrade path also work?
What happens, again as with any CHT upgrade you don’t stage, is that the upgrade will take a very long time, as it will first stage all views and then do the upgrade. So, while possible, it’s best to stage first, then upgrade.
The main issue here is that in 4.x both the “Stage” and “Upgrade” buttons are available to a Kubernetes deployment to upgrade to 5.1. So, while a Kubernetes deployment can not click “Stage” as you asked above, if a Kubernetes instance clicks the “Upgrade” button, it will do the upgrade but CHT will not have Nouveau running. CHT will be in a broken state at this point (sync and online search will be broken) and the fix will be to upgrade the helm chart and deploy it so Nouveau is installed and the CHT instance will resume working correctly.
Hi @mrjones ,
Thanks for your response. While staging on 4.22.0 from the backend, it throws an error Error triggering update and following message is in the console.
expected version 5.1.0.22454960592 does not match current version 4.22.0.18399126672
However, in /_utils at fauxton, the tasks appear to be running.
Gotcha - thanks for the update @yuv .
As you work though any bumps in the upgrade process, feel free to add some notes to the helm chart upgrade instructions which might help the next deployment trying to upgrade.
Thanks!
In our instance, multiple search_indexer ran for many hours and did no complete. So, I restarted the upgrade from the very beginning again. However, I am seeing that a search_indexer is not updated again.
| type | database | stared_on | updated_on | node | pid | status |
|---|---|---|---|---|---|---|
| search_indexer | shards/20000000-3fffffff/medic.1634011221 | Mar 11th, 8:08:47 pm an hour ago | Mar 11th, 8:09:18 pm an hour ago | couchdb@couchdb-2.xxx-prod.svc.cluster.local | 0.210907.0 | Progress: 0% 2025 Changes done. |
How do I see what’s causing an upgrade to stuck here. How does it progress ahead? I suspect restarting pods may work, but is there a way to check why it’s stuck and a way to avoid it ?
For sure restarting pods is suggested for 4.x → 5.x upgrades (effectively the restart advice on the troubleshoot page).
Do the CouchDB logs for the pod or service show anything interesting?
While staging on 4.22.0 from the backend, it throws an error
Error triggering updateand following message is in the console.
expected version 5.1.0.22454960592 does not match current version 4.22.0.18399126672
As I am working on multiple instance upgrade, this error is shown usually for larger instances where staging takes long time. But as mentioned earlier, _utils/#/activetasks will show staging tasks actually running. So, it’s advised for anyone seeing this error on console to check the active tasks on Fauxton.
As far as staging is concerned, this is advised as this reduced the overall downtime during upgrade. But ensure that you have enough space on the disk before staging.
