I’ve just updated our demo server to the branch and it worked as expected, so I suspect it’s some issue with your server. Can you upgrade to any other release, eg: 4.11.0? Can you confirm that all containers (eg: upgrade service) are running? Have you done an upgrade recently? Did you do any additional steps? Are you running on k3s, EKS, docker compose, etc?
thanks @mrjones
i have used the script to get the logs for the upgrade-service as seen below for the last 24 hours
**ubuntu@ip-172-51-41-12**:**~/cht**$ ls
**certs** **compose** **compress_and_archive_docker_logs.sh** **couchdb** **upgrade-service**
**ubuntu@ip-172-51-41-12**:**~/cht**$ curl -o compress_and_archive_docker_logs.sh https://raw.githubusercontent.com/medic/cht-core/master/scripts/compress_and_archive_docker_logs.sh
chmod +x compress_and_archive_docker_logs.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1486 100 1486 0 0 7508 0 --:--:-- --:--:-- --:--:-- 7543
**ubuntu@ip-172-51-41-12**:**~/cht**$ ./compress_and_archive_docker_logs.sh
Wait while the script gathers stats and logs about the CHT containers.
Gathering past 24 hours of logs.
Be patient, this might take a moment...
Done!
/home/ubuntu/.medic/support_logs/cht-docker-logs-2024-10-18T14.34.23+00.00.tar.gz
NOTE: Please remove the file when done as it may contain PII/PHI.
**ubuntu@ip-172-51-41-12**:**~/cht**$ cd /home/ubuntu/.medic/support_logs
**ubuntu@ip-172-51-41-12**:**~/.medic/support_logs**$ tar -xzvf cht-docker-logs-2024-10-18T14.34.23+00.00.tar.gz
./compose-api-1.log
./compose-couchdb-1.log
./compose-haproxy-1.log
./compose-healthcheck-1.log
./compose-nginx-1.log
./compose-sentinel-1.log
./docker_ps.log
./docker_stats.log
./upgrade-service-cht-upgrade-service-1.log
**ubuntu@ip-172-51-41-12**:**~/.medic/support_logs**$ cat new_project-dir-cht-upgrade-service-1.log
cat: new_project-dir-cht-upgrade-service-1.log: No such file or directory
**ubuntu@ip-172-51-41-12**:**~/.medic/support_logs**$ cat upgrade-service-cht-upgrade-service-1.log
Listening on port 5008
Listening on port 5008
Listening on port 5008
**ubuntu@ip-172-51-41-12**:**~/.medic/support_logs**$ tail upgrade-service-cht-upgrade-service-1.log
Listening on port 5008
Listening on port 5008
Listening on port 5008
**ubuntu@ip-172-51-41-12**:**~/.medic/support_logs**$
thanks @gareth tried to another release but still the same issue App Management - 18 October 2024 | Loom
yes all the containers are running
ubuntu@ip-172-51-41-12:~/cht$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
71aba0f123e2 nginx:latest "/docker-entrypoint.…" 2 hours ago Up 2 hours 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp compose-nginx-1
1fd41e5d1ef2 public.ecr.aws/medic/cht-api:4.10.0 "/bin/bash /service/…" 2 hours ago Up 2 hours 5988/tcp compose-api-1
9b11563c6651 public.ecr.aws/medic/cht-sentinel:4.10.0 "/bin/bash /service/…" 2 hours ago Up 2 hours compose-sentinel-1
b847b25042fc public.ecr.aws/medic/cht-couchdb:4.10.0 "tini -- /docker-ent…" 2 hours ago Up 2 hours 4369/tcp, 5984/tcp, 9100/tcp compose-couchdb-1
2f51bb0d8785 public.ecr.aws/medic/cht-haproxy:4.10.0 "/entrypoint.sh" 2 hours ago Up 2 hours 5984/tcp compose-haproxy-1
275a9ea785a2 public.ecr.aws/medic/cht-haproxy-healthcheck:4.10.0 "/bin/sh -c \"/app/ch…" 2 hours ago Up 2 hours compose-healthcheck-1
c7814b2f3f7c public.ecr.aws/s5s3h4s7/cht-upgrade-service:latest "node /app/src/index…" 4 hours ago Up 3 hours upgrade-service-cht-upgrade-service-1
No, this is our first time doing an upgrade on 4.x instance
We are running docker-compose on AWS ubuntu linux
I have put down the containers. and deleted the volumes , restarted afresh but still persists …
Am thinking of doing a fresh new install again, WDYT?
This is great - thanks!
I was thinking you could upload the cht-docker-logs-2024-10-18T14.34.23+00.00.tar.gz somewhere so we could download it and see all of the logs without having to ask you to run another command to get different logs over and over. If this has any production data with PII/PHI, please do not post the URL here - feel free to email it to me at mrjones@medic.org.
Hi @cliff
Just checking in to see how the launch went. Did this patch solve the RapidPro issue?
Hello @gareth
Sorry for the delayed response , the delay has been mainly on the TLS certs that we are using.
The upgrade was breaking the upgrade service which seemed not to work with certs that are not self signed, so we switched the instance to self signed certs as per the initial proxy configurations which worked .
I have tested the the patch branch and it fixes the sms duplication issue with message state being updated accordingly in the DB …
@gareth @diana However while testing i have noticed that the reports are not created in the UI upon submitting the forms ie during enrollment which i think could be the reason why the outbound is not working when enrolling the patient in the system to send them a message …
The strange bit is that the doc is created in the DB ie as seen below
{
"_id": "226262f7-f820-4667-bd2e-b71ea11e6f90",
"_rev": "1-628988ae869e51ee746f056d71ad2405",
"parent": {
"_id": "ba13a159-517b-4cfa-86f7-f03ec3a292e4"
},
"type": "person",
"date_reg": "2024-10-25",
"name": "wewew",
"name_f": "hghg",
"name1": "dss",
"full_name": "",
"aka": "",
"docket": "",
"tsis": "3482",
"address": "ccxcxcx",
"address1": "",
"phone": "+18768249050",
"type1": "",
"network1": "",
"date_of_birth": "2000-02-03",
"at": "male",
"genda": "",
"meta": {
"created_by": "medic",
"created_by_person_uuid": "",
"created_by_place_uuid": ""
},
"reported_date": 1729852022109,
"form_version": {
"time": 1729842575577,
"sha256": "701fcbfc61f465727bb8077d866109ec329cc64312e161c96513fffb4cb0e4ea"
}
}
and the corresponding outbound config
"outbound": {
"patient triggers enroll workflow in Textit": {
"relevant_to": "doc.type === 'person' && doc.tsis",
"destination": {
"base_url": "https://textit.com",
"auth": {
"type": "header",
"name": "Authorization",
"value_key": "rapidpro.app"
},
"path": "/api/v2/flow_starts.json"
},
"mapping": {
"flow": {
"expr": "'580567d6-6662-4c0e-9ea3-133798eba8ef'"
},
"urns": {
"expr": "['tel:' + doc.phone]"
}
}
}
},
the sentinel logs not giving much
ubuntu@ip-<id>:~$ sudo docker logs compose-sentinel-1 --tail=100
2024-10-25T09:57:08.938 INFO: Task dueTasks completed
2024-10-25T09:57:08.943 INFO: Task outbound completed
2024-10-25T09:57:08.950 INFO: Background cleanup batch: 541 -> 544 (3)
2024-10-25T09:57:08.962 INFO: Task backgroundCleanup completed
2024-10-25T10:02:08.919 DEBUG: Initiating all tasks
2024-10-25T10:02:08.919 INFO: Task dueTasks started
2024-10-25T10:02:08.920 INFO: Task reminders started
2024-10-25T10:02:08.920 WARN: Reminder configuration invalid: {
form: '',
text_expression: '',
cron: '',
mute_after_form_for: '',
message: ''
}
2024-10-25T10:02:08.920 INFO: Task replications started
2024-10-25T10:02:08.920 INFO: Task outbound started
2024-10-25T10:02:08.920 INFO: Task purging started
2024-10-25T10:02:08.920 INFO: Task transitionsDisabledReminder started
2024-10-25T10:02:08.920 ERROR: Transitions are disabled. Unknown transition "update_sent_forms"
2024-10-25T10:02:08.920 INFO: Task backgroundCleanup started
2024-10-25T10:02:08.921 INFO: Task reminders completed
2024-10-25T10:02:08.921 INFO: Task replications completed
2024-10-25T10:02:08.921 INFO: Task purging completed
2024-10-25T10:02:08.921 INFO: Task transitionsDisabledReminder completed
2024-10-25T10:02:08.943 INFO: Task outbound completed
2024-10-25T10:02:08.947 INFO: Task dueTasks completed
2024-10-25T10:02:08.950 INFO: Background cleanup batch: 544 -> 550 (4)
2024-10-25T10:02:09.041 INFO: Task backgroundCleanup completed
2024-10-25T10:07:08.922 DEBUG: Initiating all tasks
2024-10-25T10:07:08.922 INFO: Task dueTasks started
2024-10-25T10:07:08.923 INFO: Task reminders started
2024-10-25T10:07:08.923 WARN: Reminder configuration invalid: {
form: '',
text_expression: '',
cron: '',
mute_after_form_for: '',
message: ''
}
There is an error in the sentinel logs which explains that transitions are disabled because of an invalid configuration. update_sent_forms was deprecated in 3.7 and removed in 4.0 which is why you’re getting this error. If you update your configuration so you’re no longer relying on this transition then you can get past this error.
Thanks a lot for sharing, it helps me