Outbound Push Configs

Technical Support Development
,
1

I have been configuring Outbound Push to send data to DHIS2.
Progress so far, I’m able to correctly map the data and run the configs.

However, the data won’t reflect on the test DHIS2 Site yet I have a “Pushed…” log message on the medic-sentinel server.
I expected the request to be successful.

I have confirmed the CU/Organisation unit code is correct and well-mapped on DHIS2.

Question.
Given the server message from CHT’s medic-sentinel db…
Pushed 2aada4ee1924843b9e4a3c79481d0afb to khis_config

Where can I get the response messages from the destination server ? Was it a 200k success, or a particular error such as Error, Org Unit Not Found, or a Warning?

I was expecting an error or warning response from DHIS2 if the data was not successfully written on DHIS2.
Any suggestions will be highly appreciated.

1 Like
2

Are you able to manually curl the payload to your DHIS2 instance? What result do you get?

2 Likes
3

Yes, @derick , pushing the same CU data using curl populates it on the test DHIS site. Here’s the response.

{"responseType":"ImportSummary","status":"SUCCESS","importOptions":{"idSchemes":{},"dryRun":false,"async":false,"importStrategy":"CREATE_AND_UPDATE","mergeMode":"REPLACE","reportMode":"FULL","skipExistingCheck":false,"sharing":false,"skipNotifications":false,"skipAudit":false,"datasetAllowsPeriods":false,"strictPeriods":false,"strictDataElements":false,"strictCategoryOptionCombos":false,"strictAttributeOptionCombos":false,"strictOrganisationUnits":false,"requireCategoryOptionCombo":false,"requireAttributeOptionCombo":false,"skipPatternValidation":false,"ignoreEmptyCollection":false,"force":false,"firstRowIsHeader":true,"skipLastUpdated":false,"mergeDataValues":false,"skipCache":false},"description":"Import process completed successfully","importCount":{"imported":1,"updated":35,"ignored":0,"deleted":7},"conflicts":[],"dataSetComplete":"2023-05-08"}**%**

4

Do you see the uploaded data in the appropriate dataset in DHIS2?

5

Yes, the data is reflected on DHIS2.

6

@sheila_abby I think the next thing to do here is turn up the log level for Sentinel. If you set the NODE_ENV environment variable for your medic-os container to development and restart Sentinel, then it should start logging the contents of both the outbound request and the reply to the Sentinel logs. Hopefully that can give us more information about what is going on here!

1 Like
7

Thanks @jkuester for the suggestion, we shall be updating the NODE_ENV. I will share my feedback once that’s done.

1 Like
8

@jkuester the NODE_ENV environment was updated.
I’m able to see some response msgs from the external service when the outbound configs code has an incorrect destination URL.

**[**2023-05-12** 12:32:38] **2023-05-12** 12:32:38 ERROR: Response body: {"httpStatus":"Internal Server Error","httpStatusCode":500,"status":"ERROR","message":"org.springframework.validation.BeanPropertyBindingResult: 1 errors\nField error in object 'importOptions' on field 'orgUnitIdScheme': rejected value [COD]; codes [methodInvocation.importOptions.orgUnitIdScheme,methodInvocation.orgUnitIdScheme,methodInvocation.java.lang.String,methodInvocation]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [importOptions.orgUnitIdScheme,orgUnitIdScheme]; arguments []; default message [orgUnitIdScheme]]; default message [Property 'orgUnitIdScheme' threw exception; nested exception is java.lang.IllegalArgumentException: No enum constant org.hisp.dhis.common.IdentifiableProperty.COD]"}**

However, the outbound pushes marked as successful on CHT and NOT written data on DHIS2 DO NOT return responses looking at the medic-sentinel logs.

Looks like the core’s outbound feature code does not log 2xx status code server responses. I’m looking at the logSendError function definition here - cht-core/shared-libs/outbound/src/outbound.js at master · medic/cht-core · GitHub

Apparently, errors, and warnings encountered writing data on DHIS2 are returned as 200 ok responses - this is the tricky bit.
CHT might mark records as Pushed because the server has a 200 response but could be a 200 error response or a 200 warning message and data failed to reflect.

9

Apparently, errors, and warnings encountered writing data on DHIS2 are returned as 200 ok responses

That is super frustrating. :confused: Still, though, with the debug enabled, I would expect to see the response being logged for all outbound messages, even the 200 ones.

I’m able to see some response msgs from the external service when the outbound configs code has an incorrect destination URL.

When you say “incorrect destination URL” do you just mean when you are pointing to the wrong DHIS2 instance? I am having trouble understanding how a problem with the destination URL config could cause they error you have shown since that 500 error is apparently coming from a DHIS2 instance…

10

When you say “incorrect destination URL” do you just mean when you are pointing to the wrong DHIS2 instance?
@jkuester
Yes, the correct one should be KHIS Aggregate -updated the configs to push to /api/dataValueSets?orgUnitIdScheme=COD (missing the E in CODE).
The 500 is from DHIS - the base URL is pointing to the dhis instance but the path is incorrect.

11

The correct path is /api/dataValueSets?orgUnitIdScheme=CODE but using the incorrect /api/dataValueSets?orgUnitIdScheme=COD to trigger the CHT server to return a non-200 response since 2xx responses from DHIS are not logged by CHT outbound code.

12

So, do you see anything in the Sentinel logs that matches About to send outbound request or result from outbound request? Just trying to understand if the debug log level is actually getting picked up by Sentinel or not… We really need to have a look at one of the “successful” outbound request/response pairs to understand what is going on here. (At that should get logged out when NODE_ENV = development!)

13

@jkuester the sentinel server is picking the debug log level update, see…

[**2023-05-16** 08:22:00] **2023-05-16** 08:22:00 ERROR: Failed to push 4a74da93d70eabf3c54068c15e00171a to khis_config, server responsed with 502

[**2023-05-16** 08:22:00] **2023-05-16** 08:22:00 ERROR: Response body: "<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n<hr><center>nginx/1.18.0 (Ubuntu)</center>\r\n</body>\r\n</html>\r\n"

[**2023-05-16** 08:22:00] **2023-05-16** 08:22:00 INFO: Task outbound completed

However, the DHIS2 test site has been down. I will search for Sentinel logs that match About to send outbound request or result from outbound request once the site is up and running.
Thank you.

1 Like
14

@jkuester
The test DHIS Site is back online.
Looking at the medic-sentinel logs, I can not find any log output that matches About to send outbound request or result from outbound request.

15

Hmm, so I think that means debug logging is not enabled for Sentinel.

Can you share the steps you took when setting NODE_ENV=development?

16

Hi @jkuester

To summarize the steps I took to set the NODE_ENV = development:

  1. Updated the Docker Compose file to include the NODE_ENV environment variable for the medic-os container:
services:
  medic-os:
    container_name: medic-os
    image: medicmobile/medic-os:cht-3.9.0-rc.2
    volumes:
      - /srv:/srv
    ports:
      - "${CHT_HTTP:-80}:80"
      - "${CHT_HTTPS:-443}:443"
    working_dir: /srv
    depends_on:
      - haproxy
    networks:
      - medic-net
    environment:
      - DOCKER_NETWORK_NAME=haproxy
      - DOCKER_COUCHDB_ADMIN_PASSWORD=${DOCKER_COUCHDB_ADMIN_PASSWORD}
      - NODE_ENV=development

  1. Edited the .env file inside the medic-os container to set NODE_ENV=development

  2. Restarted medic-sentinel

  3. Restarted the medic-os container

The output of docker inspect medic-os confirms that the NODE_ENV variable is set to “development”

"Env": [
                "NODE_ENV=development",
                "DOCKER_NETWORK_NAME=haproxy",
                "DOCKER_COUCHDB_ADMIN_PASSWORD=“”,
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "DEBIAN_FRONTEND=noninteractive"
            ]

I also verified the NODE_ENV status in the medic-os container using echo $NODE_ENV and the env command. In both cases, it displayed “development”, confirming that the NODE_ENV variable was successfully set.

root@894ee904fade:/srv# echo $NODE_ENV
development
root@894ee904fade:/srv# env
HOSTNAME=
TERM=xterm
DOCKER_NETWORK_NAME=haproxy
PWD=/srv
NODE_ENV=development
SHLVL=1
HOME=/root
DEBIAN_FRONTEND=noninteractive
LESSOPEN=| /usr/bin/lesspipe %s

I’m currently facing difficulties in understanding the missing or incorrect configuration that is preventing the log level for Sentinel from being enabled.

Many thanks

2 Likes
17

Okay, sorry @Steve_Roy and @sheila_abby! I had some incomplete information here about how medic-os is actually doing things under the hood. Apparently the run script for Sentinel actually hardcodes the value for NODE_ENV when starting Sentinel :man_facepalming: which is why it did not pick up your changes to the system envars…

I double-checked with SRE and confirmed that the best way in medic-os to turn on debug logging for Sentinel is just to edit that script in your container (at /srv/scripts/medic-sentinel/run/medic-sentinel) and change the hardcoded value to development. I testing this locally and was able to successfully see debug messages being logged for the Outbound push flow. So, I am confident this should get us the info we are looking for! Sorry again for the confusion here!

2 Likes
18

Hello @jkuester, Apologies for the late response. I have been away on leave…this is well noted, no worries!
I will be touching base with @Steve_Roy to find out if the update has been made.
Thanks a lot for the support!

19

@jkuester @Steve_Roy
I am now able to see the sentinel debug logs alongside the successful cht “Pushed…” messages,
see attached…

[**2023-05-25** 08:52:50] **2023-05-25** 08:52:50 DEBUG: About to send outbound request

[**2023-05-25** 08:52:50] **2023-05-25** 08:52:50 DEBUG: {

[**2023-05-25** 08:52:50] "url": "https://test.hiskenya.org/api/dataValueSets?orgUnitIdScheme=CODE",

[**2023-05-25** 08:52:50] "body": {

[**2023-05-25** 08:52:50] "orgUnit": "600253",

[**2023-05-25** 08:52:50] "dataSet": "ovtKPo15xAg",

[**2023-05-25** 08:52:50] "completeDate": "**2023-05-25**",

[**2023-05-25** 08:52:50] "period": "202304",

[**2023-05-25** 08:52:50] "dataValues[0]": {

[**2023-05-25** 08:52:50] "value": "34",

[**2023-05-25** 08:52:50] "dataElement": "V9SqzZHKoP7",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[1]": {

[**2023-05-25** 08:52:50] "value": "671",

[**2023-05-25** 08:52:50] "dataElement": "rdcZ1ni3jrD",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[2]": {

[**2023-05-25** 08:52:50] "value": "7",

[**2023-05-25** 08:52:50] "dataElement": "ZAsCMIJLkvj",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[3]": {

[**2023-05-25** 08:52:50] "value": "67",

[**2023-05-25** 08:52:50] "dataElement": "RXgNfdYSJPh",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[4]": {

[**2023-05-25** 08:52:50] "value": "1",

[**2023-05-25** 08:52:50] "dataElement": "NjcDm9nopWu",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[5]": {

[**2023-05-25** 08:52:50] "value": "0",

[**2023-05-25** 08:52:50] "dataElement": "ylvw7Y1OB1H",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[6]": {

[**2023-05-25** 08:52:50] "value": "2",

[**2023-05-25** 08:52:50] "dataElement": "aod6Gz2QUg2",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[7]": {

[**2023-05-25** 08:52:50] "value": "34",

[**2023-05-25** 08:52:50] "dataElement": "eUakxWsxkcq",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[8]": {

[**2023-05-25** 08:52:50] "value": "0",

[**2023-05-25** 08:52:50] "dataElement": "ybfaTwL9yBA",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[9]": {

[**2023-05-25** 08:52:50] "value": "0",

[**2023-05-25** 08:52:50] "dataElement": "X9d8oaBHDqx",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

[**2023-05-25** 08:52:50] "dataValues[10]": {

[**2023-05-25** 08:52:50] "value": "0",

[**2023-05-25** 08:52:50] "dataElement": "iJpuwr4Hqj1",

[**2023-05-25** 08:52:50] "categoryOptionCombo": "NhSoXUMPK2K "

[**2023-05-25** 08:52:50] },

This is just a snapshot of it, trying to make sense of these logs. Could be I need to relook at how am mapping the data to the payload format.

With this, am positive we shall find out why the data is failing to reflect on the external site.
Many thanks.

1 Like