User-Agent header in rapidpro integration code

Hello community …

I encountered a blocking error while sending outgoing messages from CHT 4.18 to Textit .. On investigating the api logs i noticed this error

2025-04-25T16:39:01.583 DEBUG: Checking for a configured outgoing message service 
2025-04-25T16:39:01.583 DEBUG: Checking for pending outgoing messages 
2025-04-25T16:39:01.594 INFO: Sending 1 messages 
2025-04-25T16:39:02.866 ERROR: Error thrown when trying to send message: Error: 403 - "<html lang=\"en\">\n  <head>\n    <title>Page - Not Found</title>\n    <style>\n      body {\n        background: var(--color-primary-dark);\n        font-family: \"Helvetica Neue\", Arial, \"Lucida Grande\", sans-serif;\n        color: #555;\n      }\n\n      .home {\n        margin: 10em auto;\n        margin-bottom: 1em;\n        max-width: 400px;\n        text-align: center;\n      }\n\n      .logo img {\n        max-width: 100%;\n      }\n\n      .message {\n        margin: 0 auto;\n        margin-top: 10px;\n        color: #333;\n        text-align: center;\n        max-width: 450px;\n      }\n    </style>\n  </head>\n  <body>\n    <div class=\"home\">\n      <h3><a href=\"/\">Home</a></h3>\n    </div>\n    <div style=\"text-align: center;\">\n      <h3>Something doesn't seem right.</h3>\n    </div>\n    <div class=\"message\">\n      We couldn't find the page you are looking for.\n      <br />\n      If you think we've made a mistake,\n      <br />\n      contact support and we will look into it.\n    </div>\n  </body>\n</html>\n"
    at request (/service/shared-libs/couch-request/src/couch-request.js:183:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
  [stack]: [Getter/Setter],
  [message]: `403 - "<html lang=\\"en\\">\\n  <head>\\n    <title>Page - Not Found</title>\\n    <style>\\n      body {\\n        background: var(--color-primary-dark);\\n        font-family: \\"Helvetica Neue\\", Arial, \\"Lucida Grande\\", sans-serif;\\n        color: #555;\\n      }\\n\\n      .home {\\n        margin: 10em auto;\\n        margin-bottom: 1em;\\n        max-width: 400px;\\n        text-align: center;\\n      }\\n\\n      .logo img {\\n        max-width: 100%;\\n      }\\n\\n      .message {\\n        margin: 0 auto;\\n        margin-top: 10px;\\n        color: #333;\\n        text-align: center;\\n        max-width: 450px;\\n      }\\n    </style>\\n  </head>\\n  <body>\\n    <div class=\\"home\\">\\n      <h3><a href=\\"/\\">Home</a></h3>\\n    </div>\\n    <div style=\\"text-align: center;\\">\\n      <h3>Something doesn't seem right.</h3>\\n    </div>\\n    <div class=\\"message\\">\\n      We couldn't find the page you are looking for.\\n      <br />\\n      If you think we've made a mistake,\\n      <br />\\n      contact support and we will look into it.\\n    </div>\\n  </body>\\n</html>\\n"`,
  body: '<html lang="en">\n' +
    '  <head>\n' +
    '    <title>Page - Not Found</title>\n' +
    '    <style>\n' +
    '      body {\n' +
    '        background: var(--color-primary-dark);\n' +
    '        font-family: "Helvetica Neue", Arial, "Lucida Grande", sans-serif;\n' +
    '        color: #555;\n' +
    '      }\n' +
    '\n' +
    '      .home {\n' +
    '        margin: 10em auto;\n' +
    '        margin-bottom: 1em;\n' +
    '        max-width: 400px;\n' +
    '        text-align: center;\n' +
    '      }\n' +
    '\n' +
    '      .logo img {\n' +
    '        max-width: 100%;\n' +
    '      }\n' +
    '\n' +
    '      .message {\n' +
    '        margin: 0 auto;\n' +
    '        margin-top: 10px;\n' +
    '        color: #333;\n' +
    '        text-align: center;\n' +
    '        max-width: 450px;\n' +
    '      }\n' +
    '    </style>\n' +
    '  </head>\n' +
    '  <body>\n' +
    '    <div class="home">\n' +
    '      <h3><a href="/">Home</a></h3>\n' +
    '    </div>\n' +
    '    <div style="text-align: center;">\n' +
    "      <h3>Something doesn't seem right.</h3>\n" +
    '    </div>\n' +
    '    <div class="message">\n' +
    "      We couldn't find the page you are looking for.\n" +
    '      <br />\n' +
    "      If you think we've made a mistake,\n" +
    '      <br />\n' +
    '      contact support and we will look into it.\n' +
    '    </div>\n' +
    '  </body>\n' +
    '</html>\n',
  status: 403,
  ok: false,
  headers: Headers {
    date: 'Fri, 25 Apr 2025 16:39:02 GMT',
    'content-type': 'text/html',
    'content-length': '992',
    connection: 'keep-alive',
    server: 'nginx',
    etag: '"6627a17e-3e0"',
    'strict-transport-security': 'max-age=31536000; includeSubDomains',
    'x-content-type-options': 'nosniff',
    'x-frame-options': 'DENY'
  },
  [Symbol(state)]: Proxy [
    {
      aborted: false,
      rangeRequested: false,
      timingAllowPassed: true,
      requestIncludesCredentials: true,
      type: 'default',
      status: 403,
      timingInfo: {
        startTime: 167347074.999293,
        redirectStartTime: 0,
        redirectEndTime: 0,
        postRedirectStartTime: 167347074.999293,
        finalServiceWorkerStartTime: 0,
        finalNetworkResponseStartTime: 167348337.558451,
        finalNetworkRequestStartTime: 167348043.845515,
        endTime: 0,
        encodedBodySize: 992,
        decodedBodySize: 992,
        finalConnectionTimingInfo: {
          domainLookupStartTime: 167347074.999293,
          domainLookupEndTime: 167347074.999293,
          connectionStartTime: 167347074.999293,
          connectionEndTime: 167347074.999293,
          secureConnectionStartTime: 167347074.999293,
          ALPNNegotiatedProtocol: undefined
        }
      },
      cacheState: '',
      statusText: 'Forbidden',
      headersList: HeadersList {
        cookies: null,
        [Symbol(headers map)]: Map(9) {
          'date' => [Object],
          'content-type' => [Object],
          'content-length' => [Object],
          'connection' => [Object],
          'server' => [Object],
          'etag' => [Object],
          'strict-transport-security' => [Object],
          'x-content-type-options' => [Object],
          'x-frame-options' => [Object]
        },
        [Symbol(headers map sorted)]: null,
        [entries]: [Getter],
        [entriesList]: [Getter]
      },
      urlList: [
        URL {
          href: 'https://textit.com/api/v2/broadcasts.json',
          origin: 'https://textit.com',
          protocol: 'https:',
          username: '',
          password: '',
          host: 'textit.com',
          hostname: 'textit.com',
          port: '',
          pathname: '/api/v2/broadcasts.json',
          search: '',
          searchParams: URLSearchParams {},
          hash: '',
          [Symbol(context)]: URLContext {
            href: 'https://textit.com/api/v2/broadcasts.json',
            protocol_end: 6,
            username_end: 8,
            host_start: 8,
            host_end: 18,
            pathname_start: 18,
            search_start: 4294967295,
            hash_start: 4294967295,
            port: 4294967295,
            scheme_type: 2,
            [hasPort]: [Getter],
            [hasSearch]: [Getter],
            [hasHash]: [Getter]
          }
        },
        [length]: 1
      ],
      body: {
        stream: ReadableStream { locked: true, state: 'closed', supportsBYOB: true },
        source: null,
        length: null
      }
    },
    {
      get: [Function: get] { [length]: 2, [name]: 'get' },
      set: [Function: set] { [length]: 3, [name]: 'set' }
    }
  ],
  [Symbol(headers)]: Headers {
    date: 'Fri, 25 Apr 2025 16:39:02 GMT',
    'content-type': 'text/html',
    'content-length': '992',
    connection: 'keep-alive',
    server: 'nginx',
    etag: '"6627a17e-3e0"',
    'strict-transport-security': 'max-age=31536000; includeSubDomains',
    'x-content-type-options': 'nosniff',
    'x-frame-options': 'DENY'
  }
}

which i believe is happen while sending messages to the Textit /api/v2/broadcasts.json api endpoint. We can refactor the code to add a custom User-Agent header ie

headers: {
    Authorization: `Token ${apiToken}`,
    'User-Agent': 'CHT-Core-RapidPro-Integration/1.0'
  },

as we had done for the outbound feature a while back because Textit want this to be added for all api requests made
cc @diana @twier @antony

Hi @cliff

It looks like the URL that you are using to connect to textit: https://textit.com/api/v2/broadcasts.json is not correct. I got this url from your error message, and if I open it in a browser, I get the same error.
Can you please double check your configuration so that you connect to texit correctly? What should this URL be?

thanks @diana the feedback …
i have tried via postman and it works

in the configuration i have used

"sms": {
    "outgoing_service": "rapidpro",
    "rapidpro": {
      "url": "https://textit.com/"
    }
  }
1 Like

@diana i have succesfuly posted via curl

(base) cliff@cliffs-MacBook-Pro ~ % curl -X POST https://textit.com/api/v2/broadcasts.json \
  -H "Authorization: Token ff53275d-2eb9-4a80-8a52-cd320cd6c29d" \
  -H "Content-Type: application/json" \
  -d '{
    "contact": "ewda5b09-9608-409d-9172-996b7bafac74",
    "urns": ["tel:+111113333313"],
    "text": "Thanks for notifying us. Have a wonderful day.",
    "attachments": [],
    "quick_replies": [
      {"text": "Thank you", "extra": "thanks"},
      {"text": "testing"}
    ]
}'

{"id":10173413,"status":"pending","progress":{"total":-1,"started":0},"urns":["tel:+111113333313"],"contacts":[],"groups":[],"text":{"eng":"Thanks for notifying us. Have a wonderful day."},"attachments":{"eng":[]},"quick_replies":{"eng":[{"text":"Thank you","extra":"thanks"},{"text":"testing"}]},"base_language":"eng","created_on":"2025-04-28T19:03:44.769553Z"}%                                                             (base) cliff@cliffs-MacBook-Pro ~ %

@cliff - the telephone numbers looked real - so I’ve changed those in case they were PII. As well, the Authorization: Token looked real as well which would allow unauthorized access to your textit account. I’ve changed this as well, however if you’re concerned please change the password on your textit account which should expire any existing tokens.

If this was already dummy data that you posted, then you can just ignore more!

Hi @cliff

Nothing has changed, afaict, in this part of the code, I’m assuming it’s an authorization problem, where your token in CHT-Core is outdated. Can you please check?

Also, the User-Agent header was not added to this integration, but from your successful curl, it seems that it is not needed.

thanks @mrjones all the critical information and credentials is dummy

thanks @diana though the user agent is now needed for all textit api calls.. Postman and curl does add the user-agent header in the background while making the api calls
Here is the screenshot of the feedback i got from the Textit support team

@diana curl does also provide a default user agent header incase you have not provided one

We could set a default User-Agent header value in the api calls for the integration code

@diana @twier @jkuester any thoughts on this ?

Hi @cliff

Adding the user-agent header to the RapidPro calls would require an additional update made to the cht-core.

1 Like

I’ve created an issue to track this. https://github.com/medic/cht-core/issues/9936

1 Like

thanks much @diana for creating the issue

hello @diana @twier i see the work was merged here , could you kindly make a branch pre-release that we can upgrade to in the meantime

Hi @cliff. We will be releasing the change as part of 4.19.0 soon, so you can upgrade to that once it’s released.

3 Likes