Corrupt Shard on CouchDB

Technical Support
1

Hello,
Am getting the below error from Sentinel:

2024-06-14T07:26:19.259 ERROR: Task backgroundCleanup completed with error: {
  error: 'bad_return_value',
  reason: '{read_beyond_eof,"./data/shards/40000000-5fffffff/medic-user-jar-binza-meta.1685021444.couch"}',
  status: 500,
  name: 'bad_return_value',
  message: '{read_beyond_eof,"./data/shards/40000000-5fffffff/medic-user-jar-binza-meta.1685021444.couch"}',
  stack: 'Error\n' +
    '    at Object.generateErrorFromResponse (/service/sentinel/node_modules/pouchdb-errors/lib/index.js:104:18)\n' +
    '    at /service/sentinel/node_modules/pouchdb-adapter-http/lib/index.js:254:33\n' +
    '    at runMicrotasks (<anonymous>)\n' +
    '    at processTicksAndRejections (node:internal/process/task_queues:96:5)'
}

Does this mean that the shard is corrupt? how can I rebuild it?

image
image1057×62 1.94 KB

2

Hi @Job_Isabai

There’s no way to rebuild a shard, only to restore it from backup.
I believe we have seen two conditions where this error can happen (and it’s extremely rare):

  • storage write/lock hiccup - this is extremely rare
  • running two couchdb instances connected to the same data - very likely.

Can you check whether you have two couchdb instances running?

3

I am running only one instance of CouchDB.
This issue might be as a result of migration from 4.3 to 4.5. There were some failures in indexing views during the upgrade. Let me see if I can restore a backup.
Thanks

4

You did not need to migrate from 4.3 to 4.5, just the simple upgrade would have been enough. Did you actually migrate or upgrade?

5

Sorry I upgraded. I stagged 4.5 and upgraded afterwards.