Destination "Productsup Server" Issues

Incident Report for Productsup

Postmortem

Incident Overview
On May 12, we experienced a temporary service disruption affecting parts of our transport platform. This occurred during a configuration update aimed at improving our system’s routing capabilities.

Root Cause
The disruption was caused by a misconfiguration in our internal storage system. Although the change had passed staging tests, it behaved differently in production, leading to certain requests being misrouted.

Impact
During the incident, some clients were unable to access services hosted on specific subdomains of our transport platform and jobs couldn't write to destination "Productsup server". Normal operation resumed shortly after the change was rolled back.

Preventive Measures
To prevent this issue from recurring, we are:

  • Improving our deployment validation processes.
  • Enhancing staging to more closely reflect production conditions.
  • Adding more automated checks to detect routing issues earlier.

Conclusion
The issue was resolved by reverting the configuration. Processes have been updated to prevent recurrence. All systems are operating normally.

Posted May 13, 2025 - 14:47 CEST

Resolved

This incident has been resolved.
Posted May 12, 2025 - 15:37 CEST

Monitoring

A fix has been implemented and we are monitoring the results.
Posted May 12, 2025 - 14:57 CEST

Investigating

We're investigating an elevated number of errors when uploading data to "Productsup Server".
Posted May 12, 2025 - 13:45 CEST
This incident affected: Data Processing and Destination "Productsup Server".