Issue Summary

When data is incoming at our Platform API it is queued for further processing and merging. The application was not able to write to the queue for some time due to technical issues and all incoming data was sent to a fallback queue.

After investigation and repair of the main queue, our team re-created the main queue from the fallback queue and started all processes again.

During that time incoming data has beed delayed a bit due to the main queue being busy catching up and keeping the right order of incoming data. We returned to normal operations and data should come in without additional delays.

Corrective and Preventative Measures

We have taken a few approaches to prevent this from happening again in the future. One is a better Monitoring on the main queue which will send alerts faster if we receive less messages. We will work on additional preventative measures to make sure that the outage of one queue will not affect the data processing.

Productsup is committed to continually and quickly improving our technology and operational processes to prevent future outages. Unfortunately, we were not able to prevent yesterday's loss. For this, we sincerely apologize for the inconvenience this has caused you, your team, and your organization. We thank you for your continued support.