We experienced some problems over the last weeks which led to partial or major outages of the platform and/or components. We posted Postmortems to give you as much information as possible.
Unfortunately we had another outage on Tuesday which was caused by the Image Designer. The Image Designer allows real-time manipulation of images with product data to be delivered directly to (advertising) partners. What happened yesterday was a result of a partner requesting more images at the same time then we ever received before on the Image Designer. This caused the maximum capacity of queries we can send to our internal database to be reached, sadly this meant that our Platform was also unavailable.
We have taken a few approaches to resolve this issue in the future to prevent this from happening again. One being setting stricter rate limits on how many images can be requested by a single partner. The second is sending the queries from the Image Designer to another database. In next few days we’re working on adding an extra layer between the Image Designer and the database to allow for short-time caching to relieve even more stress from the systems. Also for a more global presence we’ve started the roll-out of extra servers in a different datacenter in Finland.