High CPU load on cluster node
Incident Report for NiceLabel Cloud
Post mortem:

There were some issues in South Central US region.

The main issue was that memory and CPU utilization was high in all Label Cloud deployments in this region. Our alerting unfortunately did not send out any alerts, because CPU utilization was just a few percentage points too low.

Once the issue was reported we have immediately restarted both nodes and everything normalized.

We have also fixed our alerting to make sure this does not happen again.

In the next days we will continue to monitor the situation.
Posted Jun 29, 2021 - 20:00 UTC
The issue has been identified and a fix is being implemented.
Posted Jun 29, 2021 - 14:00 UTC
This incident affected: South Central US (Dashboard, Document Storage, Print).