We are experiencing an issue causing delayed event processing

  • Now
  • Mar 29, 2024, 5:00:37 PM GMT+1
    Resolved
    We are resolving the incident as all systems are operational and a permanent fix is in place.
  • Mar 29, 2024, 3:41:26 PM GMT+1
    Monitoring
    Due to replaying certain events to guarantee delivery, there is a slight chance that some customers might experience duplicate alerts in the next 15 minutes.
  • Mar 29, 2024, 3:31:07 PM GMT+1
    Monitoring
    We have identified the issue and have deployed a temporary fix to stop events from being delayed. And are already monitoring a fully recovered event processing queue. So far the partial impact was only identified on Grafana and Zabbix events.
  • Mar 29, 2024, 3:05:17 PM GMT+1
    Investigating
    We are investigating an issue where certain monitoring tool events are processed with a delay.
Affected services
Event Ingestion
Degraded