UI and parts of the API were temporarily unavailable

Resolved
Duration: 1hJul 6, 2023

Affected services

API

Partial outage

Web Application

Major outage

Timeline

Resolved
07:08Jul 6, 20232y ago

We have resolved the incident.

APIWeb Application
Monitoring
06:49Jul 6, 20232y ago

UI and affected API parts are fully operational again since 06:26 am - we are closely monitoring the recovery of all affected systems.

APIWeb Application
Identified
06:01Jul 6, 20232y ago

A minor Kubernetes update broke internal DNS services, causing parts of the API and UI to become unavailable. Event queue processing and notification delivery continued to work. We identfied the issue during the upgrade process and were able to roll back after 2 minutes, however the rollback process of all node pools took a moment.

APIWeb Application