Back to overview
Degraded

System Degradation

Oct 14 at 03:27pm UTC
Affected services
Event Ingestion (US - Standard)

Resolved
Oct 14 at 05:08pm UTC

We’ve taken steps to address the service degradation and are closely monitoring performance.

Post Mortem

What happened: After upgrading our columnar database, an incompatibility stopped automatic consolidation on a core table.

Impact: Partial data loss and content triggering issues.

Cause: The new version didn’t work correctly with tables that use pre-computed summaries, leading to too many small data fragments.

Fix: Rebuilt the summaries to force consolidation and rolled back to the prior database version. Performance recovered.

Prevention: Added an alert on fragment count, paused similar upgrades, and expanded compatibility/performance testing before future releases.

We’re sorry for the disruption and appreciate your patience.

Created
Oct 14 at 03:27pm UTC

We are experiencing service degradation. Our team is actively monitoring and working to restore full performance.