On Friday, August 1st 2025, a significant volume of users experienced false downtime alerts for monitors from our EU monitoring region. After investigation, we identified the root causes and have taken immediate steps to address them.
Update: August 16th:
We experienced a false alert incident in our EU monitoring region on August 16th, tied to the same root causes outlined below. The long-term fixes we shared earlier this month are already in progress: expanding our EU infrastructure, retiring dormant monitors, and rolling out new IPs. However, these changes are being phased in to ensure users have sufficient time to update firewall rules and avoid disruptions. We will continue to work to improve stability in the coming week. Please contact our support team if you continue to experience issues.
What caused the incidents?
- Monitoring of decommissioned endpoints
Our systems continued to actively monitor endpoints that no longer exist or have been decommissioned. This included certain BGPs that are no longer announced, causing our monitoring activities to be perceived as indiscriminate network scanning by one of our hosting providers, leading to blocked traffic.
- Regional impact
The combination of these factors led to a 45% drop in our available capacity in the EU region, making it more likely for monitoring delays or failures to trigger false positives.
How we’re fixing it:
- Retiring monitoring on dormant endpoints: We are rolling out logic to stop monitoring endpoints from long-abandoned accounts.
- Network expansion: We’re expanding our infrastructure in the EU and globally by partnering with additional providers. This will increase our monitoring capacity and redundancy, and reduce the risk of regional outages.
- New IP addresses: We are introducing new IPs for our EU region. To ensure uninterrupted monitoring, we ask all users to whitelist our latest IPs before September 1, 2025 (with similar updates coming to other regions as well).
- API for IP management: You can now automatically stay updated using our new IP API.
- False alert cleanup: As part of our mitigation, all connection timeout incidents from the affected period will be removed from user accounts to clean up inaccurate reports.
- SMS credits refund: All SMS and voice alert credits used during the August 1st, 2025, incident period will be refunded to your account.
Looking ahead
We take stability and reliability extremely seriously. We’re deeply committed to delivering accurate uptime monitoring, and this incident has already triggered key improvements across our infrastructure. We’re actively working to prevent similar issues from happening again.
Thanks for your trust and your patience.
The UptimeRobot Team
