GitLab

Failure mode: A code release introduced the failure. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with medium confidence (taxonomy v0.3).

1. investigating 2026-10-20 21:56 UTC

   We've investigating 404s on the customers panel. We'll update as we learn more

2. investigating 2026-10-20 22:47 UTC

   We’re continuing our investigation of 404s on the customers panel. We’ll update you as we learn more.

3. investigating 2026-10-20 23:15 UTC

   We’re continuing our investigation of 404s on the customers panel. We’ll update you as we learn more.

4. monitoring 2026-10-20 23:51 UTC

   The Customers Portal is loading normally again, and we are no longer observing the intermittent 404 errors affecting sign-in and page loading. A redeployment has completed successfully, and we are monitoring the service for any recurrence before resolving the incident.

5. resolved 2026-10-21 01:21 UTC

   The monitoring period has concluded, and the incident has been confirmed resolved. See the issue for more details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23297

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with low confidence (taxonomy v0.3).

1. investigating 2026-10-20 07:08 UTC

   We're investigating intermittent HTTP 503 errors when opening project pages. At this time, this issue only occurs in specific projects. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23288

2. monitoring 2026-10-20 07:37 UTC

   This issue affects projects and groups using GitLab Secrets Manager. Users saw HTTP 500 errors, not 503 as previously stated. A mitigation has been applied and we're monitoring. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23288

3. monitoring 2026-10-20 07:37 UTC

   This issue affects projects and groups using GitLab Secrets Manager. Users saw HTTP 500 errors, not 503 as previously stated. A mitigation has been applied and we're monitoring. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23288

4. resolved 2026-10-20 09:55 UTC

   We have not seen any more HTTP 500 errors for GitLab Secrets Manager since the mitigation was applied. This incident is now resolved. Details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23288

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with high confidence (taxonomy v0.3).

1. investigating 2026-10-16 16:10 UTC

   We are investigating the cause of 503s on GitLab.com and the site moving slowly. We will update once we have additional details.

2. investigating 2026-10-16 16:25 UTC

   We are still investigating the cause of 503 on GitLab.com. We are seeing some services return but errors still intermittent. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23261

3. identified 2026-10-16 16:40 UTC

   We have identified the cause of 503s on GitLab.com and are working to remediate the issue. We are still seeing intermittent 503s and site slowness as we work to resolve the issue. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23261

4. identified 2026-10-16 16:52 UTC

   Most services on GitLab.com have recovered but we are seeing slowness with Duo requests and services and are looking to resolve it. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23261

5. monitoring 2026-10-16 17:05 UTC

   We have taken steps to resolve the slowness and 503s on GitLab.com and are monitoring the situation. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23261

6. resolved 2026-10-16 17:24 UTC

   The issue is fully resolved and operations on GitLab.com and GitLab Duo are operating normally. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23261

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with low confidence (taxonomy v0.3).

1. investigating 2026-10-16 11:33 UTC

   We’re investigating an issue affecting CI/CD on GitLab.com for a limited number of customers. Pipelines may fail with the following error, and the UI may be slow: “CI configuration fetch from Gitaly timed out. This may indicate Gitaly service slowness or an outage.” More updates...

2. monitoring 2026-10-16 12:28 UTC

   We've applied a temporary mitigation to address CI/CD failures and UI slowness affecting a limited number of GitLab.com customers. We're monitoring recovery while working on a longer-term solution.

3. resolved 2026-10-16 13:21 UTC

   The issue causing CI/CD failures and UI slowness for a limited number of GitLab.com customers has been resolved. Thank you for your patience.

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with low confidence (taxonomy v0.3).

1. investigating 2026-10-15 16:19 UTC

   We are investigating an issue affecting GitLab.com CI/CD. Some pipelines may remain in the created state for an extended period or fail to start, causing CI/CD workflows to stall. More Details at https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23234

2. identified 2026-10-15 17:23 UTC

   We have identified the cause of the issue causing CI/CD delays on GitLab.com and are working to take corrective actions. The issue may persists intermittently as we do so. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23234

3. resolved 2026-10-15 19:50 UTC

   We have taken steps to mitigate the delays on CI/CD jobs on GitLab.com and services have returned to normal. More details at: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23234

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with low confidence (taxonomy v0.3).

1. investigating 2026-10-15 04:43 UTC

   We're investigating an issue that's stopping pages on customers.gitlab.com from loading. Purchasing, renewing, and managing subscriptions may be affected. GitLab.com itself is not affected.

2. investigating 2026-10-15 05:10 UTC

   We have received reports that some groups on GitLab.com are experiencing HTTP 500 errors because the group is unable to sync with customers.gitlab.com. We are continuing to investigate the issue.

3. investigating 2026-10-15 05:27 UTC

   No material updates to report. We are still investigating the root cause.

4. investigating 2026-10-15 05:50 UTC

   No material updates to report. We'll share another update when we have more information.

5. monitoring 2026-10-15 08:35 UTC

   Mitigations have been applied to customers.gitlab.com, and we are monitoring service availability. Intermittent errors may continue to affect Customers Portal pages and GitLab.com groups that rely on the portal for subscription-related operations. We are continuing to...

6. resolved 2026-10-15 09:02 UTC

   The issue affecting customers.gitlab.com has been resolved. Customers Portal pages and subscription-related operations have returned to normal. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23240

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with high confidence (taxonomy v0.3).

1. investigating 2026-10-14 14:33 UTC

   We are investigating an issue affecting GitLab.com merge trains. Some merge train pipelines may remain in the created state for an extended period or fail to start, causing merge trains and related CI/CD workflows to stall. More details:...

2. investigating 2026-10-14 15:22 UTC

   We are investigating an issue affecting GitLab.com CI/CD. Merge train, scheduled, and child/downstream pipelines may be delayed, remain in a created or pending state, or fail to start. Mergeability analysis and webhook notifications may also be delayed. More details:...

3. investigating 2026-10-14 16:40 UTC

   We are investigating an issue affecting GitLab.com and general site errors and slowness related to the initial issue. We are actively investigating the issue. More details: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23192

4. identified 2026-10-14 16:50 UTC

   We identified the issue affecting GitLab.com and general site errors and slowness related to the initial issue. We are actively working to resolve the issue. More details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23192

5. resolved 2026-10-14 18:30 UTC

   The source of the issue has been identified and we have taken steps to remediate and resolve the issue. GitLab.com is stable now. More details: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23192

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: A provider or third-party service this vendor depends on failed. Root cause: A failure at a provider or vendor they depend on. Tagged by deepseek/deepseek-v4.1-flash with high confidence (taxonomy v0.3).

Names an upstream provider: Google Cloud.

1. investigating 2026-10-13 14:27 UTC

   We are investigating increased queue times for CI/CD jobs using GitLab.com small Linux x86-64 hosted runners. Affected jobs may remain pending longer than usual before starting. We have identified errors and delays when provisioning runner capacity and are working to restore...

2. identified 2026-10-13 15:46 UTC

   We have identified an ongoing Google Cloud Compute Engine issue affecting runner-capacity provisioning in us-east1. Some GitLab.com CI/CD jobs using small Linux x86-64 hosted runners may remain pending longer than usual before starting. Our engineering teams are working with...

3. monitoring 2026-10-13 19:31 UTC

   Job start times for CI/CD jobs on GitLab.com small Linux x86-64 hosted runners have returned to normal, and the backlog of pending jobs has been processed. The delays were caused by errors and capacity constraints from our cloud provider when provisioning runner capacity in one...

4. resolved 2026-10-13 20:59 UTC

   Job start times for CI/CD jobs on GitLab.com small Linux x86-64 hosted runners have returned to normal, and the backlog of pending jobs has been processed. The issue with the provider persists, but we are not seeing any impact on our systems. More details:...

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: The text does not say enough to tell. Root cause: The vendor does not say why it happened. Tagged by deepseek/deepseek-v4.1-flash with high confidence (taxonomy v0.3).

1. investigating 2026-10-13 07:10 UTC

   We are currently investigating reports of an increase of HTTP 50X errors on GitLab.com

2. investigating 2026-10-13 07:30 UTC

   We are seeing an improvement in errors rates in our monitoring, starting from 07:14 UTC. We are still identifying the root cause.

3. investigating 2026-10-13 07:48 UTC

   General GitLab.com service continues to improve. We're now focused on restoring Duo services.

4. monitoring 2026-10-13 07:59 UTC

   We are seeing improvements across all services on GitLab.com. We are resuming our investigation on identifying the root cause.

5. monitoring 2026-10-14 02:33 UTC

   Monitoring confirms that GitLab.com services have remained stable. We are continuing to monitor for any recurrence of the issues before marking this incident as resolved.

6. resolved 2026-10-14 23:40 UTC

   We have been monitoring this issue for a few days with no reported occurrences.

Excerpts are shortened. Read the full incident on GitLab's status page.

Failure mode: AI model returned degraded, wrong or empty output (not errors or timeouts). Root cause: A bug in the vendor's own code. Tagged by deepseek/deepseek-v4.1-flash with high confidence (taxonomy v0.3).

1. identified 2026-10-12 11:30 UTC

   We’re investigating an issue affecting some GitLab Duo Workflow operations. Duo Code Review results may be incomplete or lower quality than expected. Our engineering teams are working on mitigation and will provide further updates. More details:...

2. identified 2026-10-12 13:37 UTC

   We've identified the cause of the issue affecting GitLab Duo Workflow operations. Duo Code Review remains available, but reviews may contain less context than expected. Our engineering teams are expediting a fix and will continue monitoring until service is fully restored. For...

3. monitoring 2026-10-13 00:28 UTC

   Errors affecting Code Review tools have decreased, and the tools appear to be operating normally. We’re monitoring error rates and investigating the improvement. A fix is merged but not yet in production, so recovery is still being validated.

4. monitoring 2026-10-13 00:29 UTC

   Errors affecting Code Review tools have decreased, and the tools appear to be operating normally. We’re monitoring error rates and investigating the improvement. A fix is merged but not yet in production, so recovery is still being validated.

5. identified 2026-10-13 02:11 UTC

   Monitoring shows failures are still occurring in a tool used by Duo Code Review. The service remains available, but reviews may contain less context than expected. A fix is merged and awaiting deployment; we’ll update once recovery is confirmed.

6. resolved 2026-10-14 01:16 UTC

   The fix was deployed and continued monitoring has shown that the identified tool failure has been addressed. This issue is now resolved. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23158

Excerpts are shortened. Read the full incident on GitLab's status page.