ICA computation impacted all regions

Incident Report for Illumina Cloud Products

Resolved

We have fully recovered and systems are operating normally. Customers should see runs process as they did before. There may be some delay for runs starting as queues are being drained, and we will continue to monitor. To provide some color into the root cause, we identified a global dependency related to how we obtain statuses of filesystems across all ICA regions. The full outage in MEC1 created a cascading event that effectively throttled the other regions. We have temporarily removed MEC1 from the list, which has led to recovery. As part of our upcoming MEC1 recovery process, which will begin after AWS recovers in that region (see https://health.aws.amazon.com/health/status for updates), we will be sure to evaluate this dependency to ensure business continuity in other regions. We will then take efforts to remove this cross-region dependency.
Posted Mar 02, 2026 - 20:47 UTC

Identified

We believe we have identified a root cause and mitigation. We are starting to see recovery. As part of this recovery, you will see analyses begin to start. As we gain confidence in the mitigation, we will continue to increase the overall throughput back to normal levels.
Posted Mar 02, 2026 - 17:11 UTC

Investigating

We are investigating an issue where FSx file systems are unable to start globally. We are actively looking into the issue with our cloud provider and will be providing an update around than 10:00PST.
Posted Mar 02, 2026 - 16:19 UTC
This incident affected: USA (Basespace Sequence Hub - US East, Illumina Connected Analytics - US East), Canada (Basespace Sequence Hub - Canada, Illumina Connected Analytics - Canada), Australia (Basespace Sequence Hub - Sydney, Illumina Connected Analytics - Sydney), Germany (Basespace Sequence Hub - Frankfurt, Illumina Connected Analytics - Frankfurt), United Kingdom (Basespace Sequence Hub - London, Illumina Connected Analytics - London), Japan (Basespace Sequence Hub - Tokyo, Illumina Connected Analytics - Tokyo), Singapore (Basespace Sequence Hub - Singapore, Illumina Connected Analytics - Singapore), India (Basespace Sequence Hub - Mumbai, Illumina Connected Analytics - Mumbai), Israel (Basespace Sequence Hub - Tel Aviv, Illumina Connected Analytics - Tel Aviv), South Korea (Basespace Sequence Hub - Seoul, Illumina Connected Analytics - Seoul), Indonesia (Basespace Sequence Hub - Jakarta, Illumina Connected Analytics - Jakarta), and UAE (Basespace Sequence Hub - Abu Dhabi, Illumina Connected Analytics - Abu Dhabi).