Signals Expert Interviews

CrowdStrike Incident: Cybersecurity Expert's Perspective and Workaround Steps

Confirmed: one faulty software update can bring down 8.5 million computers worldwide. Worse, the outage presumably cost Fortune 500 companies about $5.4 billion. In this article, I sum up the course of the events, the factors leading to the disruption, and the recommendations for the affected companies and individuals.

Details of the CrowdStrike disaster 

In the early hours of Friday, organizations in Australia running Microsoft’s Windows operating system began reporting Blue Screens of Death (BSODs). Shortly after, similar reports emerged globally, including from the UK, India, Germany, the Netherlands, and the US. Windows machines worldwide were impacted, resulting in significant disruptions across various sectors, including banks, airports, TV stations, healthcare organizations, and more. Major disruptions included Sky News going offline and US airlines United, Delta, and American Airlines issuing a "global ground stop" on all flights.

banner3 (7)

The root cause was traced to a misconfigured or corrupted update pushed by CrowdStrike, a prominent cybersecurity company, to its customers. This update led to widespread BSODs on Windows hosts, impacting devices running Windows but not other operating systems like Mac or Linux. The update file had a .sys extension; however, it was not a kernel driver. The file communicates with other Falcon sensor components sharing the “sacred” space on a Windows PC with the Windows kernel. There, the components interact directly with hardware and memory. According to CrowdStrike, a “logic error” in that code resulted in Windows PCs and servers crashing and responding in BSOD within seconds after they loaded.

banner3-1

Tech things apart, a Microsoft spokesperson attributed the recent Crowdstrike failure to certain points from a 2009 regulatory agreement between Microsoft and the EU, according to The Wall Street Journal. The spokesperson claimed Microsoft agreed to give external security developers the same level of access to Microsoft's APIs as the company itself has, opening the gates to critical bugs as in the CrowdStrike update.

CrowdStrike's Response

The damage could be much worse. Luckily, CrowdStrike engineers quickly acknowledged the issue and provided a workaround for affected systems. CEO George Kurtz issued a statement confirming that the issue was a defect in a Windows update, not a cyberattack. The company deployed a fix and provided customer support through its support portal.
Last Saturday, Microsoft’s Vice President of Enterprise and OS Security David Weston announced that the company has created a recovery tool in collaboration with CrowdStrike. This solution will help Microsoft’s Azure infrastructure speed up a fix for the faulty update. The tool automates much of the recovery process and is constantly being updated.

Tap our years of expertise in Fintech solution development.
Discover INSART

CrowdStrike incident implications for businesses

This incident is a reminder of how critically important robust update management is—and how severe the potential risks associated with software updates are. To safeguard against similar issues, companies can take the following steps:

1. Ensure updates are rigorously tested in various environments before deployment.

2. Maintain comprehensive backup and recovery plans to restore systems swiftly in case of failures.

3. Implement monitoring systems to detect and address issues as soon as they arise.

4. Establish clear communication protocols to inform and guide employees during incidents.

Workaround steps

If your organization is affected by the recent CrowdStrike update issue, follow these steps to remediate the problem:

1.  Access the device via Safe Mode

Refer to Microsoft's Advanced startup options: Microsoft Support

Steps: 

a) Press and hold the F8 key as your computer restarts. You need to press F8 before the Windows logo appears.

b) On the Advanced Boot Options screen, use the arrow keys to highlight Safe Mode.

c) Log on to your computer with a user account that has administrator rights, and choose “Safe Mode with Command Prompt” or another suitable option.

2.  Navigate to the CrowdStrike directory

a) Open the Command Prompt or File Explorer.

b) Go to C:\Windows\System32\drivers\CrowdStrike.

3.  Locate and delete the problematic file

a) Find the file matching the pattern “C-00000291*.sys”.

b) Delete the file.

4️.  Boot the host normally

a) Restart the computer normally.

b) Verify that the system boots without encountering BSODs.

By following these steps, you should be able to resolve the issue caused by the faulty CrowdStrike update and restore normal operations. You can find manual remediation documentation and scripts from Microsoft here.

How INSART stayed resilient 

Fortunately, INSART has not been adversely affected by the CrowdStrike incident. Our resilience is not pure luck and can be attributed to several key factors:

  • Proactive monitoring: Our team employs proactive monitoring tools to promptly detect and respond to suspicious activities.
  • SentinelOne implementation: Unlike some organizations, INSART does not use CrowdStrike products. Instead, we use SentinelOne, which allows us to configure automatic updates and install only verified agent versions. This approach ensures that we minimize the risk of deploying flawed updates and maintain the integrity of our systems.

banner3-2

By maintaining a proactive and vigilant approach to cybersecurity, INSART has successfully safeguarded its assets and data from potential threats. We also have a multi-level security strategy to protect our clients’ data and technology while developing Fintech software solutions for them.

If you want to ensure your new product will adhere to the latest industry regulations and security standards, we can help with that. Contact us to learn more about how we engineer secure Fintech solutions.

How to build a winning engineering culture

How to build a winning engineering culture

The culture of an engineering team is the DNA that defines the principles a group of engineers will use to make decisions about technology, strategy, and planning. Wherever you work, you want to contribute to the evolution of the engineering culture to build scalable, cross-functional, and successful teams. This article will help you find out how t...

The culture of an engineering team is the DNA that defines the principles a group of engineers will use to make decis...
More

Business Analysis for Fintechs: What Tech Leaders Need to Know

Business Analysis for Fintechs: What Tech Leaders Need to Know

A deepdive assessment of your business processes and functions may reveal that you could do much better, even if you perform just fine.

For financial service companies, business analysis is necessary to optimize the workflow. For startups, it’s a chance to play it safe. Cases differ, but BA is still a king.

In this article, you’ll learn how ...

A deepdive assessment of your business processes and functions may reveal that you could do much better, even if you ...
More

How to Leverage Fintech Regulations and Standards

How to Leverage Fintech Regulations and Standards

For Fintech leaders, mastering compliance is not merely a choice; it's an imperative. Investing effort in this complex terrain paves the way for sustainable growth, trust, and legal integrity. And since technology has simplified complying with all the burgeoning regulations, acing the rules becomes less demanding a task.

In this article, I will e...

For Fintech leaders, mastering compliance is not merely a choice; it's an imperative. Investing effort in this comple...
More