Crowdstrike causes world-wide IT outages

The 2009 EU-Microsoft deal involved allowing software from others than Microsoft to replace Microsoft software on Windows. In the years before the deal, Microsoft had progressively added features to Windows that had previously been covered by software from other parties, forcing that software out of the market.
As in everything, do your research, read the terms and conditions before adopting new software. Particularly if your business depends on that software functioning correctly.

Microsoft updates have, before 2009 and after, caused Windows BSDs worldwide as well. The Crowdstrike-caused problems show moving outside the MS-universe does not protect you from BSDs.

My current employer uses a Crowdstrike alternative, not Windows Defender, and was not affected by the recent problems. Testing updates before implementing them is Standard Operating Procedure, even if exceptions are sometimes forced upon us.

When in danger or in doubt, blame EU when you find out.
 
Last edited:
I wonder if there is any vetting for the muppets that are allowed to upload this software onto Microsoft at kernel level. Seems like there should be. Maybe background wasn't an issue here, but it sure as hell could be in the future now that the weakness has been advertised.
 
it.slashdot.org

CrowdStrike Offers a $10 Apology Gift Card To Say Sorry For Outage - Slashdot

Lorenzo Franceschi-Bicchierai, reporting for TechCrunch: CrowdStrike, the cybersecurity firm that crashed millions of computers with a botched update all over the world last week, is offering its partners a $10 Uber Eats gift card as an apology, according to several people who say they received...
it.slashdot.org

slashdot.org

Microsoft Pushes for Windows Changes After CrowdStrike Incident - Slashdot

In the wake of a major incident that affected millions of Windows PCs, Microsoft is calling for significant changes to enhance the resilience of its operating system. John Cable, Microsoft's vice president of program management for Windows servicing and delivery, said there was a need for...
slashdot.org
 
yro.slashdot.org

CrowdStrike Is Sued By Shareholders Over Huge Software Outage - Slashdot

Shareholders have sued CrowdStrike on Tuesday, claiming the cybersecurity company defrauded them by concealing how its inadequate software testing could cause the global software outage earlier this month that crashed millions of computers. Reuters reports: In a proposed class action filed on...
yro.slashdot.org
 
From the Slashdot piece
Shareholders have sued CrowdStrike on Tuesday, claiming the cybersecurity company defrauded them by concealing how its inadequate software testing could cause the global software outage earlier this month that crashed millions of computers. Reuters reports: In a proposed class action filed on Tuesday night in the Austin, Texas federal court, shareholders said they learned that CrowdStrike's assurances about its technology were materially false and misleading when a flawed software update disrupted airlines, banks, hospitals and emergency lines around the world. They said CrowdStrike's share price fell 32% over the next 12 days, wiping out $25 billion of market value, as the outage's effects became known, Chief Executive George Kurtz was called to testify to the U.S. Congress, and Delta Air Lines reportedly hired prominent lawyer David Boies to seek damages.

The complaint cites statements including from a March 5 conference call where Kurtz characterized CrowdStrike's software as "validated, tested and certified." The lawsuit led by the Plymouth County Retirement Association of Plymouth, Massachusetts, seeks unspecified damages for holders of CrowdStrike Class A shares between Nov. 29, 2023 and July 29, 2024.
Click to expand...
Suppose the court rules for the claimants in this case, then Crowdstrike will have to pay. This will negatively affect the financial situation of Crowdstrike. Which in turn will hurt Crowdstrike share prices.
I might have the wrong end of the stick here, but isn't taking financial risk essential to the way investing money in shares works to better increase capital, when compared to the return of a savings account?

The Crowdstrike blunder is not about providing false financial information to investors, but about a grave operational mistake. I believe the correct response from shareholders should be demanding a change of magement, not a financial claim.
 
Last edited:

Delta Airlines estimate it cost them $500 million.

Delta has yet to file a lawsuit against either CrowdStrike or Microsoft, but a person familiar with its actions confirmed to CNN on Tuesday that it had hired the law firm of high-profile attorney David Boies to pursue compensation from the two companies. Microsoft did not respond to a request for comment on Wednesday. A CrowdStrike spokesperson would only say, “We are aware of the reporting, but have no knowledge of a lawsuit and have no further comment.”

“We have no choice,” Bastian told CNBC. “We have to protect our shareholders, we have to protect our customers (and) our employees for the damage, not just the cost but the reputational damage.”
Click to expand...
 
Arjen said:
From the Slashdot piece

Suppose the court rules for the claimants in this case, then Crowdstrike will have to pay. This will negatively affect the financial situation of Crowdstrike. Which in turn will hurt Crowdstrike share prices.
I might have the wrong end of the stick here, but isn't taking financial risk essential to the way investing money in shares works to better increase capital, when compared to the return of a savings account?

The Crowdstrike blunder is not about providing false financial information to investors, but about a grave operational mistake. I believe the correct response from shareholders should be demanding a change of magement, not a financial claim.
Click to expand...
This disaster has more than likely doomed the company in the long or even medium term. The shareholders doubtless have realised this, hence their attempt to begin recovering as much of their investment as possible while there is still time, as well as to strengthen their legal position in future bankruptcy and/or liquidation proceedings.
 
You shouldn’t be pushing out changes all at once like this to production. You usually have a small section of low impact canary clients where you test updates in case something goes wrong.The fact that they pushed kernel level changes without proper testing is pretty mind buggling.
 
siegecrossbow said:
You shouldn’t be pushing out changes all at once like this to production. You usually have a small section of low impact canary clients where you test updates in case something goes wrong.The fact that they pushed kernel level changes without proper testing is pretty mind buggling.
Click to expand...
And customer-buggering.
 
The company routinely tests its software updates before pushing them out to customers, CrowdStrike said in the report. But on July 19, a bug in CrowdStrike’s cloud-based testing system — specifically, the part that runs validation checks on new updates prior to release — ended up allowing the software to be pushed out “despite containing problematic content data.”

The bad release was published just after midnight Eastern time on July 19, and rolled back an hour and a half later, at 1:27 a.m. Eastern, CrowdStrike said. But by then millions of computers had already automatically downloaded the faulty update. The issue affected only Windows devices, not Mac or Linux machines, and only those that were switched on and able to receive updates during those early morning hours.

Thanks to the timing of the incident, organizations in Europe and Asia “had more of their work day affected by the outage, unlike the Americas,” Fitch wrote in its blog post.

When Windows devices using CrowdStrike’s cybersecurity tools tried to access the flawed file, it caused an “out-of-bounds memory read” that “could not be gracefully handled, resulting in a Windows operating system crash,” CrowdStrike said.
Click to expand...

Shaking my head in disbelief. I'm reminded of the Simpson gene. And Homer having an alert on his nuclear monitoring console - and pouring water on it for the short-circuits to "solve" the problem: by silencing the alarm once and for all. D'oh : problem solved. Let's have some donuts.
 
tech.slashdot.org

CrowdStrike To Delta: Stop Pointing the Finger at Us - Slashdot

CrowdStrike says that it isn't to blame for Delta Air Lines' dayslong meltdown following the tech outage caused by the cybersecurity company, and that it isn't responsible for all of the money that the carrier says it lost. From a report: In a letter responding to the airline's recent public...
tech.slashdot.org

tech.slashdot.org

Microsoft Hits Back at Delta in Clash Over System Breakdown - Slashdot

Microsoft said Delta Air Lines turned down repeated offers for assistance following last month's catastrophic system outage, echoing claims by CrowdStrike in an increasingly contentious conflict between the carrier and its technology partners. From a report: Microsoft employees reached out to...
tech.slashdot.org


tech.slashdot.org

Parody Site ClownStrike Refused To Bow To CrowdStrike's Bogus DMCA Takedown - Slashdot

Parody site creator David Senk has rebuffed CrowdStrike's attempt to shut down his "ClownStrike" website, which lampoons the cybersecurity firm's role in a recent global IT outage. Senk swiftly contested the Digital Millennium Copyright Act takedown notice, asserting fair use for parody. When...
tech.slashdot.org
 
Last edited:
From the Business section [Markets] of THE TIMES (Irish edition, Saturday 24th August 2024)
Microsoft responds to outage with summit

Microsoft is to host a summit in September on improving cybersecurity systems, after a faulty update from Crowdstrike caused a global IT outage last month. The conference marks the first big step by Microsoft to address the issues that affected nearly 8.5 million Windows devices on July 19, disrupting operations across industries ranging from airlines to banks to healthcare. The event will be held on September at the company's headquarters in Redmond, Washington. "The Crowdstrike outage in July presents important lessons for us to apply as an ecosystem," Microsoft said. The outage raised concerns that many organisations are not well prepared to implement contingency plans when a single point of failure, such as an IT system, or a piece of software within it, goes down. "We look forward to bringing our perspective to the discussions on the need for a more resilient ecosystem," Crowdstrike said.
Click to expand...
No, you are not imagining things, either the writer or editor left out the actual date the conference is supposed to be taking place on.
 
tech.slashdot.org

How Should Cybersecurity Evolve After Crowdstrike's Outage? - Slashdot

Microsoft will meet with CrowdStrike and other security companies" on September 10, reports CNBC, to "discuss ways to evolve" the industry after a faulty CrowdStrike software update in July caused millions of Windows computers to crash: [An anonymous Microsoft executive] said participants at...
tech.slashdot.org
 
On Friday 19 July, 2024, the world woke up to what many have called the worst digital crisis of all time. A botched software update from cybersecurity giant CrowdStrike crashed some 8.5 million computers, smearing Microsoft's dreaded "blue screen of death" across the globe. Airlines cancelled over 46,000 flights in a single day, according to the FlightAware. Hospitals called off surgeries. 911 emergency services faced disruptions in the US. Film Forum, an arthouse cinema in New York, switched to cash payments as its credit card system went down. Microsoft and CrowdStrike issued a solution, but the disruption it caused continued for several days afterwards. It's a reminder, frustrated IT experts said, to never push updates out on a Friday.

"There's a price to pay for the convenience we enjoy," says Ritesh Kotak, a cybersecurity and technology analyst. "It will happen again, and from a technical standpoint, the fix for CrowdStrike was relatively easy. Next time, we might not be so lucky."
Click to expand...


In places where the internet connection depends on one fibre-optic cable, it makes for a glaring Achilles heel. After decades of the internet worming its way into every corner of our lives through wires and WiFi connections, you might think there would be more built-in fail-safes to keep the world churning. But largely the opposite is true, according to Casey Oppenheim, chief executive at Disconnect, a cybersecurity company.

"To me this is the real lesson of the CrowdStrike event," Oppenheim says. CrowdStrike holds a massive market share in its corner of the security business, serving more than half the companies on the Fortune 500 list. "The less diversity you have in any ecosystem, the more vulnerable you become, and there's zero diversity at the top of the internet supply chain. You can pick any core area of the internet and you'll find a very short list of companies in control."

In other words, Oppenheim says, the potential for catastrophic internet failures is yet another consequence of "monopolistic forces" in the tech business. When so much depends on a single company, one wrong move can bring it all tumbling down. "As governments take on antitrust issues, it's something we may want to think about," he says.
Click to expand...

Grey Havoc said:
www.bbc.com

The bizarre history of internet outages

In a world where a single point of failure can throw our machines into chaos, everything from sharks to authoritarian governments and old ladies have brought the web to its knees.
www.bbc.com www.bbc.com
Click to expand...
 
overscan (PaulMM) said:
So, our new security manager was talking about bringing in Crowdstrike where I work. Hopefully this idea will now be kiboshed after Crowdstrike deployed an update which

1) made Windows servers and desktops crash
2) rendered Windows servers and desktops unable to boot

The only fix is to boot the affected device to safe mode, log on with an admin account and delete a driver file here is full list. Most corporate computers affected will probably use Bitlocker, so you'll also need to obtain and enter a 48 digit key. Fun.

Some companies have thousands of servers and tens of thousands of laptops affected.
Click to expand...
The CrowdStrike update caused serious issues, crashing and preventing Windows devices from booting. Fixing it requires safe mode access, deleting a problematic driver, and entering BitLocker keys on protected machines. For companies with thousands of devices, this could cause widespread downtime. Impacted organizations should work with CrowdStrike support for a patch or automated fix and pause further deployments until resolved.
 

Similar threads

Back
Top Bottom