city Bonn
Date 29.07.2024

Following the global IT disruptions on 19 July 2024, the German Federal Office for Information Security (BSI) has developed initial measures in discussion with the software vendors Crowdstrike and Microsoft to prevent similar incidents in the future. The BSI will also enter into discussions with other software vendors to further develop these measures accordingly.

In addition to a short-term analysis of the security incident, particularly with a focus on to what extent German customers were affected, BSI expects an in-depth technical analysis of the exact cause. In addition, BSI will stipulate measures with Crowdstrike to ensure operational stability of customer systems, even when installing software updates required at short notice. BSI will review the effectiveness of measures already implemented.

In discussions with Microsoft, Crowdstrike, and providers of comparable software solutions, BSI pursues the objective that the respective operating system can always be started at least in safe mode, even in the event of serious malfunctions. This should facilitate resolving any errors in the future by affected parties. BSI's long-term goal is to have new and resilient components designed and implemented offering the same functionality and level of protection as before, but which require less invasive permissions to operating systems. This aims to minimize the impact of software errors.

BSI has been in direct contact with Crowdstrike in Germany and in the USA since the incident on 19 July 2024. Following the immediate measures taken by the software vendor to prevent further incidents and the provision of an initial workaround for the affected systems, preliminary analysis reports on this incident were continually discussed between Crowdstrike and BSI and subsequently published. Based on the discussions, the evaluation of the available analyses and continued feedback from the vendor, BSI has initially developed the following measures:

Short-term measures until 15 August 2024

• Impact analysis of the security incident in Germany
• Continuous tracking of the recovery rate of affected systems (as of 25 July 2024 21:54 CEST and according to Crowdstrike, 97 percent of all systems with Windows sensors are already back online)
• Merging already issued short-term warnings with expected incident-related CVEs based on the established CVD process

Medium-term measures until 30 September 2024

• Evaluation of the upcoming detailed and final analysis report (root cause analysis)
• Review of the current and the improved test concept of Crowdstrike by BSI in coordination with other international partner agencies and discussion of necessary adjustments with Crowdstrike
• Clarification of future measures to ensure a rapid rollout of business logic/signatures while strictly guaranteeing the operational stability of customer systems
• Testing the effectiveness of the progressive and closely monitored update rollout process to customers as already announced by Crowdstrike with extended telemetry analyses by Crowdstrike for immediate detection of faults after installation of the updates
• Raising the awareness of organizations using Crowdstrike products about fundamental operational risks (cf. https://www.crowdstrike.com/terms-and-conditions-de/) and creating sufficient operational redundancies for critical deployment scenarios

Long-term measures until 31 December 2024

• Discussion of concrete possibilities for evaluating the vendor's software development processes by independent third parties following BSI TR-03183 based on announcements already made by Crowdstrike
• Establishing a cooperation between BSI, Crowdstrike, and Microsoft with the objective to ensure booting of the system at least in a restricted mode, even in the event of serious malfunction of the EDR tool
• Initial discussions with all relevant stakeholders on the architecture of EDR tools to increase their resilience

Further measures in 2025

• Design and implementation of new, more resilient architectures for running EDR tools with the minimum required privileges while maintaining the same functionality and same level of protection
• Involving all other software vendors in this product category, all relevant operating system platforms and, in general, providers of products with (currently still) high privileges

BSI is in continued contact with the vendor Crowdstrike and with Microsoft regarding the operational and strategic processing of the security incident expecting concrete results and solutions. In the meantime, Crowdstrike has published a large amount of additional information that already describes the initial implementation of the above measures.

Press contact:

Federal Office for Information Security - Press Office
Telephone Number: +49 (0)228-999582-5777
Email: presse@bsi.bund.de
Website: www.bsi.bund.de

X: @BSI_Bund
Mastodon: @bsi@social.bund.de