Your choice regarding cookies on this site
Click on Accept to agree or Preferences to view and choose your cookie settings.
Adam Banks, head of technology at the global transport and logistics giant, shares the inside story of the company’s crippling assault by the NotPetya malware — and its astonishing recovery.
It’s June 27, 2017. Adam Banks has just returned from honeymoon and is back in his role as chief technology and information officer of Maersk, the Danish transport and logistics giant, best known for its shipping containers.
Without warning, Banks finds himself in the position that no IT leader — indeed no executive — ever wishes to encounter: facing a serious malware attack that compromises most of the company’s systems and applications while wiping out its access to almost all of its data.
Two years on, Banks is willing to outline the scale of the destruction he encountered as what later become known as the NotPetya malware took hold and the company’s operations ground to a halt. “All end-user devices, including 49,000 laptops and print capability, were destroyed,” he says. “All of our 1,200 applications were inaccessible and approximately 1,000 were destroyed. Data was preserved on back-ups but the applications themselves couldn’t be restored from those as they would immediately have been re-infected. Around 3,500 of our 6,200 servers were destroyed — and again they couldn’t be reinstalled.”
|Adam Banks, head of technology, Maersk|
The cyber-attack also hit communications. All fixed line phones were inoperable due to the network damage and, because they'd been synchronized with Outlook, all contacts had been wiped from mobiles — severely hampering any kind of coordinated response.
Maersk was hardly the only company experiencing an IT meltdown at the hands of NotPetya: food and beverage manufacturer Mondelez, pharmaceutical giant Merck, advertising agency WPP, health and hygiene products maker Reckitt Benckiser, French construction company Saint-Gobain and FedEx’s European subsidiary TNT Express were among thousands of multinationals impacted.
Malware, not ransomware
Coming hot on the heels of the WannaCry ransomware surge in May 2017, many affected organizations assumed NotPetya was similarly designed to extort money. But as Nicholas Weaver, a computer security researcher at the International Computer Science Institute observed at the time: “Either it was a sophisticated actor who screwed up horribly on the part where they actually get paid — or it wasn’t about ransom in the first place.
If disruption was the motive, then NotPetya certainly achieved its goal. Set in motion by infecting an upgrade to MeDoc, Ukraine’s widely used tax software, NotPetya rapidly spread to more than 60 countries in Europe, the US and beyond.
“This code was built to destroy, not extort. The perpetrators wanted to destabilize the government in Ukraine, causing financial and social disruption. All companies that use the default software for submitting Ukrainian tax returns were compromised by the social engineering of a rogue employee,” Banks explains.
“The malware spread through the network, using a number of different methods,” he continues. “Our software at Maersk was patched appropriately but that only provided defense against one of the ways NotPetya was spreading. It exploited other weaknesses — not only technological but also procedural and behavioral.”
The virus, once activated, propagated within just seven minutes. And for Maersk, as with other organizations, the level of destruction was enormous.
As well as affecting print and file-sharing capabilities, laptops, apps and servers, NotPetya severely damaged both Maersk’s implementations of DHCP (Dynamic Host Configuration Protocol) and Active Directory, which, respectively, allow devices to participate in networks by allocating IP addresses and provide a directory lookup for valid addresses. Along with those, the technology controlling its access to cloud services was also damaged and became unstable while its service bus was completely lost.
Banks is candid about the breadth of the impact: “There was 100% destruction of anything based on Microsoft that was attached to the network.”
Resolving the crisis
Not surprisingly, there was no specific strategy in place at Maersk to deal with a cyber-attack on this scale. “There was no plan B,” explains Banks, “as the recovery plans didn’t account for the global destruction of everything — a common line of thought in asset-centric businesses.”
It quickly became clear that this was not a local management issue and the decision was made to abandon the corporate crisis management protocol in favor of a financial services model. Moreover, the senior leadership at Maersk decided to be as open as possible, internally and externally, about the situation — an approach that served the company well and has since garnered praise from institutions that include the World Economic Forum.
Banks and his team then moved quickly, focusing on reverse engineering the virus to understand how it worked. “We designed a new Windows build, based on Windows 10, that was less vulnerable to this specific virus,” he says. “We also strengthened it as much as possible so that it would be less vulnerable to other attacks.”
In a stroke of luck, they were able to retrieve an undamaged copy of its Active Directory from the Maersk office in Nigeria. It had been unaffected by NotPetya purely because of a power outage in Lagos that had taken the service offline while the virus was spreading. A rather nervous local Maersk employee was tasked with transporting the crucial data to Denmark.
Using that clean software, in days four to nine of the attack’s aftermath the Maersk IT team was able to rebuild the Active Directory as well as build out 2,000 new laptops and enable core business processes and systems. Intriguingly, the team even established contact with the individual who created NotPetya and gained insight into the malware that had caused so much destruction within its organization and beyond.
The team’s efforts paid off. Maersk became the first company in the world to reverse engineer the malware. Not only that, but the board also agreed to share this information with other affected organizations — an act of generosity that gave the firm a level of credibility that would prove useful.
It wasn’t all plain sailing. “We had to rethink one part of our plan,” says Banks. The initial strategy to avoid network congestion and move data and software using USB drivers was revised after Maersk “had exhausted all supplies of USB sticks within a 50km radius. In the end, we saved about three days of recovery time by propagating over networks. But we could only do this because we had built the credibility with a number of partners to use their corporate networks to distribute our new builds over.”
In the weeks that followed, the team that Banks had assembled continued to work though the long list of damaged and destroyed systems. After two weeks, all global applications had been restored. After four weeks, all 49,000 laptops had been rebuilt. More difficult was the restoration of non-global applications supporting regional processes. “This has been the most significant challenge in recovery,” admitted Banks.Counting the costs
Although the creator of NotPetya was focused on destruction, Banks believes that extortion was an influence too, with infected computers displaying a message demanding $300 in Bitcoin. Maersk was not among the payees.
Of course, this is not to say that the incident hasn’t been expensive for the firm. On the contrary, its chairman Jim Hagemann Snabe told the World Economic Forum in Davos that NotPetya cost Maersk between $250 million and $300 million. However, in Banks’s view, the policy of being transparent about the attack undoubtedly meant the losses were not even greater.
By being open about the malware and its effects on its systems, Maersk could be honest about the difficulties that it was facing and why — and leverage its reputation as a trusted global brand to overcome certain follow-on problems. In some cases, for instance, the company was allowed to retro-file customs paperwork. As a result, says Banks, “Despite the major disruption, around 95% of stock being carried [by its shipping containers] at the time got to the right place, more or less on time.”
Being candid about NotPetya also meant that Maersk could openly admit why it was suddenly recruiting an army of additional support staff. At its highest, it had taken on over 3,000 extra people to deal with the attack and its aftermath.
Since the incident, Banks has driven initiatives to educate all 88,000 Maersk employees to be cyber-aware. The firm’s manual workers in ports, for example, are encouraged to think about cybersecurity as well as safety.
In the time since NotPetya, Banks has been able to reflect on broader lessons for the tech-security industry. Crucially, he believes that the threat landscape has fundamentally changed. “The risk of being caught up in nation state activity is real,” he says. “And these sorts of cyber weapons are of orders of magnitude more damaging than traditional malware.
“You still need a perimeter to keep criminals out but you also need to think wider — you have to assume that the increasing number of state-level attacks are all going to be 100% successful.”
According to Banks, many common security measures are not robust enough to deal with this level of threat. Online back-up, for instance, is no longer safe: “You have to assume it’ll disappear,” he says. Patching is still necessary, but is insufficient: “And CEOs need to know this,” he warns. And cyber insurance alone isn’t enough, although it is worth having, he says, “Especially as the cost of policies is coming down.”
So how should organizations respond to such new threat levels?
“Prevention is unlikely to be an effective strategy,” observes Banks. “Automated detection and response are key. Automated protection is worth its weight in gold. And Privileged Access Management (PAM) takes on increasing importance. With a more limited number of privileged accounts, it is reasonable to assume that a much lower number of machines would have been infected, something like 5,000 rather than the 55,000 seen at Maersk," he asserts.
While historically boards might have dismissed the CIO or CISO after such an event, many are beginning to appreciate that doing so would not be an effective response. As seriously disruptive state-level malware becomes more frequent, the C-suite increasingly understands that a partnership is needed between it, IT and cybersecurity if such crises are to be dealt with successfully and avoided in the future. After all, muses Banks: “They realize that these kinds of attacks are not just a technology problem. They are a business problem.”
• Adam Banks was speaking at InfoSecurity Europe 2019 in London.
Click on Accept to agree or Preferences to view and choose your cookie settings.