[Spoiler Alert] Don’t read further unless you’ve finished watching the Mr. Robot season 2 finale (eps2.9_pyth0n-pt2.p7z). This article reveals some of the technical and plot points and hidden secrets of that gripping episode. “Thar be spoilers ahead, matey!”
I can’t believe this season is over.
LATEST IN A SERIES: Corey Nachreiner, CTO at Seattle-based WatchGuard Technologies, is reviewing episodes of Mr. Robot on GeekWire. The show airs on USA Network on Wednesdays at 10 p.m. Join the conversation on Twitter using #MrRobotRewind, and follow Corey @SecAdept.
In typical fashion, the Mr. Robot season two finale answered a few overarching questions, while introducing many new ones. Is Elliot dead (probably not)? How long have Tyrell and Angela been in cahoots? What the heck is Whiterose up to at the Washington Township Plant (WTP)?
Frankly, I can’t help you with any of those questions, but I can tell you how much hacking and technology Mr. Robot gets right. This episode only had one big proposed hack. Join me below to see whether the finale hack could actually happen in real life.
Hacking Paper Records Out of Existence
One of the big reveals, and the only hacking related plot point from this week’s finale, was Elliot finally learning/remembering his “Stage 2” plans for E Corp. As a reminder, during Stage 1, Elliot, Fsociety and the Dark Army launched a complex hack to wipe out (technically to encrypt) all of E Corp’s digital financial records, while also destroying their digital backups. Stage 2 is all about destroying E Corp’s corresponding paper records, thus totally leaving E Corp unable to prove anyone’s debt.
If you’ve ever taken a loan or bought a house, you realize there are tons of paper records associated with those transactions. While many modern companies digitize those records, they still have the original papers locked away somewhere. Elliot realized that in order to fully remove evidence of the proletariat’s debt to E Corp, he’d also need to take out those paper records. Luckily, E Corp is in the process of moving their paper records from many locations to one E Corp building in order to rebuild their data. Elliot just needs to figure out how to blow up that building, and all the records within. No big deal, right?
This episode reveals alter-Elliot’s Stage 2 plan, which is a hack designed to blow up E Corp’s building. Since the show only barely describes the scheme during Elliot’s realization monolog, I’ll start by laying out the hack step-by-step, then I’ll analyze whether or not it would actually work in the real world.
Firmware Hacks and Exploding Battery Backups
This entire hack hinges on network-connected uninterrupted power supply (UPS) systems. UPS devices are basically battery backups for computers. While most big businesses have backup generators, these systems take a few seconds to switch over, and computers don’t like even the shortest voltage sag. Between power outages and brownouts, businesses need short-term, power backups to keep their computers from switching off during these power interruptions, and that’s what UPS systems provide.
Though they look pretty high-tech, a UPS is basically a bunch of car batteries connected to a small computer that controls their output. The “lead-acid” batteries many UPSes use, though relatively stable, have chemical properties that can be dangerous. Elliot’s hack is all about overwriting the software (technically called firmware) that controls these devices with a malicious version that will disable those fail safes and enable those dangerous reactions.
Since UPSes also have network cards for remote management, and Fsociety already planted a Femtocell device on E Corp’s internal network, Elliot and Tyrell have a backdoor that allows them to access E Corp’s UPSes and update them with malicious firmware.
To connect all the dots, here are the steps to this hack:
- Elliot accesses E Corp’s internal network through the planted Femtocell.
- That internal access allows him to communicate with E Corp’s UPSes and remotely update them with malicious firmware.
- His malicious firmware disables charging-related fail safes, thus allowing the UPS systems to overcharge.
- Overcharged lead-acid batteries cause a chemical reaction that produces hydrogen gas, which Elliot will allow to build up.
- Finally, his malicious UPS firmware generates a condition that causes an element in the UPS to spark, thus igniting the hydrogen, and presumably causing a big enough explosion to level the building.
Now that we know the proposed hack, the question is; could it happen in real life?
Can a UPS Hack Level a Building?
The tl;dr answer is, many aspects of this attack are technically accurate and theoretically possible, however, I’m not convinced the chemical reaction caused by a UPS battery — or even a bunch of UPSes — would be forceful enough to level a building. Let’s break down the individual details.
There are essentially four big questions to this attack:
- Can remote external hackers gain enough access to control and modify UPS devices?
- Could a UPS actually explode, causing physical damage?
- Can modified UPS firmware disable enough fail safes to allow for this physical reaction?
- Assuming you could hack a UPS, or many UPSes to explode, would that explosion actually level a building?
If you’ve followed the Rewind articles, you know that we’ve already seen Fsociety gain access to E Corp’s network during a realistic femtocell hack. Though the main point of that hack was to hijack the FBI’s cellular phones, the femtocell also had a Wi-Fi adapter, allowing it to act as a “rogue access point” on E Corp’s network. With the femtocell in place, Fsociety (and the Dark Army) have wireless access to E Corp’s internal networks, which might give them easy access to E Corp’s UPS Systems.
That said, the jump from the “internal network” to “UPS systems” may be more difficult than the show suggests. When an attacker gains access to your internal network he does have a privileged vantage point with which to launch further attacks. However, there are internal security practices that could make the attacker’s job harder.
For instance, smart companies segment their internal networks and place network security controls between internal networks. Smart administrators might place their UPS systems on a separate IT management network. Even though the femtocell gives Elliot and friends access to one of E Corp’s internal networks, they’d still need to figure out how to jump to other networks, and evade any security controls between them.
Furthermore, even if Elliot can access the UPS devices, chances are E Corp requires some authentication to manage them. Elliot would also have to either steal a valid user’s credentials or find a flaw in the UPS management interface that allows him to bypass authentication before loading malicious firmware. In short, in the real world, hijacking these UPS systems — even with internal network access — would take a few more steps than the episode shows.
All that said, I’m still giving Mr. Robot a pass on this part of that hack. It’s 100 percent accurate. The truth is, once an attacker has control of one internal device, it’s just a matter of time before the rest of your network falls. By sniffing the network for Windows credentials, leveraging pass-the-hash, and other hacking tricks, an internal attacker can eventually enumerate your entire network, and elevate his privilege enough to control just about everything. In the security world, we call these internal hacking techniques “lateral movement” or “pivoting.” While the show doesn’t explain all the steps necessary for an attacker to get from the femtocell to the UPS systems, you can presume that it’s possible.
That brings us to the second question; can a UPS device explode, or at least have a dangerous physical reaction? In short, the answer is yes.
I already mentioned that the lead-acid battery in a UPS system is basically just a car battery. While these batteries are pretty safe overall, they’re still a bundle of chemicals that can explode or combust in certain situations. As the show accurately points out, if you overcharge a lead-acid battery, it generates hydrogen gas, which is highly flammable. You can read about this explosion risk here. In fact, this video shows what happens if you charge a car battery around sparks. This combustion/explosion risk isn’t limited to lead-acid batteries. Many of the lithium-ion and lithium-polymer batteries used in electronics today include chemicals that have dangerous potential, as Samsung recently learned with their Galaxy Note 7. Simply put, in certain situations, a UPS theoretically could combust or explode.
Next up, can malicious UPS firmware disable enough fail safes to allow for this explosive situation? This is where we step away from proven fact and go into theory.
The truth is, I don’t know enough about UPS design to know whether or not malicious firmware can disable enough safety features to cause a battery to catch fire or explode. That said, well-known hackers have tried similar hacks in the past and have failed.
In 2011, Charlie Miller — a white hat hacker who’s become famous for car hacking — showed how you could hack the firmware of a Macbook battery. His original goal was to cause a laptop battery to catch fire or explode. In the end, he found many flaws that allowed him to deliver malware from a hacked battery firmware, but he failed to cause the battery to catch fire (despite some incendiary headlines).
Hacking a device’s firmware for fun and profit is very possible. Researchers and attackers have planted malicious firmware on many devices in the past. In fact, the show even illustrates some very realistic firmware hacking tools. You may have missed it during the quick screenshots, but this episode highlighted a real APC UPS firmware you can download yourself (apc_hw05_aos_640.bin). The show also illustrated two tools hackers might use to analyze and reverse engineer firmware:
- Binwalk – a tool used to analyze the properties of a binary file, such as a firmware image.
- Radare2 – a tool for disassembling and reverse engineering a binary file, like a firmware image.
If you run the actual Binwalk and Radare2 commands the episode showed on the real firmware file, you’ll get the same result. However, just because these tools are grounded in reality doesn’t mean that you can modify this UPS’s firmware to actually generate hydrogen gas and cause sparks. So, I don’t know if that is possible or not.
One thing I do know is that it would be relatively easy to prevent. If I designed a UPS device, I would simply put a voltage measurement circuit outside of the firmware’s control. If this voltage measurement circuit detected a fully charged battery, it would prevent overcharging even if the firmware was telling you to continue to charge the battery. That alone should prevent malicious firmware from having this sort of disastrous result.
In the end, I simply haven’t analyzed APC’s UPS firmware enough to know if the software is capable of putting the battery in this dangerous state. Nonetheless, let’s give the show the benefit of the doubt, and say it’s theoretically possible.
This brings us to the final question. Assuming a firmware hack could cause a UPS battery to explode, could a bunch of UPS devices explode with enough force to level a New York skyscraper, or cause a big enough fire to weaken the structural support and level the building?
Frankly, this is where my expertise fails me. I’m a hacking and information security expert, not a chemistry and physics genius. Honestly, I don’t know how much hydrogen a lead-acid battery produces when left overcharging. Nor do I know how big an explosion would be necessary to take out the support beams of a sky scrapper. That said, I have seen videos of car batteries, UPS batteries, and lithium-polymer batteries catching fire and exploding. These explosions are certainly fire hazards, and dangerous to any humans in vicinity. Nevertheless, I’m still not convinced a UPS explosion would destroy an entire building, especially considering the automatic fire suppressant systems in most modern buildings.
Needless to say, this has been one of the hardest Mr. Robot hackuracy assessments for me. On the technology side of things, the show does illustrate firmware hacking realistically. The tools used in the episode are completely real, and attackers have targeted firmware in the past. Furthermore, the batteries used in many tech devices do contain chemicals that have dangerous properties and have caused some physical reactions, including fires. Still, I just don’t believe these UPS devices alone could cause a big enough reaction to produce the intended effect. Perhaps we’ll later learn the battery room was near a gas main or some other critical system.
Realistic or not, Mr. Robot did include enough real world technology in this hack that it’s easy for me to continue to suspend my disbelief.
Many other subtle secrets and tech notes.
While the finale only included one hack, there were many other cool hidden secrets and technical details in this episode.
- Did you notice all the detail on the FBI investigation board? This board included two new IP addresses that lead to new Easter egg sites that are likely part of the overall Mr. Robot ARG.
- Don’t forget to check out the email addresses on the FBI board. In many cases, if you send messages to these addresses you’ll get a reply.
- Besides using the Binwalk and Radare2 commands to reverse engineer a UPS firmware, the episode also shows Elliot using a “Shred” command to delete the malicious UPS firmware (before he was rudely shot). Shred is a real Linux application that securely deletes a file. The normal delete command doesn’t really remove a file from your hard drive. If you really want to remove files so others can’t recover them, you should use tools like Shred.
Learning from malicious firmware and Rogue femtocells
Whether or not this hack could really happen, the various Internet of Thing (IoT) devices we use today really are susceptible to hacks. Hackers do create malicious versions of firmware that they can upload to our devices. So how do you protect yourself from this?
First of all, make sure to update the firmware on your devices at least a couple times a year. You’re probably very used to patching your software and operating systems to prevent vulnerabilities. You need to do the same thing for your hardware devices by making sure their firmware is always up-to-date. This will help you avoid falling victim to known flaws that hackers can use to hijack your IoT devices.
Be on the lookout for Rogue APs. In this attack, Elliot leveraged the malicious femtocell to gain access to E Corp’s internal network. In this case, the femtocell was acting as a rogue access point, giving Dark Army and Elliot wireless access to E Corp’s internal network. You can avoid this type of situation by scanning for Rogue APs. Read up on the latest security solutions that can help you find Rogue APs on your network, and disable them to prevent external hackers from wirelessly hijacking your internal network.
Well, that’s it for this year. This was another great season and I look forward to more realistic tech and hacks next year. I hope you join me next season for another round of Mr. Robot Rewind. In the meantime, keep an eye out for my prediction review article, and a few others on hacking and infosec. As always, I look forward to your comments, theories, and feedback below.
