There is also intent. A vulnerability is only dangerous when an attacker actually decides to exploit it. The problem with intent is that it is definitely not obvious to measure, especially on new risks by new kinds of attackers. Here we can oppose two theses, between Bruce Schneier’s core theory from Click Here to Kill Everybody and James Andrew Lewis’ theory from his 2016 Managing Risk for the Internet of Things CSIS report.

Terrorists and Enemies

Lewis’ reasoning is that we have been promised major cyber disruptions on traditional internet for a long time and that we are still waiting to see one. His reasoning about terrorists is interesting, as he explains that terrorists tend to prefer tactics that include “direct action, bloodshed, and political drama.” I agree with him, but I still think that a terrorist group with the same financial means as the 9/11 commandos could very well use IoT today as an amplifier of their attacks, for instance by having a botnet contribute to the chaos by attacking key services.

The main difference between Lewis and Schneier, though, is about the likelihood of exploitation of IoT vulnerabilities in the context of war. Here, the assumptions are different, as Lewis considers that a massive cyber attack would be deterred by potential response from the U.S. whereas Schneier considers that (1) it could be useful in the case of an already started war, and (2) that the difficulty to attribute an attack could lead to misguided retaliation or to the absence of retaliation.

Consequences

There are also a few significant differences between Lewis and Schneier on other topics, which I outline below:

• About consequences, Lewis mentions that “most vulnerabilities found on IoT devices lead to events that would qualify as pranks.” He acknowledges that botnets can be created, but he dismisses them by mentioning improved defenses against DDoS attacks. Schneier is much more cautious, and I would be as well. Botnets could be used for other things than traditional DDoS, for instance for attacking other vulnerable devices.
• About cyberwar, the same difference in considering only repetitions of existing attacks leads to similar differences, where Lewis dismisses the risk of potential consequences of a full-scale cyberwar.
• Finally, Lewis considers that the risk will decrease as we get more familiar with the technology, and our experience grows. This is partly true, but it is only valid if we build experience fast enough to offset the increase of risk due to continued deployment of new technologies, which is not obvious today.

At this level, we are talking about opinions and predictions. Depending on whether you believe that history repeats itself or that we always get interesting new things, the conclusions are different. Well, my motto for 2019 still is “The times, they are a changin’ “, so I believe in the unpredictable.

Does it matter?

Note that it doesn’t matter that much. The conclusion from James Lewis does not differ greatly from Bruce Schneier’s. In the end, he recommends that the government “can accelerate risk reduction with the same methods we use for general cybersecurity: research, liability, infrastructure and regulation.”

The IoT insecurity issue may not be of apocalyptic scale, but it nevertheless remains an issue that needs to be considered by governments.

This is not the new normal, just a pit stop on the way to decades and decades of deteriorating conditions.

Nothing that I didn’t know, but a nice way to remind us that talking about the “new normal” is completely wrong: Things will become much worse before they get any better. And the more we wait to act, the more painful the impact of changing climate will be.

As a citizen of a developed country and most likely a member of the world’s 1% richest, I am not doing much to curb our energy consumption. I proudly bike to work, I try to isolate my house, but I still put my entire family on an intercontinental flight for the vacations. I feel the need to act on the climate, but I don’t seem to be able to decide what to do. I can blame my government (easy), Donald Trump (easier), I can request action, but in terms of actions, I am most likely not doing my share.

I am not a climate expert, and that may explain my questioning. Maybe that I should ask one of these experts: What should I do, myself, very practically? What are you doing yourself?

Beyond climate change, we have other, smaller threats to face, like IoT (in)security. And this time, I am an expert. Interestingly, IoT security shares some characteristics with climate change, including at least:

• It’s a time bomb, as the insecure devices that we deploy today will still be around tomorrow, and may come to haunt us in a few years.
• The problem is global; anyone’s vulnerable device can be used to attack somebody else’s IT service, just like any person’s CO2 contributes in the same way to global warming.
• Many citizens understand the risk (security is a top concern for IoT), but very few know what to do to lower that risk.

As an IoT security expert and a user of IoT devices, then I need to ask myself the question: Eric, what do you do about IoT security?

• I monitor my network. I have a device at home that lets me know when unknown devices come on my network or when strange things happen. I caught a few things with this, so I am happy about it.
• I use diversified passwords, and sometimes, 2FA (two-factor authentication). However, it took me years to move away from bad practices; I am still not using 2FA wherever I can, and I am still not forcing my family members to use 2FA. I even haven’t changed at least one password that appears on haveibeenpwned . Overall, I am not too happy about this.
• I have no clue about the security of the devices I use. This is bad, but I am just a customer here: my hacking skills are rusty, so I am not going to pentest all the devices that I buy and deploy at home. I have no other way to know, and I am not happy about it.

Since the beginning of 2018, I moved into a job working on the definition of security certification for IoT. When I started, my perspective was to maximize security vertically, making products more secure; that sounded natural for a chip vendor with a strong security background. After only a few months, my priority is now to maximize security horizontally, reaching as many products as possible; that is just as good for my company because our high-security chipsets are useless in a world full of default passwords and other trivially exploitable vulnerabilities.

We need security certification, we need it to be as simple as possible, we need it to be as mandatory as possible, and we need it as soon as possible. Simple? Because some good people trying to implement IoT security fail at it, and we must help them. Mandatory? Because some people think that IoT security is not their problem, and we must force them to act. Soon? Because as the clock is ticking, vulnerable devices are accumulating.

Finally, what can you even if you are not an expert? You can try to apply some good practices, and you can also ask your elected representatives to act on behalf of the community. And as you’re at it, also ask them to act on climate change.

Until recently, life was easy, and we could define levels easily. Since 3 is a magic number for levels, here is a definition that I penned myself:

• Low. Risks on individual goods and personal assets.
• Medium. Risks on collective good and community assets.
• High. Risk on human lives.

This classification can be (and has been) criticized, but it shows the idea. If something can only hurt your belongings, it is less sensitive than a thing that can impact, let’s say, your city’s traffic lights, and much less sensitive than a device that can kill you. And of course, you expect to spend less money on certification for anything with low risk.

If you work on IoT devices, then it is easy to apply this classification. My connected toothbrush is low, so is my personal security camera. The school’s security camera is medium, though, and a hospital’s connected syringes are high.

But wait. Didn’t Mirai exploit cameras to take down community assets like OVH servers? That’s the IoT issue: my camera as a personal security device to protect my house has a low risk level, but the same camera as a member of a botnet has at least a medium risk level, possibly a high if the next bad guy decides to attack emergency services instead of Web hosting services.

This is very hard to capture in a 3-level unidimensional classification. Yet, as we move towards certification of IoT devices, we need to include this collective risk. It is not enough today to consider what a device is supposed to do (watch my house), but we must consider what the device could do after being hacked, and even more importantly, what a large number of the same device could do after being hacked.

Here are three examples with different risks:

• IT risk. Any permanently connected device can be targeted by a Mirai-like malware and end up attacking any part of our digital infrastructure.
• Human risk. Anything with a battery may be led to overheat and possibly explode, and multiplying this could lead to havoc and multiple injuries.
• Economic risk. Sending Brickerbot (which destroys what it infects) to a large number of simple but essential connected parts (for instance, car parts) could lead to a shortage of parts and major economic damage (for instance, if cars can’t be fixed).

These risks are hard to capture, but they are significant. However, it is just not possible to label every connected object as high risk, because certification constraints are too high.

One solution is to define a Low or Basic level that includes a significant level of protection against hacking and malicious exploitation. Even this apparently simple solution is hard to define, but thinking about the problem in such terms is already a big step ahead.

So, remember: A single connected device is cute, but collectively, they can be very dangerous.

When you think about an online service’s lifecycle, it all makes sense. The service is deployed on servers sitting in a secure data center, then to more servers if needed; then, the service is updated to a new version, possibly moved to a new cloud provider, and all of this is quite transparent to users. Basically, security must be part of the normal lifecycle, continuously adapting to the new hardware, new software, and new threat environment.

IoT security is different, of course. IoT is about objects, not services, and securing objects is different. Their hardware is fixed, and people buy an object with a set of features, not a service. Things are evolving, of course: Tesla is selling an Autopilot service, complete with security and functional updates. Many smaller devices also come with online services. For instance, a remote thermostat comes with a service that monitors meteorological data, in-house presence, and many other parameters to optimize target temperatures. For such services, security is, of course, a process, like for any other online service.

Yet, most people buy cars, even Tesla models; people buy connected thermostats, even if they are useless without the accompanying online service. For most consumer connected devices, the online service is free (for life, whatever that means), but the consumer has little guarantees that it will keep working for a long time. Naturally, professional contracts are a bit clearer: companies pay for the connected devices and for the related services, but they get additional guarantees, for instance, that the service will run and support their devices for at least 5 years.

Companies usually have a business view of it: a device is supposed to last for 5 years, so it is financed over 5 years, including support and associated services. After 5 years, the device is evaluated; either it is replaced with a new one, or it still works fine and its support and associated services can be continued for a few more years. The U.S. Senate is trying to formalize this for federal suppliers.

What happens when a consumer’s connected car ages?

So, what happens when a consumer’s connected object ages? Let’s consider a connected car, for example:

• For a few years, everything goes fine. The manufacturer regularly updates the car software. The features may evolve or not, but security threats are taken care of. The servers also keep working without problems.
• After a few years, things become more difficult. Some services start to disappear, either because they are outdated and the server doesn’t work anymore, or because they are flawed and cannot be fixed. But the car keeps working.
• Then someday, maybe after 10 years, a key hardware component gets defeated by hackers, to a point that software can’t fix/mitigate. From then on, the car is accessible to hackers. So, what should be done? Should the manufacturer disable the hardware (i.e. the car)? Should they be forced to design a replacement part based on more robust/recent hardware? But then, for how long?

Basically, the problem comes from the mismatch between hardware and software. Let’s make a parallel between computers and connected cars: In 2037, using Autopilot on a 2017 Tesla will be like running a 1997 version of Apache on a 200MHz Pentium Pro/Windows 95 machine in 2017: a very risky business.

The difference between consumer and business IoT is the ownership model (or more generally, the business model). Some people may always lease new cars and basically act like businesses, but some people keep their cars for 10 or 20 years, and some people buy used cars.

IoT security is a process is a service

Connecting cars is a great idea, but it doesn’t work well with the current car business model. There are many reasons that would push us not to own cars in the near future, and security is one of them: We shouldn’t own connected cars, and simply use a transportation service. Then, things become much clearer: It is the service provider’s duty to maintain the cars, software, and hardware, and to replace the cars when they are not secure/safe anymore.

More generally, IoT is a service, and IoT security is a process. The IoT « devices » are just a part of the hardware required to implement a given IoT service, even the big ones; they are just like the servers on the backend side, under the responsibility of the service provider, and their sourcing and maintenance should be under their responsibility.

These methodologies present many advantages, and one of the most obvious ones is their ability to have a quantitative side, where a value can be assigned to a risk or threat, or to a countermeasure, allowing management decisions to be taken.

Top-down is an owner’s view, not an attacker’s view

However, the top-down approach has some limits, especially when doing a threat analysis, and when getting closer to implementation. The top-down approach has two big problems, which limit how it models reality.

First, a top-down approach doesn’t faithfully represent the way in which attackers work, which is often much more opportunistic. An attacker often has an abstract goal, for instance to break into a system or harm some company’s reputation. This is because the attacker is in fact an attacker ecosystem, with many actors involved. Some people actually perform attacks on actual targets, but other actors identify vulnerabilities attack paths by investing in security research. The market for 0-day attacks on major software components is an example of this market. In that case, the researcher does not have any idea of the way in which the attack will be exploited, and doesn’t care.

Second, and maybe more importantly, a top-down analysis privileges intent over reality, and it can’t represent the blatant bugs that essential to many vulnerabilities. When a developer assesses the difficulty or cost of an attack, his view is necessarily optimistic. The attacker will not consider that he will make a stupid mistake, or that his algorithm is flawed. In real life, though, many vulnerabilities are linked to such situations.

Attackers are opportunistic

Let’s consider an example. We use an attack graph, because it better matches the reality of an attacker ecosystem, building complete attack paths from smaller attacks and vulnerabilities. Graphs can show how attacks are interconnected. If individual attack edges are weighted with the cost of the attack, then the attacker’s job is to identify the path with the smallest cost, or at least one of the smallest costs. This is shown on the left, with two paths that are better than others (total costs of 8 and 9, respectively), but the differences are not that great, with the costliest path at 11.

Now, look on the right. The graph has been corrected by simply taking into consideration one single stupid bug that makes one attack much easier than expected. And suddenly, a new path appears, that was not initially considered, because it was supposed unlikely, with a much lower total cost of 5. This minor change can lead developers to reconsider some of their hypotheses about threats.

And we have been nice, here. In real life, new attack edges and nodes are likely to be added to the attack graph, defining shortcuts to existing attacks, and often leading to completely unexpected attack paths.

Build attacks from low-level vulnerabilities

That’s where a bottom-up analysis can greatly help. The idea is to look at individual components and security measures, identify how they could be abused, what vulnerabilities are likely, in the most practical way possible. Such an analysis can be a bit messy, with many directions, so they will not yield a nice set of slides to show to a manager or customer. However, they are likely to identify a few interesting threats and attacks that cannot be identified through a top-down approach.

Later in the development cycle, similar results can be achieved through a security evaluation, especially in black-box testing. As the evaluators look at their target, they will have an opportunistic approach by first trying to find a few easy vulnerabilities.

[EDITED] Eric Diehl published a very good article about white-box testing vs. black-box testing, that I have to agree with. It changes a bit my last reference about black-box testing: white-box testing is generally more efficient than black-box testing, and if you are in a position to use it, a bounty program also is more efficient than black-box testing. Yet, I still believe that a foray into black-box testing can be useful to get a different insight into a product’s security.

If you are not familiar with threat analysis or modeling, try Adam Shostack’s Threat Modeling: Designing for Security. It really helped me when I started working on threats, and it provides good tools for a bottom-up analysis.

Metasploit is a well-known tool for web apps, and extending it to objects simply makes these objects as easy to hack as web apps. Indeed, there are many aspects in common between a Web App and a Connected Object: they can be reached from internet, they run complex software, they can be targeted by ransomware, and they can be misused.

Then, there are differences, and these differences make objects easier to hack than web apps. The most commonly cited is the lack of supervision of connected objects. This has allowed the creation of very large botnets of connected objects, and will most likely continue and worsen in the next few years, because few people realize that their fridge is in a botnet.

There is another difference, more subtle, but with great consequences on security: the objects’ availability. When targeting a web app, an attacker may know some of the software that has been used to build the app. He may even be aware of some vulnerabilities in this software. However, the attacker will know nothing of the product’s configuration, or of the security products and countermeasures deployed to protect it.

In order to design an attack, information must be gathered by probing the web app “live”, taking the risk of being detected by a sophisticated IDS or IPS, or of hitting a honeypot that will record his attack techniques. Metasploit helps by providing tools and a database of known issues, but such attacks still require great skills.

Now, let’s consider a connected object, even a complex one like a car. The attacker can tear the object apart, get access to memory chips, dump their content, and analyze it. This analysis will take time and resources, but there is no risk of getting caught, and countermeasures will eventually be exposed together with vulnerabilities. Metasploit will make the task even easier if the device includes known vulnerabilities. In the end, the attacker will obtain a viable attack path, by working in the security of a lab.

Detection is very difficult in such conditions. An IDS will protect a connected car, for instance, against random remote attacks. But a skilled attacker will only perform a live attack on a Connected Car after verifying that the IDS doesn’t catch it.

And this is only the beginning. Security research on connected objects is rather new, and focuses on the easiest targets. Skills will improve over time, making more sophisticated targets vulnerable. Now, let’s add to this equation an AI to assist in the reverse engineering and the identification of vulnerabilities. Traditional defenses are toast.

In a few years, if there is a vulnerability in a connected object, someone will find it and maybe exploit it. Encryption will provide a temporary protection. Trusted Execution Environments will add some hurdles. But these are just new countermeasures, not game changers.

That’s what I like in formally proven software. A mathematical proof that a piece of software does not leak information is a good countermeasure against reverse engineering AIs. With such a proof, finding a vulnerability is not about finding a software bug, it’s about finding an issue in a formal model that could lead to a wrong proof that could hide a bug. And that’s orders of magnitude more difficult, even for an AI.

Originally published on LinkedIn.

• Adi Shamir has made gloomy predictions about security in the next 15 years.
• Bruce Schneier has published a long essay about IoT security, with a vibrant and desperate call for action to governments.
• An AI is winning at poker against some of the best human players.

From these three sources, it sure looks like things will get worse before they get better. Let’s start by Shamir’s views on cybersecurity:

1. Cybersecurity is terrible, and it will get worse.
2. The Internet of Things will be a security disaster.
3. Cyber warfare will be the norm rather than the exception in conflicts.

All of this sounds very real today, so Shamir is just implying that it’s not going to get better any time soon. Schneier, on the other hand, is not giving up, by asking for a new government regulatory agency, and also for a body of public-interest technologists, who would be provide expertise into the public debate about technology.

That brings us to the third piece of news. An AI managed to win against really good poker players over several days. This requires strategy, bluffing, and much more. By extension, an AI can now pose as humans or beat humans in any narrowly defined situation, even when complex strategic thinking is required. OK, but even as I write it, I have a problem grasping the consequences of this sentence. I know a things or two about computing in general, and even about AI. Yet, it is very hard to see where that leads us, and how it applies to other fields, like cybersecurity.

So yes, Schneier is probably right in asking for regulatory agencies and public-interest technologists, because of the complexity of today’s technological issues. Shamir is probably right too, because Schneier is not going to get what he asks for, at least not in Trump’s USA (and I wouldn’t bet that we will get it in today’s Europe, either). So, what consequences does it have for “us”?

First, we have to stand strong ourselves. What we are doing at Prove & Run is right. Making devices stronger and more resilient/resistant to attacks is essential in the fight against attackers. And even if a majority of IoT actors don’t care, there remain enough responsible vendors to make a huge market for ProvenCore and applications.

Then, we need to remain humble. Stronger devices are not sufficient by themselves. 10 years ago, as an evaluator, I have used static analysis on Java Card programs to detect vulnerabilities that were very hard to find “manually”, with amazing success. So, if a small research team can program an AI that beats professional poker players, how long will it be before some team of hackers programs an AI that designs attacks on IoT systems? And if that happens, how much will our formal proofs matter? Even if our software is not broken, how easy will it be to bypass it?

French students learn in school about the great ligne Maginot, a very strong line of defense against Germany built in the early 1930’s. The Germans did not break it, they circumvented it.

I believe more strongly than ever that high-assurance security components and formally proven software are essential components of future secure systems. But we have to face a difficult challenge: make sure that no human or AI is able to bypass our highly resistant technology, effectively making it a 21st century ligne Maginot.

We can and will succeed. Our “We are the most secure” arguments are needed to attract our customers’ attention, but we must be careful to move to more complex “We are the foundations of the most secure systems” arguments as their understanding of the issues at stake improves and we get closer to implementation.

And let’s keep an eye on this AI thing.

Originally published on LinkedIn.

• The device performs its tasks normally, but the attacker also runs other software on it, for instance to make it participate to DDoS attacks.
• The device apparently works normally, but its behavior is somehow altered, for instance by providing wrong information.
• The device doesn’t work or obviously dysfunctions, for instance a car that won’t start.

A second factor to analyze the consequence of an attack is the duration of the attack, or the delay until recovery. Here, we can consider four situations:

• Until next reboot. I am confident that the firmware hasn’t been modified, so a reboot will remove the attack vector from RAM and address the problem.
• Until next hard reset. I am confident that the device will not boot on modified firmware, so a hard reset will restore a « good Â» copy of the last known correct firmware.
• Until next update. I am not sure about the full firmware, but I know that a firmware update will remain possible and will address the issue.
• Until replacement. The firmware update mechanism is corrupted as well (or the device is permanently damaged), so the only solution is to replace the device.

We could go further, but let’s stop here for now. The impact looks essential, but the differences between the three scenarios aren’t that great. An obvious dysfunction is bad, but a device whose behavior has been altered by an attacker can be just as bad, and a device that is controlled by a hacker can easily become dysfunctional. In all three situations, it is obvious that some fix is required.

That’s where duration and recovery are important. The two first options (reboot and hard reset) are only temporary fixes. Even if the attack is transient and can be removed, the vulnerabilities that made the attack possible are still present, and the attack can be reproduced easily. Ideally, a reboot or reset/restore must be accompanied with some operational restrictions (like a “safe mode”) until an update is performed.

What we are analyzing here with recovery methods is resilience, or according to Random House, “the power or ability to return to the original form, after being [attacked]”. It is a valuable property for systems that are submitted to high levels of stress. Resilience is about recovery, in opposition to resistance, which is about protection.

Resilience and resistance are complementary. Resilience is often considered easier to achieve than resistance, but maybe more importantly, resilience may still happen when resistance has failed. Even if an attack on a system has been successful, its recovery is possible is the system is resilient to that attack.

We can here use an analogy with resilience against natural disasters. Although a hurricane can cause massive destruction to coastal areas, (resistance is limited), most U.S. coastal areas are resilient against hurricanes because hurricane evacuation routes have been defined, and shelters are ready to host evacuated people. Such resilience measures rarely fail, and we are shocked when this occurs, like in New Orleans after Katrina.

For an embedded device, resilience is mostly about:

• Secure boot, to restart from a clean sheet upon reboot, and detect a potentially persistent attack.
• Firmware update, to load a new version that is not vulnerable to the attack.

More generally, resilience can be achieved through simple infrastructure means like roads and shelters. In the case of connected devices, resilience can be achieved through low-level features, within or below the operating system, and highly independent of the use case.

However, although resilience mechanisms are simple, they also need to be very robust. A hurricane shelter must be built on high ground with strong material like reinforced concrete that offer strong guarantees of resistance against a hurricane. Similarly, the secure boot and firmware updates of a connected device must be designed and implemented to resist attacks. In both cases, this resistance is easier to build because it is applied on a smaller scale, and it is mutualized:

• Smaller scale. A shelter is a single building, typically used for other purposes (like a stadium) that is relatively easy to reinforce. Similarly, a secure boot is a small mechanism, so is the security-critical subset of a firmware update mechanism. These mechanisms offer a small attack surface, and are therefore easier to protect than an entire software stack.
• Mutualized. A shelter is shared by all members of a community, who will use it together in case of emergency. Similarly, secure boot and firmware update are independent of the device’s role and application. The development and reinforcement efforts can therefore be mutualized, reducing the cost for every device.

Resilience must be proactive

Resilience is the capacity to recover from an attack, and as such, appears to be a highly reactive technology, that comes after the fact. It is of course true that resilience mechanisms are triggered after the fact, but in order to succeed, they need to be proactively prepared.

Just like evacuation routes and shelters need to be planned in advance and designed to resist a hurricane, resilience measures in IoT devices similarly need to be prepared beforehand. A firmware update mechanism is crucial, and it needs to be prepared with great care proactively. In particular, this mechanism must be designed to be highly resistant against attacks.

In the world of connected devices, we know that resistance to attacks will be difficult to achieve, because it requires a very large investment from vendors to fix all vulnerabilities and make their systems resistant to attacks. Resilience, on the other hand, can be achieved on a wide scale with a limited budget, and has the ability to provide a strong basis on which IoT security can gradually be built.