In the following years, we wasted a lot of time working on a much more ambitious version of Java Card, which never made it, but we never worked on addressing this issue of unverified bytecode, mostly because we had the cryptographic protections provided by GlobalPlatform. If nobody can load unverified bytecode, where is the problem?

As an evaluator, this allowed me to reverse engineer many types of Java Card, as I was allowed to download unverified (and very much illegal) bytecode into cards. Over time, most virtual machines have included runtime countermeasures, and some implementations have been difficult to attack with illegal bytecode for quite a while.

Yet, Java Card has become an industry standard. Strong implementations exist, in which loading unverified bytecode is about impossible, and then, exploiting unverified bytecode is challenging. But on the other hand, weaker platforms also exist, and recently, following a combination of mistakes from various stakeholders, one of them was broken; luckily, by a researcher, not by a criminal.

The real surprise is not that this has happened. It is that it took over 25 years for it to happen. The lack of on-card verification has always been a weakness in the Java Card story. Over the past 25 years, there have been a few attempts to address the issue, but the industry never adopted them. Isn’t a solution to this issue a bit overdue?

The smart card industry is a small industry, with few actors, and Java Card has been a great success, its interoperability providing the basis for the deployment of billions of products every year. But the industry has been complacent, and although I am not part of it any more, I bear some of this responsibility since I have headed the Java Card Forum’s Technical Committee for a few years. I defended off-card bytecode verification many times, for instance in 2011 and quite vehemently in 2013.

But that was over 10 years ago, and I would now argue that it’s time to fix this issue. Despite minor updates, the core Java Card technology is over 25 years old. The similarity between the latest version of the Java Card Virtual Machine specification with its first interoperable version is striking. There have been almost no changes. The CAP file format is antiquated, and an update could allow all vendors to ensure that no illegal bytecode is allowed to run on the any card.

From a now outsider’s viewpoint, I believe that after the recent developments and an issue impacting the security of millions of unsuspecting users, the usual denials sound really out of touch with reality, so I am asking the Java Card community a simple question: Since you are pushing for Java Card to be the basis for our personal security, and asking us to trust you with our sensitive data, including our identity, isn’t it now the right time to put this bytecode issue behind us once and for all?

What? This includes my kids’ Honor-branded phones, and as far as I know, a significant portion of the kids in their middle school, as Honor has been proposing phones with a great value for a few years, and they are popular in that population who are usually not getting the top-of-the-line models.

I could also have titled this blog more provocatively as “Google denies basic security to their customers,” “Donald Trump throws millions of kids in hackers’ hands,” or “Evil Americans exercise extra-territorial power over people around the world.” There are plenty of opportunities here to be angry, but the problem is elsewhere.

There is here a trust and liability issue. When I buy an Android phone, I expect some service from the vendor, but I also expect some services from Google. In my professional life, I am battling for improving IoT security, making updates mandatory and secure, among other things. Until now, this was a battle against slackers and profiteers, but today, politics is getting in the way. If hackers benefit from this, who can be held liable? Is this just Huawei? Doesn’t Google share some responsibility for stopping their support? My kids have done nothing, for sure.

Most comments seem to emphasize that Google dealt a big blow to Huawei, but Google has also dealt a big blow to themselves: Huawei didn’t cut my kids’ updates, Google did. This really has some consequences on the Android model: When you buy a phone with Android, you introduce a dependency between you and both the device vendor and Google, and you will be a collateral victim if their relationship turns sour. This almost sounds like Apple; when you get an iPhone, you belong to Apple, but at least, only to Apple.

It makes me rethink seriously my dependency on Google, so it’s time to take some strong decisions. I will switch my family streaming subscription from Google Play Music to Spotify, just to make sure that my kids still enjoy music on their unsupported phones. And if this madness continues, I will move them to Huawei’s app store as well…

Terrorists are not easy to distinguish from normal drivers before it’s too late

In the real life of a security consultant, terrorists are a problem in a risk analysis. Whatever technology you think about can be somehow abused by terrorists, and terrorists are really an annoying kind of attacker:

• Terrorists don’t care about being caught or killed, which greatly limits the efficiency of many countermeasures, which are designed to make sure that the bad guys eventually get caught (like good logs). With terrorists in a car, the only countermeasure that works is the one that stops them immediately.
• Terrorists are often “bad users” rather than intruders, which means that countermeasures against them must be applied against users. If a car decides to crash itself following a perceived attack, it better be sure that it is actually driven by a terrorist, not just a careless driver.
• Any drastic countermeasure that is designed against terrorists may be misused by other attackers (or even by terrorists themselves) to create havoc. Self-destructing cars are not a pretty sight.

In the end, because of these and similar issues, in a classical risk analysis, terrorists are not listed among the bad guys, and if they are, they are explicitly ignored. Yet, Mark Cuban’s question makes sense, so should we do something about it? I have browsed some of the answers to his question, and I am now reaching my own conclusions:

• First, whatever we will do on vehicles now will not affect older vehicles, so terrorists will still have access to a large number of weapons for many years to come.
• The first consequence is that it is necessary to work on the infrastructure. In Nice, the Promenade des Anglais is now protected physically against weaponized vehicles. They have done a rather good job, using small poles and palm trees as obstacles.
• Also, infrastructure has an IoT component, such as the V2I (vehicle-to-infrastructure) communication. The infrastructure can therefore emit an emergency signal to surrounding vehicles when it detects a potential attack.
• Beyond responding to such infrastructure events, adding countermeasures in cars is difficult. The guy who drives on the sidewalk may be escaping from terrorists, so such countermeasures would still be hard to define.
• Any measure based on deep learning and analysis of drivers’ behavior is also very hard to define and enforce without moving straight to a police state.
• In the long term, full and mandatory automation sounds like a good countermeasure, which also addresses many more problems.

But then, none of this is easy. If we consider the emergency signal sent by the infrastructure in case of attack, there are many potential issues:

• If someone simply crashes into a barrier, who will decide that it is not an attack, and how long will that take?
• Emergency vehicles should not be directly affected by the emergency signal. But then, we need to ensure that terrorists don’t steal emergency vehicles.
• How can we refrain bad guys (terrorists or not) from sending emergency signal just to stop traffic?

There is always a trade-off between our protection and our freedom

In the end, I am not sure that we are ready yet to do anything to deter terrorists from weaponizing our vehicles. The main reason is that the security measures required to do so impose strong contraints on us. So far, we have accepted additional constraints in airports and planes, but we balk at a laptop ban in long-haul flights. And I am quite sure that our tolerance threshold in our cars is very low.

So, we should of course protect our cars from terrorists, but I am afraid that we will not do anything about it for now.

Alain Colmerauer passed away this week. I have plenty of memories about him, starting from classes with him in Marseille, where his way to present constraint programming was as strange as it was passionate. Even before that, during my studies at Marseille’s Groupe d’Intelligence Artificielle, heavy on logic and Prolog, Alain Colmerauer was the name that made us all dream about research and fame.

Constraint programming at Prologia was intended to solve practical problems, in scheduling, optimization, and other NP-complete problems. That was fun for me, but Alain didn’t care: for him, what counted most was the beauty of the language, the ability to describe the language with the simplest theory possible. Prolog III, the first constraint language he developed, used linear programming on rational numbers, which could not solve all problems, but was mathematically exact.

In Prolog IV, we wanted to solve more general problems, and we started using intervals on less exact floating-point numbers. Alain was not enthusiastic at first, but things got better when we realized that floating-point numbers are actually rational numbers (i.e., exact numbers, not some approximation).

While I was writing my dissertation, I spent some evenings with Alain discussing potential formalizations of our constraints, and we ended up defining a notion of approximation to map a set to a smaller set of approximated values, and building something on it. I was quite proud of it, and I still am, but Alain was disappointed by the fact that the properties defining an approximation was too complex.

For me, that’s the legacy of Alain Colmerauer: even in the most complex thing, program, or language, a simple and elegant view of it is carefully hidden, and can be uncovered if you look for it carefully enough.

RIP, Alain.

If the resources spent on interactive security, such as firewalls and antivirus and the like, were spent on improvements in the logical functioning of devices and a big improvement in quality of programming, we would get much better results
Whitfield Diffie

Close to 15 years ago, I was working on a static analyzer for Java ME, and still hoping that mobile devices could avoid the viruses and malware that were already plaguing PCs at that time. Well, that could have happened, but Symbian was already going bad. Smartphones nailed it: mobile would have malware.

The good thing with IoT is that there has been no suspense. Botnets have come very early in the game. The consequence is that everybody seems to assume today that connected devices will necessarily be ridden with bugs and vulnerabilities. That is a major factor pushing towards preventive security measures, because we now assume that there are problems everywhere.

The reason for my mobile optimism came from smart cards and Java Card. These things have been attacked, but malware has never been a problem. The reason is very simple: smart cards are very far from open. Only their provider can load software on them, and they make sure to protect this privilege.

The good news is that most connected devices are just as closed. New software can only (legally) come from their developers. And like smart cards, their software is often relatively simple, simple enough to deploy a few good security measures.

The bad news is that the current botnet epidemics is the consequence of blatant negligence in the development of connected devices (mostly because security is an externality). There will be bad devices.

In the end, we have three complementary approaches to addressing connected device security:

• Fight negligence. Enforce regulation to make vendors liable for the security problems they cause through their negligence.
• Deploy preventive measures. Deploy tools that detect attacks, mitigate them “live” as they happen, and learn about the attacks for the future.
• Make devices better. Spend more on the design and development of devices to reduce security vulnerabilities and make devices harder to attack.

We will need all three, for sure, but the priorities are difficult to set. It’s easy to guess that my preference goes to making devices better, and I am quite happy to hear that Whitfield Diffie leans in the same direction.

Metasploit is a well-known tool for web apps, and extending it to objects simply makes these objects as easy to hack as web apps. Indeed, there are many aspects in common between a Web App and a Connected Object: they can be reached from internet, they run complex software, they can be targeted by ransomware, and they can be misused.

Then, there are differences, and these differences make objects easier to hack than web apps. The most commonly cited is the lack of supervision of connected objects. This has allowed the creation of very large botnets of connected objects, and will most likely continue and worsen in the next few years, because few people realize that their fridge is in a botnet.

There is another difference, more subtle, but with great consequences on security: the objects’ availability. When targeting a web app, an attacker may know some of the software that has been used to build the app. He may even be aware of some vulnerabilities in this software. However, the attacker will know nothing of the product’s configuration, or of the security products and countermeasures deployed to protect it.

In order to design an attack, information must be gathered by probing the web app “live”, taking the risk of being detected by a sophisticated IDS or IPS, or of hitting a honeypot that will record his attack techniques. Metasploit helps by providing tools and a database of known issues, but such attacks still require great skills.

Now, let’s consider a connected object, even a complex one like a car. The attacker can tear the object apart, get access to memory chips, dump their content, and analyze it. This analysis will take time and resources, but there is no risk of getting caught, and countermeasures will eventually be exposed together with vulnerabilities. Metasploit will make the task even easier if the device includes known vulnerabilities. In the end, the attacker will obtain a viable attack path, by working in the security of a lab.

Detection is very difficult in such conditions. An IDS will protect a connected car, for instance, against random remote attacks. But a skilled attacker will only perform a live attack on a Connected Car after verifying that the IDS doesn’t catch it.

And this is only the beginning. Security research on connected objects is rather new, and focuses on the easiest targets. Skills will improve over time, making more sophisticated targets vulnerable. Now, let’s add to this equation an AI to assist in the reverse engineering and the identification of vulnerabilities. Traditional defenses are toast.

In a few years, if there is a vulnerability in a connected object, someone will find it and maybe exploit it. Encryption will provide a temporary protection. Trusted Execution Environments will add some hurdles. But these are just new countermeasures, not game changers.

That’s what I like in formally proven software. A mathematical proof that a piece of software does not leak information is a good countermeasure against reverse engineering AIs. With such a proof, finding a vulnerability is not about finding a software bug, it’s about finding an issue in a formal model that could lead to a wrong proof that could hide a bug. And that’s orders of magnitude more difficult, even for an AI.

Originally published on LinkedIn.

• Adi Shamir has made gloomy predictions about security in the next 15 years.
• Bruce Schneier has published a long essay about IoT security, with a vibrant and desperate call for action to governments.
• An AI is winning at poker against some of the best human players.

From these three sources, it sure looks like things will get worse before they get better. Let’s start by Shamir’s views on cybersecurity:

1. Cybersecurity is terrible, and it will get worse.
2. The Internet of Things will be a security disaster.
3. Cyber warfare will be the norm rather than the exception in conflicts.

All of this sounds very real today, so Shamir is just implying that it’s not going to get better any time soon. Schneier, on the other hand, is not giving up, by asking for a new government regulatory agency, and also for a body of public-interest technologists, who would be provide expertise into the public debate about technology.

That brings us to the third piece of news. An AI managed to win against really good poker players over several days. This requires strategy, bluffing, and much more. By extension, an AI can now pose as humans or beat humans in any narrowly defined situation, even when complex strategic thinking is required. OK, but even as I write it, I have a problem grasping the consequences of this sentence. I know a things or two about computing in general, and even about AI. Yet, it is very hard to see where that leads us, and how it applies to other fields, like cybersecurity.

So yes, Schneier is probably right in asking for regulatory agencies and public-interest technologists, because of the complexity of today’s technological issues. Shamir is probably right too, because Schneier is not going to get what he asks for, at least not in Trump’s USA (and I wouldn’t bet that we will get it in today’s Europe, either). So, what consequences does it have for “us”?

First, we have to stand strong ourselves. What we are doing at Prove & Run is right. Making devices stronger and more resilient/resistant to attacks is essential in the fight against attackers. And even if a majority of IoT actors don’t care, there remain enough responsible vendors to make a huge market for ProvenCore and applications.

Then, we need to remain humble. Stronger devices are not sufficient by themselves. 10 years ago, as an evaluator, I have used static analysis on Java Card programs to detect vulnerabilities that were very hard to find “manually”, with amazing success. So, if a small research team can program an AI that beats professional poker players, how long will it be before some team of hackers programs an AI that designs attacks on IoT systems? And if that happens, how much will our formal proofs matter? Even if our software is not broken, how easy will it be to bypass it?

French students learn in school about the great ligne Maginot, a very strong line of defense against Germany built in the early 1930’s. The Germans did not break it, they circumvented it.

I believe more strongly than ever that high-assurance security components and formally proven software are essential components of future secure systems. But we have to face a difficult challenge: make sure that no human or AI is able to bypass our highly resistant technology, effectively making it a 21st century ligne Maginot.

We can and will succeed. Our “We are the most secure” arguments are needed to attract our customers’ attention, but we must be careful to move to more complex “We are the foundations of the most secure systems” arguments as their understanding of the issues at stake improves and we get closer to implementation.

And let’s keep an eye on this AI thing.

Originally published on LinkedIn.

Traffic camera evidence incriminates a car, not a driver. The paper, written by a law professor, is interesting because it shows that the current processes around such evidence are not well covered by law (at least by U.S. law). When reading this paper, it becomes obvious that a “normal” citizen (i.e., not a law professor) would have great difficulties challenging the traffic camera evidence.

Of course, this becomes scary when we add to the mix the gazillions devices that are currently spying or reporting on us. Google Maps tells us that Bob’s phone was around here at 10:33, Bob’s alarm system tells us that his phone or his badge was used to turn off the alarm at 10:36 exactly, Bob’s security camera recorded something at 10:39 exactly, Bob’s fitness sensor tells us he ran between 10:43 and 10:47. I am not sure that police would be able to get all this data, but I am quite certain that they will be able to get some in the very near future.

And I am also quite certain that most of these devices are hackable, clonable, or that cybercriminals could misuse them to plant digital evidence against many people (especially someone sleeping alone at home). Not sure that this will get anywhere, because there are ways to make a lot of money misusing connected stuff without doing anything that sophisticated.

Yet, the Gone Girl story reminds us that some people are highly motivated to do such things, and most likely ready to pay good money for it. I am really wondering how the legal system will deal with the flow of cyberevidence and other IoT data, and how they will combine it with the possibility that most systems collecting this data are fully automated (no real witnesses), and subject to many hacks. It will be interesting to follow how this evidence will be trusted in courts, compared to traditional forensic evidence like fingerprints, DNA, and other things who can also be planted.

We are moving into a post-truth world, not only in politics, and this lack of certitude will deeply impact our society.

So, until some “velocity checks” (counters) are added to CVV2 validators, this is the lowest hanging card. The funny thing is that, since it only takes 6 seconds, changing the CVV every hour doesn’t really work, here, so the new Motion Code is not a good countermeasure.

Smart card hardliners will tell you that this isn’t a smart card issue. Sure, but it’s related, mostly because however high, there is always a lowest hanging card. Smart cards (with EMV) have been quite efficient at curbing card-present fraud, because the chip computes a dynamic verification code for every transaction. During the EMV rollout, fraud was taking place in countries where smart cards were not used. As this rollout is getting closer to completion, this opportunity is slowly going away.

The new lowest hanging fruit is online transactions. EMV doesn’t work online, mostly because all attempts to introduce card readers on normal PC’s have failed, so our smart cards are useless here. And because consumers haven’t been used to use their cards’ chips during online transactions, they won’t do it on mobile transactions either.

Sadly, commerce is moving online these days, so it is not good news to find the lowest hanging fruit there. The CVV2 check will be fixed, and more merchants will use 2-channel verification methods like “Verified by Visa”. Then, it is not obvious to know what will be the new lowest hanging fruit.

One solution is to use mobile payment, which offers a much better security these days. It works for in-person payments, and it is starting to work for online payments made on phones. I haven’t seen mobile payment used for to verify online transactions not made on a phone, but this would be very easy to do.

Not my problem.

That’s the 3-word definition of an externality: something that you don’t need to deal with, because the adverse consequences are not affecting you directly. This has been an issue for cybersecurity forever (Schneier, 2007), and it is widely known that the issue is particularly pressing with IoT (Schneier again, 2016).

I have been writing material on IoT device security as my day job for a few months now. But I also speak at conferences, where the public is more general: if there are device vendors in the room, I am not sure of what they think about this, so I mention the issue.

After doing so a few times, I got convinced that this “Externality of IoT Security” is rapidly becoming THE issue around IoT, and possibly the entire internet. We are deploying devices today that do not include any kind of security, because nobody cares.

We don’t care for a variety of reasons.

Some of us are clueless:

• Some vendors. Some vendors, initially many of them, have no idea that security could be a problem. Typically, this occurs with vendors of gadgets and other devices that don’t seem to be related to security in any way. Today, such cluelessness should become uncommon, as IoT-based attacks make headlines.
• Most users. That’s one of the scariest aspects of this. If a hacker ensures that pwned devices keep working normally most of the time, the end users will most likely not even notice a problem. Most IoT devices are unattended; if they work normally, there is no reason to worry.

Some of us are helpless:

• Merchants. Most devices are sold by some local/online store. Don’t expect these guys to provide much help, though. I tried to ask a few questions, and the best answer I got was a pointer to the vendor’s product support page (which said nothing about security, of course).
• Some users. Some users may worry about security, but their actions are quite limited, unless they have some understanding of security and computing.

Some of us optimize profits:

• Builders. The device builder is often some unknown company in China or elsewhere, which builds a device and sells it to vendors that will integrate it in their offer. At best, these guys follow the spec provided to them: they are low-cost companies, they cannot afford to work otherwise.
• Some vendors. The device vendor is the company with the brand name. They may care about security because of potential liability and bad publicity, but in most cases, this remains a bad economic calculation. Adding security will cost them more than not, so they don’t.
• Users. When making buying decisions, many users (including businesses and governments) will not include security as one of the important criteria, often favoring price. That of course pushes builders and vendors to ignore security.

Of course, not everybody thinks like that. Some companies include security in their priorities, or at least understand that security should be one of their priorities. This is a step forward, but they can still be a long way from secure products, because of two very common attitudes.

Some of us are in denial:

• Vendors. Denial is often visible when an attack occurs. This may of course be part of a communication strategy, but there are two strong indicators of denial. If the communication uses “highly sophisticated” to describe the attack, or “highly unlikely” to describe its exploits, then denial is likely. It is dangerous, because the vendor may then decide to minimize the impact of its actions, whereas in reality, most attacks we se today are rather simple, and most exploits are not implemented because “bad” business models remain unclear. For instance, insulin pump vendors have been in denial. Wirelessly controlled pumps can be hacked and could be used to kill people; this is not used today, most likely because this is not a tool that they are used to (although it has many advantages), but that can change any day.
• Users. Denial is obvious for many users, who know that there is a risk and will nevertheless not take any action. An example from outside of IoT is our behavior towards passwords, which are often much weaker than what our knowledge of security issues would mandate.

Some of us lack expertise (or don’t apply it appropriately):

• Vendors. Some systems include security measures, together with poor design or implementation. The Jeep attacks became legendary, some attacks on Philips Hue bulbs have been quite sophisticated but generated more buzz. In both cases, security measures were present (firmware update, authentication, usage restrictions and more), but their implementation was not sufficiently secure.This could be a lack of expertise, a lack of judgment, or a combination. In the end, the result is the same: they got hacked.

Denial and lack of expertise are different, of course. In these cases, security is not strictly considered as an externality. However, these attitudes also show an underestimation of their liability level, meaning that IoT security is not sufficiently considered as being their responsibility, and too much considered as an externality.

Today, poor evaluations of responsibility levels are the main blocking point for IoT security.

As long as most people and companies consider IoT security as somebody else’s problem or underestimate their required contribution, we will not see any significant progress. This is why externality is THE issue to be addressed today.

And this is no small issue. It is very easy and tempting to establish a parallel between IoT security and climate change. Climate change is an externality, and many powerful people are trying to address it. Yet, even with international treaties, it largely remains an externality for most actors.

IoT is not on the same scale in terms of life-threatening impact, but it is facing the same issue with similar actors. Global solutions are not going to come easy.