Breaking a secure product is not big news. Breaking a certified product is less common, so it makes the news. Now, the reactions are worth analyzing, because it is very hard to figure out what certification is about, in particular when related to security. Certification schemes are not perfect, but they still have plenty of advantages.

I am not going to describe the attack. All we need to know is that (1) the crypto algorithm in the USB drive was not borken, and (2) the authentication has been broken, but not by a brute force attack.

Now, about the certification. We are here talking about a FIPS 140-2, Level 2 certification. First advantage of the certification, some trace is available on the Web, giving very a brief summary of the evaluation results, as well as a security policy.

Of course, these documents tell us many things. For instance, that authentication is password-based:

Password Only: The module persistently stores the password hashed and AES encrypted in FLASH, which is outside of the cryptographic boundary.

Then, it defines the strength of the mechanism:

The probability that a random attempt will succeed or a false acceptance will occur is 1/62^4, which is less than 1/1,000,000. The user is locked out after no more than 100 contiguous login failures, therefore the random success rate for multiple retries is 1/62^4 divided by a customer specified number ≤ 1001, which is less than 1/100,000.

OK. The “62^4″ seems to mean that a 4-character alphanumeric password is required (26 letters, uppercase + lowercase, + 10 digits, that makes 62 symbols), which represents a bit over 14 million combinations. I don’t really get the “customer specified number ≤ 1001″ bit, but divided 14 millions by 100 (possible tries) yields a number greater than 100000, so the statement is about right.

Next, we can look at the certificate, which states:

Roles, Services, and Authntication: Level 2

So, this means that this authentication mechanism has been evaluated, and that it satisfies Level 2. I am not a FIPS140-2 specialist, so I need to look at the certification’s definition document what that means:

For Security Level 2, a cryptographic module shall employ role-based authentication to control access to the module.

So far, so good. The roles are here basic, but they are present. Of course, the specification says more than that. Here are the basic requirements for authentication:

The strength of the authentication mechanism shall conform to the following specifications:

• For each attempt to use the authentication mechanism, the probability shall be less than one in 1,000,000 that a random attempt will succeed or a false acceptance will occur (e.g., guessing a password or PIN, false acceptance error rate of a biometric device, or some combination of authentication methods).
• For multiple attempts to use the authentication mechanism during a one-minute period, the probability shall be less than one in 100,000 that a random attempt will succeed or a false acceptance will occur.
• Feedback of authentication data to an operator shall be obscured during authentication (e.g., no visible display of characters when entering a password).
• Feedback provided to an operator during an attempted authentication shall not weaken the strength of the authentication mechanism.

This definitely reminds us of the security policy, which satisfies all these rules. Now, what is missing? Well, the attack that succeeded is missing. There is no mention of the protection of the authentication mechanism against tampering. The standard only defines the properties of the normal use of the authentication mechanism, but not the properties of its defenses.

This is because FIPS140-2 focuses on cryptography, and not on ancillary functions. Maybe that a Common Criteria (CC) certification would have been better at finding this flaw, because CC looks at all security functions, beyond cryptography, and all labs know that authentication often is the weakest point of the crypto products. Of course, we will never know what CC would have found.

Now, we know one good point about certification: it only took us a few minutes to find the relevant information (security policy, certificate, and FIPS140-2 definition). And a security-literate computer professional would only need a few minutes to understand what features have been analyzed or not. Of course, very few people ever do that kind of analysis.

One of Schneier’s commenters mentions the fact that these products don’t really deserve to be blamed because “to err is human” and the vendors have acted swiftly after the discovery of the flaw.

Well, this may also be a very good point about certification. When NIST sees headlines about FIPS-certified products getting broken, it is quite likely that they take some action to analyze the problem, and that they get in contact with the vendor. And that’s a pretty good incentive in favor of action from these vendors.

So, overall, the certification brought some advantages to the customers, even to those who do not understand much about security. And the other good news is that now, their product is likely to be more secure, once they will have applied the corrective actions suggested by the vendors.