On the road to Bandol

Java Card performance has always been a contentious issue: first, between the Java Card vendors; then, between the Java Card supporters and the native card supporters. The issue is sensitive, because it is a commercial argument, which has been used and misused across the years.

The main issue is that implementors have to make trade-offs between several important parameters:

• Speed. Many factors influence the speed of the platform, which can be an important differentiation factor.
• Security. Security often implies redundancy, which in many cases contradicts performance. One exception, though: security measures can be controlled finely in assembly code, which also favors speed.
• Compliance. Compliance to all specifications and recommendations can be costly, and little “cheats” can ease both speed and security. This is often tempting, especially in corner cases, although it is a problem for application portability.

For compliance, there exist test suites, the most important one being Sun’s TCK, that all licensees should pass before delivering any product. For security, there exist several evaluation schemes, some standard (Common Criteria), some private (such as those performed in the banking industry). But for speed, there is no standard test. This is an issue, because it makes it impossible to make a complete assessment of a platform. For instance, a platform that is somewhat weak on security (but not too bad) can be a noteworthy quality product if it achieves good computing speed, which is crucial in some markets.
There has been several works by individual researchers and engineers:

• In 2000, Castella et al. [Cas00]have compared the relative performance of 4 cards (Bull’s Odissey, Sclumberger’s Cyberflex, Gemplus’ GemXpresso, and G&D’s SmartCafé).
• In 2005, a Master’s thesis by Karima Rehioui [Reh05] (performed at IBM Zurich, in cooperation with INRIA) has asked interesting questions about the measurement of execution speed, and has provided interesting answers.

However, these works are individual contributions, which have been kept private. Pierre Paradinas, at CNAM, has a differnt objective: Since 2004, his objective is to develop a reference banchmark for Java Card, and to use it on most major implementations. Some initial results have been published in 2004 [Par04], and the continuation of the work is the MESURE project, which is a collaboration between CNAM (Pierre Paradinas), the University of Lille (Gilles Grimaud), and Trusted Labs (myself). Our objective is to develop a benchmark and to open source it at the end of the project. It has started last May, and it goes on until the end of 2007. More news will be available on this site.