Trusted Labs is involved in the Mesure project, whose goal is to develop open benchmarks for Java Card. There are few partners to this project (CNAM, INRIA/POPS, and us), and in particular, no major manufacturer. Yet, the idea is here to start a community that will publish some results. Performance is a sensitive piece information about any product, and even more about products that are never publicly tested and compared.
One good news is that other people are thinking about Mesure as a way to publish benchmarks. In particular, the people who developed random number generation benchmark (from the universities of Bordeaux and Limoges) mentioned in their presentation that their work could be associated to the results of Mesure.
Publication time is getting closer, and I hope that some people will actually use the benchmarks, and that this will lead to interesting results. However, there will also be a very strong challenge for us: will we be able to build a community around Mesure, in order to improve it and make it live? The publicly funded project will end soon, so this will become a true open source project. We will need more contributors, and I hope that the research teams involved in smart card research, as well as the industrial R&D teams, will help us build this community.
Performance measurement and quality measurements are simply a first step here, and there are many more things that could be shared between teams. What we need here is a platform, shared by many actors, and which provide shared services to all of us. I hope that the news will be good next year, with a few results derived from Mesure, and a few additions to the project.