A compliant but also effective solution

Having no problems is the biggest problem of all.
Taiichi ÔNO

For at least ten years now, I have been telling prospects, students, employees, etc. that a security evaluation can be interpreted as an assessment of effectiveness in relation to security objectives. In other words, an evaluation (in the field of IT security) seeks to demonstrate that a product (or system) meets defined objectives in a compliant and effective manner.
The day after my eldest daughter’s birthday, barely recovered from the gargantuan meal… I had to answer some questions (which I consider relevant after reflection) about the final objectives of a security evaluation: “How does an evaluation prove the effectiveness of a product or solution? What exactly are you talking about? Why talk about compliance and effectiveness? How does a security evaluation prove any compliance and/or effectiveness?”
Oh my God…!! Why are you asking me these questions? Why now? I thought I could rely quietly on my achievements… My explanation seemed so clear to me… To find an appropriate answer, I’ll contact a friend: the ANSSIAgence Nationale de la Sécurité des Systèmes d’Information. The National Cybersecurity Agency of France (ANSSI) is a French government organisation reporting to the Secretary General for Defence and National Security... More (the French National Cybersecurity Agency) who, in its catalog of qualified solutions, states clearly and explicitly that: “the evaluation consists mainly of two components: a product compliance analysis […] a vulnerability analysis […]”.
Phew! I am relieved, I haven’t broadcast a false message for all these years, I will be able to continue to sleep with both my eyes closed. However, I have to admit that I may have been a little too quick… my explanation of the objectives of a security evaluation did not seem to be very clear (compliance of my purpose but no effectiveness a priori).
Indeed, from my point of view, in the testing phase, a security evaluation must include two types of tests: the so-called compliance tests, where the target is functionally evaluated, and the so-called robustness tests, during which the target is put in default. To summarize, compliance tests are performed from the “developer” perspective while vulnerability tests are performed from the “attacker” perspective. As a reminder, security evaluation work is always carried out in a specific operational context, i.e. on the basis of a clearly defined platform.
Let’s take a very simple example and suppose the following security objective: “the product to be evaluated must protect the confidentiality of the link between A and B, where A and B are in an uncontrolled network.” Now, let’s suppose that I developed the JCVD product (aka The Muscles from Brussels) that protects the link between A and B using the cryptographic primitive DES (Data Encryption Standard). During the evaluation, it is certain that JCVD will comply with the objective because the confidentiality of the link is protected (if and only if the DES primitive implemented in JCVD corresponds to its own specifications). However, JCVD is not at all effective because it uses a primitive that is not robust (and not recommended). Therefore, JCVD is compliant but not effective so the evaluation will conclude that the product has failed. On the other hand, if, in the JCVD product, the DES primitive was replaced by a more robust primitive recommended by the ANSSI, the evaluation could validate a priori the effectiveness of confidentiality protection (and therefore a possible success of the evaluation if the primitive complies with its specifications).
In conclusion, a product (a system) can be compliant with the objectives without necessarily being effective. On the other hand, the reciprocal is impossible (and has no sense).
Let’s now consider a real case. The security functions dedicated to the identification, authenticationPrimary or secondary authentication Authentication allows a user to guarantee his or her identity before accessing a resource or service. Primary authentication will give user access to the workstation (Windows... More and traceability implemented in the IPdiva Secure 8.0 solution from Systancia, have been evaluated according to the French CSPNCertification de Sécurité de Premier Niveau. The First Level Security Certification (CSPN), awarded by the ANSSI (French National Agency for the Security of Information Systems) aims to certify the robustness... More (Certification de Sécurité de Premier Niveau) procedure. The evaluation work has shown that Systancia Gate, formerly IPdiva Secure, provides an adequate (compliant) and efficient (effective) response by protecting the user authentication and access control to protected resources of an information system.
Therefore, I insist and emphasize, the success of an evaluation proves the compliance and effectiveness of a target in an operational context at a given time with respect to a security issue (provided that the hypothesis relating to the context of use of the target or its environment are respected). In conclusion, the CSPN certification issued by the ANSSI (based on evaluation work controlled and validated by the ANSSI certification center) certifies that Systancia’s IPdiva Secure 8.0 solution offers proven features and a proven level of security to the security issue defined in its security target defining the security objectives.