Security Metrics

I have done a quick survey on best practices of measuring perfomance of security management. Since it is a quite interesting topic for many professionals in the field of Security Management, I have made a wrap up of what I found out.

General Introduction
Security metrics or key performance indicators try to give an answer to business related questions like:

• How much better is my security relative to last year?
  
• Which differences are in the different security domains
  
• Are my security euros well spent? Which measures were effective?
  
• How do I perform relative to peers?
  

These questions are hardly adressed by security experts, but are needed to justify the investments and build support for new investments.

Today existing approaches to security measures are still abstract. The need to measure and quantify the level of security and spendings are well recognized, but most of the times a security expert is needed to translate an approach to a practice. To implement metrics without a clear control model is good for statistics, but does not add to the business.

A first measurement is needed as a starting point, in case the level of security is not defined in absolute figures. Trends are relative to this first measurement. When absolute figures are wanted a quantative risk analyses will be needed to find out whether loss expectnacies are reduced when controls are implemented.

Auditing
Measuring security can be done by audits against an accepted framework: COSO, CobiT, ISO/IEC 17799-2 . This gives an indication how the implementation of measures improves the level of security in a organization and the amount of control on security processes. In fact this is what certification bodies do at organizations which certify ainst ISO17799:2. An organization is benchmarked against the ISO-norm.

An alternative is the Gartner TCO approach, which provides benchmarks for peer organizations for the complete IT-organization, including security.

A practical approach is execution of penetration tests and security checks with automated tools.Doing this on a regular basis (with updated tools) provides insichts in weak spots and the speed of fixing them. Capgemini provides these services as Vulnerability Assesments or SecuCheck service offerings, but others do, too.

Indicators
There is not one list of handy indicators for security yet. However, there are approaches to get there:

• Top-down: the NIST has published a top-down approach to define and implement KPI's. In this approach, KPI's are targeted at security goals and are viewed in the context of business processes.


• Bottom-up: start with the indicators, which are already available. Get used to report and judge the indicators an improve in several iterations. See Geer/Soo Hoo/Jaquith .

I received some suggestions for indicators: the number of incidents (but what is an incident?), requests for authorizations, requests for password resets, number of infected platforms, etc. Most security managament tool report periodically their statistics.: Symantec Enterprise Security manager, Macaffee ePo Console, Safestone DetectIt, etc. Apart from these, obvious indicators are the financial figures: what is spend on security services, licenses, checks and audits, etc. In the US about 0,5% of the turn over of an organization is spent on security, which is considered low.

Another approach is to measure the perception of security: through questionaires and interviews an indication of the awareness of security incidents and risks and the way these change.

Regular testing
Of course trends can only be found if the indicators are measured and reporter regulary. Co-operate with the internal aditing department where possible, and try to have security indicators in the monthly service level report.

More info

• The Dutch GvIB has published a paper (in Dutch) about security KPI's.


• The site of securitymetrics.org provides papers and backgrounds in English on this subject