Security Metrics by Caroline Wong - A Book Review

Posted on January 20, 2026  •  12 minutes  • 2523 words

The Book and the Author

Caroline Wong is an information security professional with years of depth experiences with different corporations. Her slogan is “Making Cybersecurity Accessible to All.” She prepared some training content on LinkedIn. “Security Metrics, A Beginner’s Guide” has been published on November 2011 and available on Amazon. Her recent contribution to the field, a new book The AI Cybersecurity Handbook is going to be published on March 2026 by Wiley.

The book gives examples and deployment methodology of a security metrics program for corporations. It asks fundamental questions about metrics. For instance, if we calculate an aspect, what will be the impact for corporation or decision makers? If it will not serve any purpose for business, do we really need that metric?

Her guideline provides a really good project method especially for inter-departmental communication. Because, security is not just for Information Technology departments’ concern. So, it should cover all stakeholders’ requirements if it will be a long term project.

Infographic Credits: Microsoft Copilot

I’d like to thank Caroline Wong for her contributions to the cybersecurity field.

Those are the points that I’d like to highlight and cite from her book.

Citations from the book

Remember that although this is a book about security metrics, and security metrics are important, security metrics exist to serve security programs, and security programs exist to serve the business. (p.33)

Seatbelts prevent total ejections from a car during a crash; 75 percent of occupants who are totally ejected during a crash are killed (NHTSA). (p.37)

Under the Clinton administration, the “Presidential Initiative for Increasing Seat Belt Use Nationwide” set goals of 85 percent usage by 2000 and 90 percent usage by 2005. This is a great example of defining objectives and goals for use in tracking a metric. (p.38)

In developing a security metrics program, it is very important to know and understand the company’s risk tolerance. (p.40)

Information security can be thought of as a war or a game between two competing entities: the “good guys,” otherwise known as the information security team (as well as sponsors and stakeholders in an organization), and the “bad guys,” otherwise known as hackers, fraudsters, attackers, information stealers, and information abusers. However, the “game” of information security is unlike any other. There are no rules and regulations governing the actions of the “bad guys,” whereas the “good guys” are constrained by not only rules and regulations, but also corporate bureaucracy, schedules, and limited resources. (p.43)

Information security is not, and may never be, an exact or hard science. (p.43)

Managing information security is not like building a car, a house, or a website—it’s a job that’s never finished. (p.44)

Security experts formed the nonprofit Cloud Security Alliance in 2008, whose mission statement is, “To promote the use of best practices for providing security assurance within Cloud Computing, and provide education on the uses of Cloud Computing to help secure all other forms of computing.” (p.53)

When practitioners begin to put these ideas and recommendations into practice, they sometimes run into real challenges and discover that “best practice in theory” is not the same as “best practice in reality.” (p.58)

The most effective security controls are aligned with your specific business’s strategy and objectives. (p.60)

The most valuable analytics pass the “who cares?” test, meaning that there is a direct association between the result provided and a decision or action to be taken. (p.68)

Qualifying and quantifying security efforts is not easy to do, but implementing good project management methodologies is a step in the right direction. (p.94)

As with problem and solution statements, a statement of the risks of not implementing the project is most useful when it is specific. (p.101)

For information security especially, identifying the risks of not implementing the project is important not only because managing risk is a major objective of the information security program, but also because it informs and educates others in the organization about the threats that the information security team is addressing. Many people are oblivious to the work that we as security professionals do every day, so putting in writing what might seem obvious to us can help enlighten the organization. (p.101)

As the project progresses, project plans should always be updated to reflect more accurate timelines. (p.103)

One example of this type of report is patching reports that show on average how long it takes to patch systems. At first glance, this might sound like a meaningful report. But what can you do about the results of this report? How easy is it to determine what’s desirable or undesirable, and how easily can you clearly direct a change for the better. (p.116)

Making decisions or providing recommendations to executives about risk tolerance and mitigation, or creating and managing policy governance councils, should be in-house work. (p.116)

…offshoring refers to work done by resources in a different country (either third party or in-house). (p.117)

For information security teams that are under-resourced, leveraging global resources can result in a significant, cost-effective increase in the number of full-time workers. (p.118)

…the metrics for a security program focused on policy and architecture will be dissimilar from the metrics for a security program focused on operations. (p.123)

…at least one of the following objectives should be fulfilled as a result of measuring, analyzing, and reporting each of the targets that you identify. If one of these objectives will not be fulfilled by focusing on a particular target, then there likely isn’t a good reason for measuring it.

• Visibility: Information security metrics provide visibility into the current status or state of an existing area or process.

• Education and a common language: Information security metrics provide a lexicon for the information security team to communicate with and educate stakeholders and sponsors. (p.124)

• Improvement: Information security metrics improve an information security program by enabling better management, promoting informed decision-making, and driving change throughout the organization (p.124)

Another way to think about this is, how does my information security program add value to the business? Compliance is a very important factor to consider for public companies, companies that manage customer and employee information, and customers that manage credit card transactions. Risk management is another important factor. Security activities and projects can also manage and drive down risk for an organization. Finally ask yourself, how does my security program enable business. (p.124)

The specific metrics that are used to evaluate the controls and assign a color to each should be determined by the information security compliance team and can be collected and reported at an operational level. (p.125)

Going one level more granular and tactical than “Are we going to pass or fail?” may include additional, quantitative metrics questions such as “What percentage of the controls have been tested?” and “What percentage of the controls that have been tested successfully passed?” (p.125)

When attempting to identify targets for a metrics program by evaluating what’s important to an organization, an increasingly significant question is, “Which information security practices enable the business to operate, and how can these be measured?” (p.127)

I recommend considering basic information security program functions as initial targets for metrics, to ensure that you have the fundamentals in place. Then, if something bad happens, you can state matter-of-factly that your team has performed the basics, making it much less likely that the incident will be attributed to a lack of due diligence on your part. (p.132)

As a brainstorming exercise, we posed some questions for consideration:

• What current security projects are in progress?
• What are the security risks to the organization?
• What are known gaps in the security program or in the security posture of the organization?
• Which of these areas do we want to fix sooner rather than later? (p.145)

Do we have the resources required to tackle these issues? (p.146)

Laying out all the work for the management team to analyze and discuss laid the framework for many good discussions about what we were working on, why we were working on it, and what each item required in terms of resources, dollars, support, and so forth. It brought us closer as a team in terms of our understanding of the overall program and provided each manager with a big-picture look at the program. This is decidedly different from each manager simply managing their own work in a silo and not having discussions with the rest of the team. (p.146)

Attaining perfect information security is impossible, so as real-world security practitioners, we must strive to do the best we can with the limited resources, headcount, and budget that we have. Ultimately, prioritization is an exercise in determining relative importance of tasks, projects, and initiatives. (p.147)

Identifying what cannot be accomplished enables the team to dedicate the time and resources to what can be accomplished, and is just as important as defining what is the highest priority. (p.147)

When using compliance as a prioritization factor, make sure to give it an appropriate level of importance in the context of your specific business strategy and organizational needs, such that anything and everything related to compliance does not get a “free pass” simply because it may be vaguely related to compliance efforts. (p.149)

Sometimes information security professionals, with the best of intentions, pursue projects that turn out to be pointless or a waste of time (when compared with the positive results that might have been achieved by reallocating the budget, headcount, and resources dedicated to the original project). (p.152)

If you want to avoid the potential territory grab or politics that can sometimes arise when team members are asked to prioritize projects in their area of ownership versus areas they do not own, then enforce a sticky-note voting rule where each team member is only allowed to vote on issues not in their direct area of responsibility. (p.156)

Information security does not happen in a vacuum, and every company or organization has key people and teams whose support is required to run an effective information security program. These key people and teams are the stakeholders in the security metrics program and typically include roles such as CEO, CFO, CRO, CTO, BU leader, CIO, Director of Physical Security, and Director of HR. Security metrics require support from stakeholders to be successful, just like any other component of an information security program. (p.159)

A key message here may be that the sophistication of the threat is rapidly increasing, an appropriate introduction to which might be to provide the CEO with industry-level metrics on the number of new malware programs detected per day. (p.163)

The CFO is often one of the most important stakeholders for any security program. The CFO is the one who determines how much investment to make in security and doles out the headcount and budget. CFOs measure and quantify everything. To cut costs, CFOs may use methods that are not popular with employees, such as outsourcing. They respond best to hard numbers and data. (p.163)

CIOs are often concerned about availability and resiliency of their systems. They want their systems to be available continuously to support business projects and technology, and they want their systems to be robust so that there are fewer problems to fix or broken systems to replace. (p.167)

A CISO may meet with a CIO to request resources to work on key information security projects related to the technical infrastructure. Metrics to support this discussion may include the number of patches deployed within the requirements of the SLA, the percentage of new systems deployed according to a standard secure build, the percentage of systems in compliance with secure configuration standards, the percentage of systems with up-to-date antivirus protection, and the percentage of unmanaged systems. Trend lines over time will be useful to determine if security status is improving or getting worse. (p.167)

Sales and communications are not necessarily the first skills that come to mind when thinking about the traits required for an effective information security team; however, both of these skills will play a role in any job you take as an information security professional. (p.171)

…first thing to present to your stakeholders is a description of what you’re trying to achieve through your security metrics project. (p.178)

Each new technology should be added only if its benefits are relevant to the goals of your metrics program. (p.180)

Metrics that correlate two independent processes What is the relationship between security awareness training and certain types of security incidents? What is the relationship between a business unit’s use of a centralized internal audit service and the number of external audit findings? Is there a relationship between application language and discovered security flaws? (p.188)

Metrics that measure “treatment effect” What is the difference in security incident frequency before and after installation of a security information and event management (SIEM) solution? What is the difference between service desk workload and installation of a new identity and access management system? What is the difference in percentage of e-mail traffic that is spam before and after introduction of new antispam technology. (p.188)

Almost all analysis involves following many avenues of investigation with only a few discovered nuggets to show for all your work. The primary value added by metrics to raw data is precisely this analysis. One effective way to rescue your audience from traveling the same unproductive paths that you did is to not give them the opportunity. (p.194)

It is common for the data collection portion of a metrics project to dominate the budget in both cost and effort. In medical research, it is not unusual to see a budget that allocates as much as 80 percent of the cost to data collection, with the remaining 20 percent dedicated to design, calculation, and communication. From my experience, you should expect a similar 80-20 ratio for security metrics projects. (p.197)

If you elect to perform your own integration with components such as relational or OLAP repositories and modeling engines such as SAS or R, then I need to warn you, it takes time, resources, skill, and buy-in from your company. Attempting to do it on a shoestring or as an extracurricular unofficial project rarely yields success. The good news is that when you build a homegrown custom metrics system you get exactly what you need. The bad news is that it can be very expensive. (p.202)

Defining success criteria and metrics at the start rather than at the end of a project is always best. (p.229)

…when people hear the word metrics, they tend to think numbers, but numbers alone are insufficient to serve as metrics. In order to be valuable, the numbers need context and meaning. (p.234)

If a team waits for perfect data before it begins to report, it might never begin to report. (p.245)

…the importance of balancing reactive and strategic activities in a security program. Many organizations would benefit from moving toward a 50/50 split between responding to requests and proactively seeking out a greater understanding of the business issues and priorities. (p.249)

Having clear goals, objectives, success criteria, and metrics definitions is even more critical when work is being outsourced than when it’s being done by the in-house security team. (p.249)