My input to the ROI spat

Richard Bejtlich wrote a blog post entitled Are the Questions Sound? which has prompted a number of replies. One in particular, Bejtlich and Business: Will It Blend? took him to task on several issues.

I'll say up front that Richard is a friend of mine, and I've always found him to be very focused on how security adds (or detracts) from an organization's business. He's the rare person that possesses both deep technical knowledge and the ability to understand and communicate business drivers. If he stays long enough, he'll probably end up being GE's CISO one day.

Back to the task at hand. The thing that struck me most about Kenneth's posting was the false dichotomy he laid out in his first point:

1. Either the CISO does not know what he’s talking about at all or Richard clearly misinterpreted him

I suggest another possibility: the CISO was intentionally being a dick.

Yes, this is extremely common in the financial world. I have consulted with several large insurance and brokerage firms, and the problem is epidemic, especially on Wall Street. Executives like to push around their vendors, probably to reinforce their ideas about their own power and influence.

I've had this happen to me, and I've seen it happen to others. It's quite possible that the CISO knew what he was talking about, but chose to bully the consultant, just to see how he'd take it. I once had the VP of a Wall Street firm call my cell phone on Christmas Eve (Friday) and demand that I have some consultants on site the following Monday or he'd call my CEO and tell him "what kind of people he has working for him."

We'd already been there for most of a year, and they had already stiffed us on $300k worth of consulting fees, so I just told him not to threaten me, and then gave him the CEO's phone number. He backed down immediately. I later found out that he was in the process of being fired, and was trying to get us involved so he could point fingers. I'm sure that company is just a little better off today without him there.

Kenneth's second point is all about security ROI:

2. Information Security does not have an ROI and that security mechanisms are not business enablers

My friend in the financial risk department read Richard’s statement that “Security does not have an ROI” and he laughed. He commented, “Just let some hackers change some numbers in a banks financial system and you’ll see that security has ROI.”

It happens that I do agree that security can have an ROI, but the scenario given is not an example of that. It's an example of loss prevention and, to a certain extent, business enablement (to enable the bank to survive, which it really wouldn't if any Joe could log in and change account balances at will). You can best see this by examining the underlying requirement:

• No one should be allowed to alter account balances without proper authorization.
  

Given that requirement, you can implement the countermeasures in various ways. For example, you could:

1. Apply an application-layer defense to validate changes before they happen.
   
2. Report on suspicious changes that have already occurred, and roll back anything that was not authorized.
   

Both achieve the same end, and both are used in real world situations. Given the requirement, both are functionally equivalent, but the second method more clearly shows the loss-prevention aspect.

If you'd like an example of how security can be an enabler, go back to my original comment about any Joe changing the database. Even better, you can see how it can become a positive source of revenue by examining Bank of America's Sitekey system. They're clearly selling the security of their online banking as a key differentiator, hoping to drive customers their way. And I'm pretty sure it's working.

Finally, Kenneth's point #3 is a bit inconsistent:

3. Some quotes are just wrong or nonsensical because they take the five-digit accuracy too literally instead of using modeling to understand risk :

* “Assumptions make financial “five digit accuracy” possible.”
o Actually mathematics make five digit accuracy possible: I can assume anything

There's more to this quote, but I'd just like to point out that if you're using a "model", this is a short word for "a set of assumptions we believe are close enough to true to be useful for predictive purposes."

Overall, I'm really kind of puzzled by the whole ROI debate. I guess I can understand that some people have staked their reputations on one side or the other, but really it's a lot like "VI or EMACS" for the infosec crowd. It's more of a philosophical debate based on which models you are more comfortable with.