When you’re a CISO, people assume you live in a bunker, whispering dark secrets about zero-days and nation-state actors. In reality, a decent amount of my time is spent on benchmarking. Team sizes, budgets, org structures. After all, you don’t have to be faster than the bear, just faster than the guy next to you. The unglamorous but necessary work of proving to my CFO that, no, I’m not, in fact, “hoarding cybersecurity engineers like Beanie Babies in the ’90s.” While it’s not entirely accurate, it’s a witty, pragmatic one-liner that I use to make cybersecurity investment sticky with my leadership team and board.
Traditionally, you’d hire a management consultant for this: shiny slide decks, cryptic buzzwords, and invoices that rival your EDR budget. But recently, I’ve been experimenting with a more affordable advisor: AI tools like ChatGPT. Think “McKinsey without the tie.”
Here’s what I learned…
Step 1: Asking the Big Benchmarking Questions
Why it’s important: As CISOs, we’re constantly asked, 'How many people do you really need for identity, cloud, or SOC?' The answer determines headcount requests, budget approvals, and whether your CFO smiles or throws their coffee mug.
How ChatGPT helps: Instead of flipping through a $20K Gartner report, I can ask the AI: “How many FTEs should I run IAM for 18K users with Okta, SailPoint, and CyberArk?” In seconds, I get back community data, analyst reports, and staffing ranges. What is most interesting about the results I get back for directed questions like that is that they appear to be mostly sourced in user community discussions rather than organizational surveys. In other words, when it’s available, you get real-world results and not an educated take based on surveys.
Trade-off: Unlike consultants, AI doesn’t caveat every answer with, 'It depends.' It gives you ranges and averages. You still need to layer in your org’s quirks, like whether half your workforce is in manufacturing plants with badge readers that date back to Y2K.
Step 2: Validating Against Real-World Data
Why it’s important: Benchmarks are only useful if they’re tethered to reality. One CISO’s 'overstaffed' is another’s 'barely keeping the lights on.'
How ChatGPT helps: I can cross-reference AI’s answer with industry forums, user communities, and actual peers. For example, when it told me 13 was reasonable for IAM at 18K users, I checked against a peer with 300K users who runs a very similar stack he had 13 total too. Score one for AI.
That means using your peer communities is still an incredibly important tool in your decision making. Organizations like Cyber Security Tribe are an excellent way to meet other practitioners who are going through the same challenges that you are. After all, we’re all in this together at every level of this, not just when it comes to sharing engineering or operational information like IOCs.
Trade-off: Sometimes AI sources marketing whitepapers (“Deploy Okta and reduce FTEs by 72%!”). That’s like asking Peloton if their bike pays for itself. Make sure you take whitepapers for what they are: Usually helpful because they tend to be based on real world experiences and data but salted heavily with skepticism because it is a marketing product after all.
Step 3: Breaking Down Roles and Workloads
Why it’s important: Even if you have the right total headcount, you need the right mix. Too many engineers and not enough analysts? Burnout. Too many managers and not enough engineers? Bureaucracy.
How ChatGPT helps: It’ll spit out a role-by-role breakdown: engineers for Okta, SailPoint, and CyberArk, an architect for integration, a scripting specialist, analysts for workflow, and a couple of managers. I used this list to validate whether I was missing functions and sure enough, I hadn’t budgeted for someone to own automation, which is becoming increasingly important as we’re perennially squeezed to “do more with less”.
Trade-off: AI is excellent at listing roles but doesn’t tell you who’s the rockstar generalist who can cover three of them. Consultants tend to interview your people and sniff that out; with AI, you need to bring that context yourself.
Step 4: Building the Case for the CFO
Why it’s important: None of this matters if you can’t make the budget case. The finance team wants numbers, benchmarks, and a promise you won’t be back in six months asking for 'just two more engineers.' If you’re doing this right, you should have a multi-year timeline with expected growth that is aligned to the corporate growth plan. After all, it’s much harder for my CFO say no to what is essentially their own plan.
How ChatGPT helps: AI outputs are structured, data-rich, and—let’s be honest—look fantastic when reformatted into a slide deck. I can pull an AI-sourced benchmark table, sprinkle in community anecdotes, and suddenly my 13-person ask looks downright conservative. I can also keep asking AI to slice the data in another way, or to do additional research on related topics. Doing this with management consultants is a drawn-out process where getting the answer some specific nuance can take weeks. With AI I get the answers within minutes.
Trade-off: Consultants bring authority (and blame). If you’re wrong, you can say, 'Well, Gartner said…' If AI is wrong, the finger points to you. Using ChatGPT means owning the analysis, which is both empowering and terrifying. It also means you need to read the room. If your leadership team or board isn’t supportive of AI to begin with this whole approach is a really bad idea as it erodes your own credibility, which is one of your most important personal assets.
Step 5: Continuous Benchmarking
Why it’s important: Org structures evolve, and benchmarking once is like weighing yourself only after Thanksgiving dinner. Regular updates matter.
How ChatGPT helps: I can rerun the same prompts quarterly, ask for the latest data, and see if my ratios are still in line. No contract renewals, no waiting six weeks for consultants to 'workshop the findings.' The trick here is either making sure your chosen AI has memory turned on, or you capture the prompts or chats you used to generate the data in the first place.
Trade-off: Consultants will chase you with updates (for a fee). AI won’t unless you remember to ask. The savings come with the responsibility to keep pulling the thread.
Final Thoughts
Using AI for benchmarking is like replacing a Michelin-starred chef with an Instant Pot. You don’t get the white tablecloth service, but you get something hot, fast, and surprisingly tasty.
The upside:
- Rapid insights at zero marginal cost.
- Role breakdowns that consultants would charge five or six figures for.
- Enough benchmarking data to avoid both overstaffing and CFO rage.
The downside:
- You’re the QA department. AI doesn’t self-validate.
- Marketing whitepapers sometimes sneak in.
- No scapegoat if the board hates your org chart.
Would I trade it for consultants entirely? Not always. But for 80% of the benchmarking grind, ChatGPT has become my “budget-friendly advisor.”
And hey, if nothing else, it hasn’t once tried to charge me $500 for a slide deck that says, “It depends.”
