GRC Effectiveness — What to Close Before You Buy a Tool

Three complaints get recycled endlessly in GRC discourse: manual evidence collection, slow implementation, and six-figure price tags. They are usually treated as three problems with three fixes. I think they are one problem wearing three costumes — and the fix is not a tool.

What This Post Covers

  • Tracing the three most-cited GRC issues — fragmented evidence collection, the exposure window, and cost structure — back to their actual sources
  • Showing that the three are not separate problems but symptoms of one structural cause
  • Stating the one conclusion that follows, and the three operating propositions it forces


TL;DR

The conclusion: GRC effectiveness is not decided by which tool you buy. It is decided by whether evidence is produced automatically as a byproduct of operations. Three operating propositions follow from that.

  • Detach control mapping from procurement. Mapping needs no tool — a spreadsheet starts it, and the output migrates to any platform later. The exposure window is not created by slow deployment. It is created by deferring work that never needed a tool.
  • Spend on Tier 2 evidence. Evidence that systems already emit needs only a retention setting. Evidence that requires human judgment should be explicitly excluded from automation. The return comes from promoting what currently lives in email and documents into system workflows.
  • Change the success test from "implementation complete" to evidence freshness. Freshness is hard to game — if evidence does not emit automatically, it necessarily decays with time.
  • Corollary — the answer to cost is not a cheaper tool. Zero license fee leaves the mapping and workflow-definition effort exactly where it was.
  • One widely cited number deserves scrutiny. The 71% figure does not measure "audit failure caused by manual collection." It is the complement of 29%, from a survey run by a GRC automation vendor.

Checking the Numbers — What the 71% Actually Measured

Start with the evidence base. All three issues are real, but there is a gap between the circulating summary and the original source.

The GRC Chaos report. Swimlane published this research in April 2025, surveying 500 IT and security decision-makers across the US and UK. The headline finding is that only 29% of organizations say their compliance programs consistently meet internal and external standards. The commonly quoted 71% is the complement of that figure — closer to "cannot confidently say they consistently meet standards" than to "will fail an audit." Fragmented workflows, manual evidence gathering, and poor collaboration between security and GRC teams are the causes the report identifies, not the basis on which the 71% was calculated. A separate figure from the same survey — that 54% still rely heavily on manual processes — is the more precise citation if you want to argue causation.

One more thing has to be attached to this. The survey was conducted by a vendor that sells GRC automation. It measures the size of the problem its own product solves. That is not a reason to discard the number, but it is a reason not to cite it as independently verified industry statistics.

The exposure window argument. This does originate with Josh Sokol. He is not a neutral practitioner, though — he is the founder and CEO of SimpleRisk. What he actually recorded, in the source I can verify, is not a critique of SOW drafting timelines. It is an observation from a CISO roundtable where several participants said they were still implementing their GRC programs and had been connecting all the wires for about a year. His own judgment, stated separately, is that such activities could add immediate value and need not wait weeks or months.

The specific framing of "weeks to months spent writing SOWs" is not confirmed in that source. The direction of the argument is the same, but it is safer to attribute it as practitioner testimony about implementation delay rather than as Sokol's own words.

The cost structure. The founding story is public: in 2013 Sokol brought a $500K GRC quote to his VP and was told his budget was zero. That anecdote supports the general magnitude. But it is a 2013 quote, and SaaS compliance automation has since entered and fragmented the price range considerably. The lag is too large to describe today's market directly.

Reframing the Issues — Three Symptoms, One Cause

Treat the three as separate problems and you get three separate prescriptions. Collection is painful, so buy an automation tool. Deployment is slow, so buy a lightweight tool. Cost is prohibitive, so go open source. Every one of these converges on "change the tool" — and cases where all three problems resolved together are rare.

I think one cause runs through all three symptoms. Evidence is treated as the output of a project rather than a byproduct of operations.

Project output means a separate person does separate work at a separate time to produce evidence. Audit approaches, someone tours the systems taking screenshots, downloading logs, filling spreadsheets. In this structure the three symptoms are inevitable — collection is always manual (what needs automating is a human habit, not a process), deployment always takes long (the tool has to replace that habit), and cost is always high (habit replacement is consulting).

Byproduct of operations is the opposite. Run an access review and the review record exists. Run the CI pipeline and the build approval trail exists. Close a ticket and the change authorization exists. No separate act produces evidence. In this structure, audit preparation becomes a query rather than a collection exercise.

For a platform engineer this structure is familiar — it is the same transition observability already went through. From SSHing into a box to grep logs after an incident, to applications emitting structured telemetry by design. GRC evidence needs the same move.

Proposition One: Detach Control Mapping — From the Tool and From Procurement

The real substance of the exposure window problem is not "deployment is slow." It is that an intellectual task has been chained to a procurement schedule.

Control mapping — aligning the requirements of the frameworks you are subject to (ISO 27001, SOC 2, NIST CSF, and so on) into a single internal control set — requires no tool at all. It requires the framework text, a list of controls currently operating, and someone to reconcile the two. It starts on one spreadsheet, and its output transfers intact to whatever platform you eventually buy.

Defer it and you will do the identical work after the tool arrives. The pre-mapped control libraries vendors ship are a starting point, not an answer — no vendor knows what your actual controls are or who owns them.

The execution shape looks like this.

  1. Fix the in-scope frameworks and expand their requirements into rows
  2. For each requirement, record only whether a control currently operates, and if so who owns it — do not attach evidence yet
  3. Extract the rows that came back as "no control" into a separate list. That list is your actual exposure window
  4. Escalate that list at the executive reporting level. Remediation begins independently of any tool decision

These four steps run on the order of days to weeks depending on organization size. Procurement may still take months, but you are not defenseless during it — you at least know what is missing.

Proposition one — the exposure window is not created by slow deployment. It is created by pushing work that needed no tool behind the procurement cycle.

Proposition Two: The Evidence Pipeline — From Collection to Emission

Once mapping is done, the next question is what demonstrates that a control is operating. Here I sort evidence into three tiers. This classification is my own framing, not a standard.

Tier 1 — evidence systems already emit. IdP access logs, cloud audit trails, merge history in version control, approval records in the ticketing system. Generated without human intervention and timestamped on creation. The only work required is setting retention and securing a query path.

Tier 2 — evidence systems could emit but currently do not. Policy exception approvals, vendor due diligence results, periodic access reviews. Today these are scattered across email and documents, but moving the workflow into a ticketing or workflow system promotes them to Tier 1. This is where GRC investment actually returns.

Tier 3 — evidence that necessarily involves human judgment. Risk acceptance decisions, management review minutes, effectiveness assessment of awareness training. Not automation targets. Attempts to automate this tier are what make GRC projects bloat.

Securing effectiveness means concentrating resources on moving Tier 2 into Tier 1, and explicitly declaring Tier 3 out of scope for automation. Tool selection runs on the same criterion — how many of our Tier 2 items does this tool promote to Tier 1? I think that question, rather than a feature matrix comparison, should be the selection criterion.

Proposition two — the investment target is Tier 2 evidence, not the tool. Trying to automate Tier 3 is what bloats GRC projects.

Proposition Three: Effectiveness Metrics — Durability, Not Completion

GRC implementations are usually judged to have failed not at implementation but at the second audit. The project team carries the first audit across the line; at the second, someone discovers the evidence has migrated back into spreadsheets.

So "percent complete" cannot be an effectiveness metric. I propose these four alternatives — again, my own framing rather than a standard from any specific framework.

Metric Definition Why watch it
Evidence freshness Median age in days of the latest evidence per control Exposes controls refreshed only just before an audit
Automated evidence ratio Share of controls backed by Tier 1 evidence Real progress of the pipeline transition
Control owner gap rate Share of controls with no owner or a departed one The first thing to collapse after a reorg
Audit rework rate Share of evidence newly collected in response to auditor requests The gap between readiness and what auditors actually ask for

There is a reason freshness comes first. The other three can be made to look good with effort. Freshness is hard to game — if evidence does not emit automatically, it necessarily worsens with time.

Proposition three — the moment of judgment is the second audit, not implementation sign-off. Evidence freshness measures whether you survive to it.

A Counterpoint on Speed — Lightweight Tools Are Not Instantly Usable Either

The prescription "enterprise GRC is slow, so go lightweight" has a counterexample.

One independent review of an open-source GRC platform assesses that while a technically capable team can stand up the application itself, configuring framework mappings, customizing risk formulas, setting up role-based access, and building control testing workflows is not a weekend project — and that reaching a state you would be comfortable showing an auditor takes two to four weeks of part-time effort. Vendors emphasizing installation in minutes and the time to actual audit usability are two different stories.

What this counterexample implies is clear. Most of the deployment time comes from mapping and workflow definition, not software installation. And that work is yours regardless of which tool you pick. Swapping tools does not make it disappear — which is exactly why I argued for detaching mapping and doing it first. Reverse the order and the same work slides behind procurement and gets booked as delay.

The cost question yields to the same logic. Zero license cost leaves the internal labor for mapping and operations untouched. Total cost of ownership should be compared not on licensing but on total effort to promote Tier 2 evidence to Tier 1.

A Platform Engineer's View — GRC Is a Data Pipeline Problem

The bottleneck in GRC effectiveness is not deployment speed but whether evidence emerges as a byproduct of operations. Translate that sentence into platform engineering vocabulary and GRC stops being a compliance problem and becomes a data pipeline problem.

  • Sources — the systems that leave traces when controls operate (IdP, cloud, VCS, ticketing)
  • Schema — a common control framework keyed on control identifiers
  • Load cadence — the evidence freshness target is the SLO
  • Consumers — internal audit, external auditors, board reporting

Framed this way, the GRC platform's role becomes obvious. It is not where evidence is made. It is where already-made evidence gets bound to control identifiers and made queryable. Establish that definition first and what you look for in a vendor demo changes — not whether the screens are attractive, but whether connectors exist for your source systems, and whether the API is open enough for you to build the ones that do not.

Organizations putting AI agents into operational paths gain one more source. Agent tool-call history, approval records for actions requiring sign-off, delegation scope. No framework currently requires these as explicit control items, but I think it is worth adding them to the mapping sheet even as empty rows. Retroactive collection is impossible once the requirement arrives.

Conclusion — Tool Selection Is the Last Decision, Not the First

To restate it: GRC effectiveness is not decided by which tool you buy. It is decided by whether evidence is produced automatically as a byproduct of operations.

The standard prescriptions for the three issues — automation tool for painful collection, lightweight tool for slow deployment, open source for cost — all converge on changing the tool. But if the three symptoms share one cause, the remedy should also be one, and that one is not a tool. It is moving the producer of evidence from people to systems.

So the execution order inverts. The conventional order is select tool → deploy → map → collect evidence. If the conclusion above holds, the order is map → classify evidence tiers → fix the Tier 2 promotion list → then select a tool against that list as the requirement. The tool comes last, because without the first three steps you cannot define what you are buying.

How each of the three issues gets handled under that order is this post's answer. The exposure window closes early by detaching mapping from procurement. Fragmented collection resolves by promoting Tier 2 evidence into system workflows. Cost gets judged on total effort rather than licensing — because a zero license fee leaves the mapping and workflow-definition effort exactly where it was.

Compressed to one sentence: GRC is not something you buy. It is plumbing that draws evidence out of systems you already operate — and evidence freshness is the gauge on that plumbing.

Caveats

  • The 71% figure comes from a GRC automation vendor's own survey, not independently verified industry statistics. It is also self-reported.
  • Sokol's exposure window argument comes from a positioning context emphasizing his own product's faster deployment against competitors. The argument stands on its merits, but it is an interested claim.
  • The $500K quote is a single data point from 2013. It does not represent current market pricing.
  • The three-tier evidence classification and the four effectiveness metrics proposed here are my own framing, not results validated by applying them in a real organization.
  • The "days to weeks" estimate for mapping is likewise my experiential guess. It varies substantially with framework count and existing control documentation maturity.
  • This post contains no measurement section. Actual data on evidence freshness or automated evidence ratio in a live environment would change the weight of these claims.

References

Post a Comment

Previous Post Next Post