Press TechRound interviews Secure.com CEO on the future of AI security
Read

The ECC Self-Assessment Trap: Why Scoring Yourself Compliant Isn’t the Same as Being Audit-Ready

Scoring 90% on an NCA ECC self-assessment does not mean you will pass an audit. Here's where the gap opens and how to close it.

TL;DR 

NCA ECC compliance gets evaluated through self-assessments, periodic compliance tool reports, and field audits. That first step feels like the finish line. It isn’t. A self-assessment asks whether a control exists. 

An audit asks you to prove it worked, on a date you did not choose, with evidence you did not have time to manufacture. Most failed ECC assessments are not security failures. They are evidence of failures. The fix is building controls that generate proof continuously instead of scrambling for it two weeks before the assessor arrives.

Where the Gap Actually Opens

A compliance manager at a Saudi utility runs the ECC checklist. Governance policies approved. MFA rolled out. Backups running. Access reviews done. The spreadsheet says 91%. Leadership relaxes.

Then the assessor arrives and asks a different question: show me the access review from Q2. Not the policy that says you do access reviews. The actual review. Who ran it, what they found, what changed after.

The spreadsheet had no column for that.

ECC-2:2024 restructured the framework into four domains, roughly 110 controls across 28 subdomains. Every one of those controls has two lives. There is the control as written, which a self-assessment can confirm. And there is the control as operated, which only evidence can prove. Self-assessments almost always measure the first and quietly assume the second.

What a Self-Assessment Can and Cannot Tell You

Self-assessment is a legitimate part of the ECC process. NCA explicitly uses it. The problem is what people expect it to do.

What it does well 

Confirms a control has been assigned to an owner. Flags controls you have not started. Gives leadership a rough progress number. Helps you scope the program before spending money on assessors.

What it cannot do 

Weight domains by how hard assessors actually push on them. Score a control that is half implemented. Judge whether your evidence would survive a follow-up question. Tell you if the person who ticked the box understood the control the same way NCA does.

SecurityWall puts it plainly in their ECC checklist: a self-read percentage is not an assessment result. A checklist confirms a control exists. It cannot weight domains by audit scrutiny or score partial implementations.

That distinction is the whole article.

The Five Ways Self-Scores Get Inflated

1. Policy is counted as practice 

You have an incident response policy. Approved, versioned, signed. The self-assessment counts the control as met. The assessor asks for the last three incident tickets and the timeline from detection to closure. If those do not exist, the policy proves you intended to do something. It does not prove you did it.

2. The person scoring is the person responsible 

Nobody grades their own homework honestly. Not out of malice. A control owner who spent six months rolling out logging will read “log monitoring implemented” generously. An assessor with no stake reads it literally.

3. Partial gets rounded up 

MFA covers 84% of accounts. The remaining 16% are service accounts and three legacy systems nobody wants to touch. The self-assessment marks it green because the project is “done.” ECC does not grade on effort.

4. Evidence exists but cannot be retrieved 

This one hurts the most. The proof is technically somewhere. In Okta logs, in a Jira ticket, in someone’s inbox, in a screenshot folder on a shared drive. Assessors do not accept “we can get that for you next week.” Evidence you cannot produce on request is evidence you do not have.

5. Scope was drawn too narrow 

ECC scope is interpreted more broadly than most private companies expect, and under-scoping tends to surface as a finding at the first assessment. If you scoped out a system the assessor scopes in, your 91% was calculated against the wrong denominator.

What Assessors Actually Look For

The pattern is consistent across ECC domains. Assessors want three things per control, and self-assessments usually capture only the first.

  • Exist. The control is documented, approved, and owned. Easiest to prove. Least meaningful on its own.
  • Operate. The control ran on a schedule, produced output, and someone reviewed that output. This requires logs, tickets, reports, timestamps.
  • Correct. When the control found a problem, something happened. A gap was closed, a risk was logged, a system was fixed, and there is a record of it.

The ECC compliance domain is explicit about this loop. Cybersecurity reviews must be conducted periodically by the cybersecurity function to assess compliance with the organization’s controls, with additional review by a function independent of cybersecurity, such as internal audit. That independence requirement exists precisely because self-review has a known ceiling.

Two ECC Areas Where Self-Scores Break Hardest

Governance reporting lines 

ECC recommends the cybersecurity function report directly to the head of the organization or their delegate, without creating a conflict of interest. Many organizations tick this control while the CISO reports into the CIO. On a checklist, the box says “cybersecurity function established.” In an assessment, the reporting line is the finding. SecurityWall names this as the most common governance failure they see: the cybersecurity function reporting into IT, which is the exact conflict the domain was designed to prevent.

Saudization of Cybersecurity roles 

ECC-2:2024 expanded the staffing requirement. Where the 2018 version applied full-time Saudi professional requirements to the cybersecurity function head and related supervisory and critical positions, the updated version refers to cybersecurity positions being filled with full-time and qualified Saudi cybersecurity professionals. Organizations that ticked this control based on the old senior-only reading are carrying a gap they do not know about.

Nationality documentation alone is not enough either. Assessors may want evidence that the person is actually performing the role and holds the qualification or experience the role requires. That is an HR evidence chain most security teams have never been asked to produce.

The Self-Assessment to Audit-Ready Gap, Visualized

Here is what the same control looks like at each stage.

Control: Periodic access reviews.

Self-assessment view.

Question: do you conduct periodic access reviews?

Answer: yes. Score: 1 of 1.

Time spent: nine seconds.

Audit view. Show the Q1, Q2, Q3, and Q4 review records. Show who performed each. Show the systems in scope and how that scope was decided. Show the accounts flagged. Show the revocations that followed, with timestamps. Show what happened to the accounts that were flagged but not revoked, and who approved that. Time spent: three weeks of retroactive archaeology, or zero, if the system was recording all of it along the way.

Same control. Same “yes.” Completely different outcome.

How to Close the Gap Before the Assessor Does

Score against evidence, not memory

Change the self-assessment question from “do we do this?” to “can I attach the proof right now?” If you cannot attach it in under five minutes, mark the control amber. Your percentage will drop. That drop is the most valuable number in the program.

Get an independent read

ECC already expects review by a function outside cybersecurity. Use internal audit, or a formal gap assessment, on the domains where you scored highest. High scores with no external challenge are where surprises live.

Fix the denominator first

Confirm scope with your sector regulator or through a formal assessment before you calculate any percentage. A precise score against the wrong scope is worse than no score, because it creates false confidence.

Make evidence a byproduct, not a project

This is the structural fix. If your controls only produce proof when a human remembers to screenshot something, you will fail an unannounced check. Because ECC compliance can be evaluated through tools, reports, and field audits, the program has to be built so controls generate evidence continuously, not in a one-off document sprint.

Track recurrence

If the same gap appears in three consecutive reviews, the control does not work. It is being remediated, then drifting back. That is a design problem, not an execution problem.

How Secure.com Helps

Secure.com’s Compliance Teammate turns the telemetry your security team already generates into audit evidence, so proof accumulates while controls run instead of getting reconstructed before an assessment.

  • Pulls evidence automatically from assets, vulnerabilities, access management, application security, and incident records, then maps it to framework controls
  • Shows live compliance status per framework with heatmaps for gaps, so your score reflects evidence coverage rather than a manual tick
  • Generates exportable, audit-ready reports with per-control drilldowns in minutes instead of weeks
  • Links compliance gaps to your risk register so high-risk findings get closed first, not the easy ones
  • Flags compliance drift and recurring gaps across review cycles, so controls that keep failing get redesigned instead of re-remediated

FAQs

Does NCA ECC require third-party certification? 

ECC compliance is generally demonstrated through self-assessment and regulatory audits rather than a certification body process, though this varies by sector. Under the 2026 classification, Class A organizations (250+ employees or SAR 200 million+ revenue) face mandatory independent audits, while Class B audits are recommended.

How does NCA actually evaluate compliance? 

Through a combination of self-assessments, periodic compliance tool reports, and field audits conducted by NCA or designated third parties. Any of the three can be the moment your evidence gets tested.

What is the penalty for ECC non-compliance? 

Reported penalties reach up to SAR 25 million, alongside operational restrictions and exclusion from government contracts. Confirm current figures directly with NCA guidance, as enforcement details change.

We passed our self-assessment. Do we still need a gap assessment? 

If your self-score came from control owners scoring their own controls with no evidence attached and no independent review, yes. The gap assessment is not there to lower your number. It is there to tell you which parts of the number are real.

Does ECC compliance cover SAMA requirements too? 

No. Financial sector organizations typically fall under both NCA and SAMA CSF, and meeting one does not satisfy the other. The technical controls overlap substantially, so map them together rather than running two separate programs.