Module 30: Control Testing & Effectiveness Evaluation

A control that is not tested is assumed — not effective.

Control effectiveness is not automatic.

CRISC expects structured evaluation of:

• Design effectiveness
• Operating effectiveness
• Ongoing performance
• Impact on residual risk

Controls must be proven — not presumed.

What the exam is really testing

When control testing appears, CRISC is asking:

• Is the control properly designed?
• Is it operating consistently?
• Is it monitored?
• Has effectiveness been validated?
• Has residual risk been reassessed?
• Is remediation triggered if ineffective?

CRISC favors evidence-based governance.

Design effectiveness

Design effectiveness answers:

Is the control capable of mitigating the risk as intended?

Questions to ask:

• Does it address root cause?
• Does it reduce likelihood or impact appropriately?
• Is scope correct?
• Is it aligned with policy?
• Is it enforceable?

If design is flawed, operating effectiveness is irrelevant.

Operating effectiveness

Operating effectiveness answers:

Is the control functioning consistently over time?

This involves:

• Sampling
• Evidence review
• Monitoring output
• Exception tracking
• Reperformance (in some cases)

A well-designed control that is not consistently performed is ineffective.

The most common exam mistake

Candidates often:

• Confuse implementation with effectiveness.
• Assume preventive controls are always effective.
• Ignore operating failures.
• Fail to reassess residual risk after testing.

CRISC separates design from operation clearly.

Testing methods

Control testing may include:

• Walkthroughs
• Observation
• Documentation review
• Log review
• Sampling transactions
• Automated control testing
• Key risk indicator (KRI) tracking

CRISC does not require tool knowledge — but expects testing discipline.

Example scenario (walk through it)

Scenario:
An access review policy is well documented. Testing reveals reviews are only performed sporadically.

What type of weakness exists?

A. Design deficiency
B. Operating deficiency
C. Inherent risk
D. Threat modeling gap

Correct answer:

B. Operating deficiency

Design exists — operation is inconsistent.

Slightly harder scenario

A new encryption control is implemented to mitigate data exposure risk. Testing reveals encryption applies only to stored data, not transmitted data.

What is the PRIMARY issue?

A. Operating deficiency
B. Risk appetite misalignment
C. Design deficiency
D. Excessive mitigation

Correct answer:

C. Design deficiency

The control design does not fully address exposure.

Reassessing residual risk

After control testing:

• If effective → confirm residual risk level.
• If ineffective → adjust residual risk upward.
• Escalate if exceeding tolerance.
• Trigger remediation.

Failure to adjust residual risk is a governance gap.

Continuous monitoring

Effectiveness is not one-time.

Organizations should:

• Define testing frequency
• Use metrics and KRIs
• Track control failures
• Monitor exception trends
• Escalate repeated failures

Recurring deficiencies may indicate root cause.

Metrics and performance indicators

Effective evaluation may include:

• Percentage of access reviews completed
• Time to remediate control failures
• Exception aging metrics
• SLA compliance metrics
• Incident trend analysis

If no metrics exist, effectiveness cannot be objectively measured.

Slightly uncomfortable scenario

A control is tested annually and consistently passes, but recent changes in business processes have expanded exposure beyond the control’s scope.

What is the MOST significant concern?

A. Weak operating effectiveness
B. Outdated design effectiveness
C. Excessive appetite
D. Poor BIA

Correct answer:

B. Outdated design effectiveness

Design effectiveness must be reassessed when environment changes.

Control failure escalation

If testing reveals control failure:

• Document issue
• Assign remediation owner
• Update risk register
• Reassess residual risk
• Escalate if tolerance exceeded

Testing without escalation is meaningless.

Design vs operating quick comparison

Design failure:
Control is incapable of mitigating risk.

Operating failure:
Control is capable but not functioning properly.

CRISC frequently tests this distinction.

Quick knowledge check

1) Which question evaluates design effectiveness?

A. Is the control performed consistently?
B. Does the control reduce the intended risk?
C. Is monitoring frequent enough?
D. Are logs reviewed daily?

2) If a control is properly designed but inconsistently executed, this indicates:

A. Design deficiency
B. Operating deficiency
C. Risk acceptance
D. Threat modeling gap

3) After identifying a control failure, what must occur?

A. Ignore if impact is low
B. Increase inherent risk only
C. Update residual risk and escalate if needed
D. Remove from risk register

Final takeaway

Control testing must:

• Validate design effectiveness
• Validate operating effectiveness
• Measure performance
• Reassess residual risk
• Trigger remediation
• Support escalation when required

Design tells you if it should work.
Testing tells you if it does work.
Governance ensures someone fixes it when it doesn’t.

CRISC rewards evidence-based risk reduction — not assumed security.