Provision 29: When might Operational Effectiveness testing be of use?

As an adjunct to my recent post on approaching Provision 29 assurance, I did want to comment on what Operational Effectiveness testing I would see as useful; in short, any testing that has a day-to-day, demonstrable business benefit.

Therefore, I would not see a case for very much specific OE testing for Provision 29 purposes. In fact, if our focus is on Material Controls operating to give Exec-level management oversight of their areas of accountability, then the assurance that key (to use a SOX term) controls are operating should already be baked in – if it’s not, and there’s a perceived need for control testing in the run-up to year end, then that may be evidence that the design of the Material Control is not effective.

That doesn’t mean OE testing has no role to play. Where I do think that there is a case is in the testing of compliance frameworks in regulated industries. Taking UK utilities as an example, most organisations apply a ‘responsible persons’ approach to regulatory compliance, where one senior individual is accountable for ensuring that the regulation is complied with by the business with responsibility at a (often very junior in hierarchical terms) Line 1 level.

It should be binary; you either comply or you don’t. In reality it’s far messier; a combination of regulatory ambiguity, historic data problems, process latency etc combined with Government pressure, regulatory overlap, and public outrage make this very complicated. Non-Executive Directors want the truth about compliance performance, particularly, as with the troubles of Southeast Water, it’s often NEDs who are called to explain the company’s woes to the relevant Parliamentary committee. At some point soon (but not in the next 6-months), AI will provide a neater answer, but until then OE assurance testing of design and application of compliance frameworks (if there’s a clear/large enough short-term consequence) would, in my view, be essential.

As in my previous post, I’m going to again make a case for revisiting the principles of COCO (Criteria of Control). A proper focus on the human operating that L1 control and ensuring that they understand the purpose of the control, that they are committed to its operation, that they have the right skills, tools and experience to operate it and there is a feedback loop to tweak its performance is at least as useful as more policies, more controls, more testing etc..