The Imperative of Assurance

The lessons of history tell us that any organization that fails to satisfy its interested parties has set itself on the path to self-destruction. Whether it is a quick death or a slow demise, the eventual outcome is the same, the business will cease to exist.

Top management’s primary duty is to ensure that this never happens, and that the diverse requirements of the organization’s stakeholders are consistently satisfied, at least to the greatest practical extent.  

In the previous article in this series we focused on the Imperative of Governance. We learned that in order to be effective, corporate governance not only requires organizations to be clear as to what they are seeking to achieve, they must also ensure that this Vision is consistent with meeting the needs of their stakeholders.

If, as auditors, we can evidence this link between intention and desire, we can now move forward to consider whether the organization has implemented a framework which is capable of translating their Vision into business deliverables. We are now ready to consider the Imperative of Assurance.

The Imperative of Assurance

Assurance – ‘A positive declaration intended to give confidence’ (source English Oxford Dictionaries)
This definition, whilst basic, neatly encapsulates what assurance is all about. It’s about being able to state that things are as they should be.

In the early days of manufacturing, things were simple. We allowed the customer to test our products for us.
 

 
This approach had many advantages, not least of which it did not require the employment of expensive quality specialists. As customers became more discerning and the choice of alternative providers became wider however, organizations quickly realised it was no longer acceptable to simply ship out their products and hope for the best. Customers could and would go elsewhere.

With the rise of mass production came the introduction of routine inspections. Here products were checked at key stages in the production process against pre-defined criteria with the results being recorded in a pass/fail checklist. Whilst unsophisticated, it can be seen that these inspections certainly qualified as assurance activities based on the definition above, providing both the organization and the customer with a degree of confidence that the end product was fit for its intended purpose.

The main problem with inspection then, as it is now, is that the product or service has actually been partially or fully produced or possibly even delivered when the assessment is conducted. Correcting mistakes at this point is in the production cycle is expensive, requiring scrapping, reworking and/or the supply of a replacement product. There had to be a better way to provide necessary assurance.

The 1950’s saw the rise of an alternative, pro-active approach to quality, which sought to plan quality into products and the processes that produced them. Quality Assurance was born. Today quality assurance is widely practised, though inspection still as an important role to play and should not be dismissed. The next time you get on a plane be grateful that pre-flight inspections are still being performed, much as they were in the early days of flight.  

As audit professionals, we must feel completely at home operating in the Assurance arena. After all, providing evidence based, impartial assurance that things are as they should be lies at the heart of what our profession was established to do. So, when we are called upon to provide assurance what should we as management systems auditors be looking to give confidence about?

The Chartered Quality Institute’s (CQI’s) Competence Framework helps us here by suggesting two questions we should seek to answer. The first of these is ‘is management intent effectively implemented?’ The second is ‘does it produce the intent outcomes?’ 

Put more simply, these can be translated as ‘top management have decided what the organization should do but is the organization actually doing this?’ and ‘if the organization is doing what it is expected to do is this producing the results we want?’.

For the present, let’s concern ourselves with the former.

Determining whether management intent is effectively implemented

As assessors, what we are looking for here is audit evidence to confirm that a means exists to translate the policy decisions made by top management into delivered products and/or services which are capable of consistently meeting stakeholder requirements.

‘Audit evidence’ typically comprises ‘records, statements of fact or other information which are relevant to our audit criteria and verifiable’ (source ISO 9000:2015 - Quality management systems -- Fundamentals and vocabulary) whilst the ‘means’ is typically a business management system which may or may not be based on an international standard such as ISO 9001:2015 or ISO 14001:2015.
What we would hope the audit evidence to identify is a ‘golden thread’ which ties together organizational policy, strategy, objectives, plans, projects, processes and individual tasks into a single coherent management system which consistently generates the outcomes stakeholders are expecting to see.  

Annex SL provides a blueprint which can be applied in order to evaluate any management system irrespective of whether it is based on an ISO standard.

Annex SL Clause 4.4 requires compliant organizations to establish, implement, maintain and continually improve their management systems, including the processes needed (for their operation) and their interactions, whilst Clause 5.1 requires Top Management to embed these processes into ‘business as usual’.  

Clause 5.2 obliges Top management to develop and communicate a Policy Statement which draws on their understanding of Organizational Context and which commits the organization to satisfying applicable requirements, including stakeholder requirements. The Policy Statement must additionally provide a framework for the setting of the management system’s objectives, which top management must ensure are compatible with overall strategic direction of the organization.

Clause 6.2 then builds the link between management system objectives and system level planning for their achievement, and clause 8.1 links system level planning to process design, implementation and control. Finally, clause 9.1 requires the organization to evaluate the performance and effectiveness of the management system overall.

If all of these elements are found to be in place the auditor is likely to conclude that a means for implementing management intent has indeed been established. Furthermore, if the auditor can additionally evidence these elements as working together well, seamlessly delivering their intended outcomes, the auditor may go a step further and conclude ‘management intent is being effectively implemented’.

Simply having a joined-up system in place however is no guarantee that the organization will generate the outcomes it has designed the system to deliver. Auditors therefore need to ask a second assurance question - ‘does the management system produce the intended outcomes?’

Determining if the organization is producing the intended outcomes

We’ve now reached the point where we have established that the organization has a management system in place which has the potential to produce its’ intended outcomes. But having the potential and actually achieving practical delivery are two entirely different matters.

It is now time to examine what is happening on the ‘factory floor’. Our focus shifts to monitoring, measurement, analysis and evaluation as we move away from process assurance into the realms of product/service assurance.

Clause 9 of annex SL based management systems addresses performance evaluation. Its purpose is to ensure that organizations have suitable arrangements in place to monitor, measure, analyse and evaluate their activities.

The organization must determine what it should monitor and measure, how it should monitor and measure in order to ensure valid results, when it should monitor and measure and when the results of monitoring and measuring should be analysed and evaluated. These requirements apply not just to the products and services being produced but also to the operation of the management system itself.

Whilst these are ultimately decisions for the organization, the auditor should be prepared to challenge where audit evidence suggests that the organization’s decision is wrong. If, for example, a high level of non-conforming output can be directly attributed to a decision to employ low cost monitoring instead of more expensive measurement, then the auditor would be justified in raising a non-conformance.

Whilst the organization is required to retain appropriate documented information’ to evidence the results of its performance evaluation, the auditor may find that this is stored in a myriad of formats. For this reason, auditors need to be comfortable working with electronic records as well as hard copy ones.

‘Being comfortable’ includes not only being able to navigate the use of the ICT system, but also (crucially) being able to interpret what the data and information contained therein is telling them.

The results of internal audit (9.2) and management review (9.3) will further inform the auditor’s assessment as to whether the system is producing its intended outcomes and therefore whether the assurances sought by commissioning the audit can now be provided. 

Providing assurance is nothing without credibility

Ultimately, in order for any assurance provided by an auditor to be regarded as credible, it is essential that the person commissioning the audit (the audit client) considers the auditor competent. Annex SL defines competence as ‘the ability to apply knowledge and skills to achieve intended results’. So, auditors must understand how audits should be performed and be able to carry audits out in this manner.

ISO 19011 Guidelines for Auditing Management Systems is essential reading for those who are serious about improving their auditing ability. Underpinning this standard are six (soon to be seven) principles of auditing, the high-level ‘rules’ which auditors must comply with.

These include:

All auditors, including internal auditors, are additionally expected to possess knowledge and skills relating to audit principles, processes and methods, management system standards or other references, the organization and its context and applicable legal requirements and other requirements.

ISO 19011 also sets out expected personal behaviours for those conducting audits. These include being:

• Ethical          i.e. fair, truthful, sincere and discrete

• Open-minded          i.e. willing to consider alternative ideas

• Diplomatic          i.e. tactful in dealing with people

• Observant          i.e. actively observing physical surroundings and activities

• Perceptive          i.e. aware of and able to understand situations

• Versatile          i.e. able to readily adapt to different situations

• Tenacious          i.e. persistent and focused on achieving objectives

• Decisive          i.e. able to reach timely conclusions based on logical reasoning and analysis

• Self-reliant          i.e. able to act independently whilst interacting effectively with others

• Acting with fortitude          i.e. acting ethically even when this may make you unpopular

• Open to improvement          i.e. willing to learn from situations

• Culturally sensitive          i.e. observant and respectful to the culture of the auditee.

• Collaborative          i.e. effectively interacting with others 

In order to ensure that auditors have achieve the necessary competence and are exhibiting the expected behaviours, DIS ISO 19011:2017 clause 7.3 requires the organization to establish criteria for evaluating auditor performance whilst clause 7.4 requires the organization to decide the method it will use to evaluate auditor performance. Clause 7.5 sets out expectations where auditors are not found to be operating at the required level whilst clause 7.6 recognises that maintaining competence is an ongoing process, not a one-off activity.

By using ISO 19011 as a framework for designing and conducting their internal audit programmes and for the qualification of their auditors and audit team leaders, organizations can demonstrate that they have credible arrangements in place to allow for objective assessment of their business management system.

Should stakeholders require additional assurance, they may require that the organization is assessed by a ‘third party’. Third party audits are carried out by suitably qualified individuals or organizations who are independent of the organization being assessed. Typically, these individuals will work for a certification body, brought in to evidence conformity with an international standard or a regulator, brought in to determine compliance with a legal or statutory framework. In both cases their auditors will work to strict guidelines, following well defined processes which are designed to ensure that their audit findings are representative of what is actually going on in the organization.

Where third party auditors are able to determine an organization is complying with the requirements they are able to issue a certificate to confirm that this is the case. This certificate can then be used for marketing purposes and to pursue more business. Certification is very popular as evidence by the latest ISO (International Organization for Standardization) survey which overall shows an 8% growth in certificates issued.
 


The survey identified that as at 31.12.16 certification bodies had issued 1.64 million certificates, with the overwhelming majority of these being to organizations wishing to provide assurance that they were meeting the requirements of the quality management system standard ISO 9001. Of this number 93% were still working to ISO 9001:2008 with only 7% working to the new version ISO 9001:2015. The biggest growth areas are energy management and IT.

Unfortunately, not all certification bodies operate to the same high standard and therefore the simple fact that an organization holds a certificate is not in itself sufficient to provide assurance that it is a good organization to work with. Additional checks need to be carried out, usually by means of a second party assessment.

Second party audits are audits carried out by one organization on another to determine whether they are suitable for doing business with. In this case the audit criteria are set by the organization seeking to appoint. The criteria could be a ‘one off’ contract – can the potential supplier deliver against this specific piece of work? or a more generic supplier qualification questionnaire.

Third party vs first party assurance

Whilst third-party certification plays an important role, in the majority of instances a suitably robust internal audit process will provide better insight into what is really going on in the organization due to its proximity to and understanding of the business. Internal auditors are viewed too often as being secondary to third party assessors and are undervalued by their organizations. This is an incorrect judgement. Organizations must instead invest in their internal audit functions, developing both the people and the process. 

All of this effort will of course count for nothing if Top Management are not prepared to act on audit findings. An essential part of providing assurance to stakeholders is that when audits do discover non-conformity or non-compliance, top management can be relied on to act rapidly to address this. If Top Management are not prepared to address the deficiencies auditors identify, the organization’s stakeholder will never be fully assured. 

Next time

Internal auditors, as a result of working in the organization day in, day out, are ideally placed to both identify and drive through improvement. All too often however their remit is constrained to providing assurance alone. This article examines how the role of internal auditor can add significant value to an organization if audit criteria are extended beyond assurance and into the realms of improvement.