Audit Quality And the Problem with Sampling

The audit quality smoking gun

Audit quality issues rarely arise because professionals are unaware of the standards. More often, deficiencies occur when audit work is incomplete, inconsistently executed, or insufficiently documented to demonstrate that the standards were applied as intended.

When regulators identify audit deficiencies, they almost always point to the audit file. Required documentation may be missing or insufficient. Risks may not be clearly linked to audit procedures. Conclusions may not be adequately supported by audit evidence. Reviews may have occurred, but their robustness is called into question by the questionable contents of the workpapers.

The fact that the audit quality smoking gun is usually hidden deep within an audit file is not surprising. For any given audit engagement, the audit file is the official record of the audit work performed and must stand on its own.

While most quality issues surface deep within the audit file, new quality control standards are calling for more rigorous quality control at a firmwide level to identify and address systemic risks and root causes for failures in a more robust and effective way.

What the new quality management standards require

The shift is driven by the new generation of quality management standards, including ISQM 1, SQMS 1, and the PCAOB’s QC 1000 standard.

These standards move firms away from static quality control programs toward a quality management system that must be designed, implemented, and operated. Firms are required to identify quality risks, design responses to those risks, and demonstrate that those responses are operating effectively in practice.

The standards do not mandate specific technology or require the use of artificial intelligence. What they do require is evidence. Firms must demonstrate that monitoring activities are functioning as designed and that deficiencies are identified, evaluated, and remediated in a timely manner.

Under ISQM 1, SQMS 1, and the PCAOB’s QC 1000 standard, firms are expected to design monitoring activities that provide sufficient and appropriate information to evaluate whether the quality management system is operating effectively. While sampling remains permissible, firms must be able to justify that their monitoring approach provides meaningful insight into how quality risks manifest across the practice, particularly for systemic or recurring issues.

Sampling a sample and the visibility problem firms have always faced

In practice, most firms continue to rely on engagement reviews and periodic inspections as their primary monitoring mechanisms. Inspections examine only a limited number of engagements each year and are often performed after the work has been completed and the audit report has been issued. Once an engagement is selected for review, only portions (or a sample) of the engagement workpapers are subject to a detailed review. So, each year, a sample of engagements is reviewed, and a sample of the workpapers for each engagement is reviewed in detail.

This model has always involved tradeoffs. Firms have long accepted partial visibility because there was no practical way to review every audit file in full, and sampling theory indicates that common/systemic risks would be picked up through the sampling process. Even if that is true, the sampling-based approach would obviously not capture all the quality issues. At best, it would identify a representative sample that could be targeted for improvement via focused continuing professional education for next year’s audit cycle.

As expectations under ISQM 1, SQMS 1, and QC 1000 have increased, reliance on limited, retrospective sampling has become more difficult to support as the primary basis for concluding that quality controls are operating effectively, particularly where risks are systemic rather than isolated.

How Tellen QM fits into a firm’s system of quality management

Tellen QM is a software platform and structured framework that firms use to design, implement, and operate their system of quality management.

It helps firms define quality objectives, identify and assess quality risks, design responses to those risks, and document the policies and controls they rely on to manage quality. In this way, Tellen QM supports the full lifecycle of quality management, from design through operation, monitoring, and evaluation.

Monitoring is a required component of the firm’s system of quality management. Tellen QM does not prescribe a single monitoring approach, but provides a framework that firms can use to define, operate, and document their monitoring controls.

Monitoring as a component of the system

Under the new standards, monitoring is not an afterthought. It is a core component of the system of quality management.

Firms are expected to design monitoring activities that are responsive to their quality risks, proportionate to their size and complexity, and capable of identifying deficiencies on a timely basis. Monitoring must also include evaluation and remediation, with documentation that shows how issues are assessed, root causes are considered, and corrective actions are taken.

Within this monitoring component, firms may define multiple controls. Engagement review and inspection remain important. Increasingly, firms are exploring more structured, recurring forms of monitoring to address scale and visibility challenges.

Defining Continuous Practice Monitoring as a control

One monitoring control that firms may define within their system of quality management is Continuous Practice Monitoring.

Continuous Practice Monitoring is a monitoring control designed to assess, on a recurring, structured basis, whether audit work across the practice is being performed and documented in accordance with the firm’s quality objectives and responses to quality risks.

At present, Continuous Practice Monitoring should be understood as a conceptual and evolving control rather than a fully mature one. It is designed to complement engagement review and inspection, not replace them.

How Continuous Practice Monitoring is currently implemented

Today, Tellen has introduced a capability, QualityScan AI, for firms to use within Tellen QM as part of their Continuous Practice Monitoring activities, and it is in production with clients. It is currently operating in an early stage and is used primarily by quality control reviewers after audit completion.

Using QualityScan AI, firms review the entire audit file for every engagement, not a sample of workpapers and not a subset of audits. Every file is scanned. The initial focus of the scan is completeness, identifying missing required documentation and structural gaps in the audit file.

Over time, the depth and scope of scanning will expand deliberately, from completeness to consistency and then to compliance with firm-defined quality expectations. The scan does not replace professional judgment and does not reach conclusions about audit quality. But it does look at every engagement and surfaces potential quality signals that require human evaluation.

Why AI matters, even at this early stage

A scan alone is not a control. The control is the firm’s documented process for evaluating scan results, determining significance, identifying root causes, and executing remediation within its system of quality management.

Artificial intelligence supports this form of monitoring by enabling consistent review across a larger population of audit files. The monitoring control itself remains grounded in human judgment, documentation, and follow-up, consistent with the requirements of SQMS 1 and the PCAOB’s QC 1000 standard.

While QualityScan AI does not yet operate live during open engagements, it establishes population-level visibility and repeatable evidence of how monitoring is performed. This creates a foundation for Continuous Practice Monitoring to mature over time.

Partnership bridge

Grant Thornton began this work by recognizing that quality management needed to move beyond static documentation and periodic review. In response, Grant Thornton developed qm.x as a structured, system-level framework for defining quality objectives, identifying quality risks, designing responses, and documenting controls that could scale with the firm. Tellen is acquiring the qm.x assets and transforming them into Tellen QM, a framework that firms use to design, implement, and operate their system of quality management, integrated into Tellen’s AI platform to support the operation, monitoring, and evolution of that system in practice. As part of this transaction, Tellen and Grant Thornton are entering into a partnership to continue the next phase of development, ensuring continuity while expanding the system’s capabilities over time.

Where Tellen fits

Tellen connects the framework to execution.

Tellen QM provides a structured framework that firms use to support their system of quality management. Within that framework, firms define monitoring controls, including Continuous Practice Monitoring. QualityScan AI supports the implementation of that control in its current form by scanning every audit file upon completion and helping quality control reviewers identify potential issues.

As the scope, timing, and integration of scanning expand, the control can evolve from a conceptual design to a more continuous operating model. Engagement review and inspection remain essential, but they are increasingly informed by consistent, population-level visibility into the work itself.

Audit quality improves when firms can see the work being done. The integration of Tellen QM and QualityScan AI is designed to help firms build that visibility in a way that aligns with ISQM 1, SQMS 1, and the PCAOB’s QC 1000 standard.