How to create module comprehension audits

Summary A module's comprehension audit is created in the Module Publisher. You choose the audit type, the auditor (usually the supervisor) and when the audit is triggered, add a pre-audit checklist and an overview, then build the audit's questions, each with a correct and an incorrect answer. Supervisors can then check, hands-on, that students can apply their learning in real-world situations, rather than just pass an online quiz.

See also: Comprehension Audits, how they work for learners.

Steps

Introduction. PaT's experience shows that a comprehension audit can increase the value of a module by over 300%: it takes learning beyond the classroom and applies knowledge in real-world environments. Business unit leaders play a key role in making sure training is high quality. Every module should include a comprehension audit, to confirm that knowledge is genuinely understood. This makes sure standards are truly met, and stops leaders from simply checking boxes, which undermines the development and growth of their teams.

Why audits matter. Comprehension audits evaluate how well students apply what they've learned on the job, using real-world tools and environments. Online quizzes can test knowledge, but multiple-choice questions often miss important learning gaps. Comprehension audits, run one-on-one by a person, are essential for finding those gaps, and give a fuller picture of how well the student has grasped the material.

Note. The strength of comprehension audits comes from open-ended questions, such as "Show me", "Where is the menu and what item do you need to select?", "How would you respond if this [insert statement] was said?" and "Why is this important?". They make students show real competency in practical, real-world scenarios, without hints or clues. That gives a much more accurate assessment of their skills and understanding than passing an online test, or signing a form to say they understood the material.

How an audit is triggered. When a module includes a comprehension audit, PaT automatically assigns a task to the supervisor after the staff member completes the online quiz, with a due date 5 days later. The supervisor then schedules time to run the audit, whether that is showing how to use software, lifting correctly, or role-playing a difficult conversation. Comprehension audits significantly raise skill levels.

Note. Comprehension audits are designed for practical, hands-on, situational learning, and to be completed on a mobile phone, so supervisors can run the audit directly in the context of the learning material.

Setting up the audit

Step 1

In the Module Publisher, with the module open, click the comprehension audit button: the third of the three icons under the PaT logo at the top left (a checklist with a person). The create/edit an audit window opens.

Step 2

Choose the Audit Type, at the top left of the window. It is set to Comprehension automatically, but you can also choose an Evidential Audit or a Notification Audit. The How to create module evidential audits module covers evidential audits.

Step 3

Choose the Auditor, to the right of Audit Type. It is set to Supervisor automatically, but you can also choose any staff member to be the auditor.

Step 4

Choose when the audit will trigger in Audit Trigger, under Audit Type. There are 2 options. Every triggers the audit every time the module is passed.

Step 5

Recurring triggers the audit on a recurring basis instead, every so many days or months. Two more fields appear to the right of Audit Trigger: set the number in Audit and the unit, such as days, in Frequency.

Step 6

Add a checklist for the auditor to verify that the student is prepared for the audit. Click the + at the top right of the Pre-Audit Checklist box, on the right of the window. The add a new checklist item box opens. The checklist makes sure the student is ready to show competency, and can include things like: - Confirm they have 5 minutes scheduled for the audit. - Make sure they have access to the software application needed. - Check they have the tools ready to show proper use.

Step 7

Type the checklist item in the box, then click Done to its right.

Step 8

The item now shows in the Pre-Audit Checklist, with a tick box at its left. Some audits need props or business tools, such as computers, so the student can show practical knowledge of the subject; the checklist explains what is needed.

Step 9

In Overview, under Audit Trigger, write an overview of the audit, so the auditor can explain to the student why the audit is important. The box starts with a sample sentence.

Adding questions and answers

Step 10

Now create an audit section: click the + to the right of the Audit(s) heading, at the bottom left of the window. The add an audit box opens.

Step 11

Type an overview of the audit section, then click Done. The section appears in the Audit(s) list, with its ID and overview.

Step 12

To edit or delete an audit section, use the edit (pencil) or bin icon at the right of its row.

Step 13

To build the section's questions and answers, click the section in the Audit(s) list, so that it is highlighted in dark grey. You may end up with several audit sections, which is why you select the one you want to work on.

Step 14

The Question(s) and Answers boxes appear to the right of the list. Click the + at the top right of the Question(s) box to add a question. The add a new question box opens.

Step 15

Type your question, then click Done at the bottom of the box. The question appears in the Question(s) list.

Step 16

Click the question in the list, so that it is highlighted grey. This tells PaT it is the question you want to add answers to.

Step 17

To add an answer to the question, click the + at the top right of the Answers box, on the right. In the Add a new Answer box, type the answer and click Done.

Step 18

Add the correct answer, for example a statement that the student did what the question asked.

Step 19

Add the incorrect answer the same way, for example a statement that the student could not do it. Always give each question both a correct and an incorrect answer, so that PaT can certify or decertify the student.

Step 20

In the T/F column, at the left of the Answers box, tick the box beside the correct answer.

Previewing the audit

Step 21

Click Save, under the Pre-Audit Checklist, then Preview (save first) to its right. A preview of the audit opens, titled Please complete this Audit.

Step 22

Near the top, the preview shows the Auditor, the Person to complete this Audit and the audit's Overview.

Step 23

Under the overview is the Pre-audit checklist. Click Start Audit at the bottom of the box.

Step 24

The first question appears in a blue box: it is the question the auditor will ask. The auditor should not give the answers to the student. It is up to the student to recall and recite the answer as well as they can.

Step 25

The answers are listed under the question. The auditor selects the one that best represents the student's result, and it turns light blue. In the background, PaT checks whether that answer is the correct one, and passes or fails the student based on how you set the audit up. The auditor then clicks Next Question > at the bottom of the box and repeats this for each question you designed, until the audit is complete.

Tip. Take time to design audits that challenge students and find where they may need support. Children are naturally curious, but by adulthood that passion often fades, because the education system didn't nurture it. Even when learning feels forced, it can boost a person's self-worth, and auditors can help restore it. Learning, failing and getting one-on-one guidance from a caring auditor can reignite a student's love of learning and make a real difference to their growth.

Why audits work. Comprehension audits bring learning from books, PCs or classrooms into the real world. Students are challenged to show what they've learned, and get guidance where they need support. This "train-to-the-gap" approach recognises that students can master 80% of the material on their own, with the teacher focusing on the remaining 20%. It makes good use of resources, and showing their skills in practical settings gives students a strong sense of achievement and self-worth. Comprehension audits are an investment in better learning outcomes.