Hand-Simulated AI Session
A person plays the AI solution behind the screen
- PhasePrototype
- Group3 to 6 people
- Time90 minutes
- SettingIn the room
- ExecutivesNo
What it is
A hand-simulated AI session lets people try an AI solution before it exists. A person plays the AI solution behind the screen, following written rules, while the tester uses it as if it were real. It is the AI version of the technique known as Wizard of Oz testing.
When to use it
When you need to see how people react to an AI solution’s behaviour before building anything. This is often called a Wizard of Oz test, after the film where the wizard is a person behind a curtain.
When to skip it
When the question is about technical performance, such as speed or accuracy. That needs a real model.
How to run it
- 10 minBrief the roles: tester, person playing the AI solution, observer, scribe. Say the prototype label aloud.
- 45 minRun two or three scenarios. The tester uses the “system”; the player answers from the written rules.
- 15 minDebrief with the tester. Tell them what was simulated and why.
- 20 minCapture reactions in the Feedback Grid.
About 90 minutes of method time. Add ten to fifteen minutes to open and close the session.
What to watch for
The person playing the AI solution being faster or better than a real one. Not following the written rules. Not telling the tester afterwards what was simulated.
The AI lens
Include one scenario where the AI solution is wrong on purpose. Note whether the tester checks the output, and how long they take.
What you leave with
A test-ready simulated session, and notes on how people used it.
An example
A tester types questions into a chat window. A team member, following rules on a Prompt Prototype Sheet, writes the replies. The team learns what people ask, and where the rules break, before any model is configured.
How it compares
A Hand-Simulated AI Session tests behaviour with real people. A Prompt Prototype Sheet only writes the behaviour down, and a Role-Play Script acts out the process around it.
Related methods
Questions
What is Hand-Simulated AI Session?
A hand-simulated AI session lets people try an AI solution before it exists. A person plays the AI solution behind the screen, following written rules, while the tester uses it as if it were real. It is the AI version of the technique known as Wizard of Oz testing.
When should I use Hand-Simulated AI Session?
When you need to see how people react to an AI solution’s behaviour before building anything. This is often called a Wizard of Oz test, after the film where the wizard is a person behind a curtain.
How long does Hand-Simulated AI Session take and how many people do I need?
Hand-Simulated AI Session takes about 90 minutes and suits 3 to 6 people, plus a facilitator. Add ten to fifteen minutes to open and close the session.
How is Hand-Simulated AI Session different from other methods?
A Hand-Simulated AI Session tests behaviour with real people. A Prompt Prototype Sheet only writes the behaviour down, and a Role-Play Script acts out the process around it.
Can I run Hand-Simulated AI Session remotely or with executives?
Hand-Simulated AI Session works best in the room, because it needs people physically working together on the same material. Plan a room, wall space and the printed canvas. Executives: No, run it with the working team.