Ali Mrani Alaoui·Portfolio

Immersive AI Roleplay with Avatars

Avatar-based roleplays for the hardest conversations educators face, with on-the-fly lip-sync, dynamic facial expressions, multilingual speech, and AI voice evaluation.

AI RoleplayAI AvatarVoice Evaluation
Industry
Education / Soft Skills Training
Timeline
10 weeks
Tools
HeyGen, Synthesia, ElevenLabs, Custom code
Learners take part in an immersive Roleplay built for the hardest conversations educators face.
A Roleplay Coach avatar introduces the persona and the goal of the session.
The avatar speaks in multiple languages, switching naturally mid-conversation.
Facial expressions shift dynamically, with lip-sync generated on the fly.
Learners speak their answer, and the AI evaluates tone, pacing, and word choice.
A personalized debrief sends them back into the next attempt.

Project Overview

A teacher training program for new educators in their first two years on the job. The instructional coaching team was burning out doing 1:1 roleplay sessions every cohort. They wanted a self-serve practice tool that handled the same scenarios coaches were running in 1:1s — parent conferences, classroom redirection, peer disagreement — without requiring a live coach in the loop.

The brief was a self-serve practice tool for the program's new-teacher cohort. It had to handle the scenarios coaches were running in 1:1s — parent conferences, redirection conversations, peer disagreement — without requiring a live coach in the loop.

A Roleplay Coach avatar introduces the persona and the goal. A second avatar plays the partner. From there, learners speak with their own voice, and the avatar speaks back with on-the-fly lip-sync and shifting facial expressions. The AI evaluates each spoken turn on tone, pacing, and word choice, and surfaces a personalized debrief with a one-click retry.

The avatar switches languages mid-conversation when the scenario calls for it. The voice rubric was co-authored with the coaching team so the AI scores what the coaches actually score. Coaches now use 1:1 time to review transcripts the learner has already practiced, not to act out scenarios from scratch.

What's Inside

8 roleplay scenarios

Across parent meetings, classroom redirection, peer disagreement, and admin debriefs.

2 consistent avatars

A Roleplay Coach for intro and debrief, and a roleplay partner for the live scenario.

Voice input with live scoring

Streaming Whisper transcription feeds the same evaluation as typed text.

Voice evaluation rubric

Tone, pacing, and word choice, scored against a coach-authored rubric.

Multilingual avatar

Switches languages mid-conversation; voice, lip-sync, and rubric all follow.

Personalized debrief

Strengths, focus areas, and a one-click retry button.

What the Client Provided

  • Recorded 1:1 coaching sessions with consent (the source for the scenario library)
  • The coaching team's “hardest conversations” list (Google Doc)
  • Persona briefs for each of the 8 scenarios
  • Existing tone-and-pacing rubric from the coaching team, in both languages
  • Bios and avatar reference photos for the Coach and Partner avatars

Design and Build Process

01

Scenario Authoring

Worked with the coaching team to pull the scenarios educators actually dread. Each got a goal, a persona, an escalation map, and a clear definition of done.

02

Avatar Pipeline

Built the avatar pipeline on HeyGen with custom expression triggers and a streaming visemes layer for sub-frame lip-sync.

03

Voice Eval Rubric

Co-authored the rubric with coaches: what counts as a clear tone, a measured pace, and language that lands. Tuned the AI to the rubric, not a generic sentiment model.

04

Multilingual Layer

Added language-switching mid-conversation. The rubric was co-authored in both languages so scoring stays consistent across the switch.

05

Pilot and Retry Loop

Piloted with two cohorts of new teachers. Reworked the debrief screen until 80 percent of learners hit the “Try Again” button without prompting.

Tools and Stack

AI Avatars
HeyGen and Synthesia for avatar rendering, with custom expression direction.
AI Voice
ElevenLabs for the partner voice, OpenAI Whisper for streaming learner input.
AI Engine
OpenAI and Anthropic Claude for in-character dialogue and tone/pacing scoring.
AI Video
Veo 3.1 for cinematic scenario intros.
Authoring
Articulate Storyline 360 for the surrounding lesson structure.
Custom code
Custom-built roleplay interface with multilingual switching and a debrief surface.
Delivery
Embedded directly into the training program's website, with SCORM 2004 + xAPI packages for Cornerstone, Canvas LMS, and LearnDash.

Deliverables

  • Live roleplay tool (web app)
  • 8 scenario packages (persona, goal, escalation map, rubric)
  • Voice evaluation rubric (Google Doc, in both languages)
  • Avatar expression and animation library
  • Coach-facing transcript viewer for 1:1 review
  • Embedded directly into the training program's website, with SCORM 2004 + xAPI packages for Cornerstone, Canvas LMS, and LearnDash
  • Train-the-coach guide and a recorded walkthrough