Duolingo’s AI-powered Video Call brings Lily to life

Lily — Duolingo’s purple-haired, perpetually unimpressed teen — doesn’t seem like the chatty type. But despite (or maybe because of) her dry wit and detached demeanor, she’s one of the language-learning app’s most beloved characters. So when Duolingo set out to create Video Call, a more immersive way to practice real conversations, Lily was the perfect candidate. 

Designing an AI character you want to talk to

The challenge was transforming a 2D animated character into an interactive AI-powered conversation partner, one that listens, reacts, and, most importantly, feels alive. To make that happen, Duolingo needed a system that could handle real-time animation, adapt to user input, and create the illusion of a natural conversation, all while keeping file sizes manageable and engineering overhead minimal.

So, they turned to Rive. 

Below is a message from Lily herself.

Why Rive?

Rive is a production-ready design tool for building interactive graphics that respond dynamically to user input. Unlike traditional design tools that export static assets or linear animations, Rive enables stateful animations. In other words, graphics that change based on real-time events, making them perfect for interactive applications like Duolingo’s AI-powered video calls. 

“When I first started at Duolingo, I had never heard of Rive. I thought I was going to be in trouble,” laughs Jasmine Vahidsafa, a senior animator at Duolingo. “But now it’s so core to our workflow, I can’t imagine doing this project without it.” 

Bringing Lily to life with the Rive State Machine

To make Lily feel expressive and responsive, the animation team relied on the Rive State Machine, a visual system that lets animations transition fluidly based on inputs. 

“In the beginning, Lily wasn't engaging enough,” Jasmine recalls. “She would talk and occasionally emote, but we wanted her to feel expressive, actually listen to you, process what you’re saying, and respond in a way that felt dynamic.”

Instead of relying on static, pre-baked animations for every possible reaction, the State Machine dynamically blends animations in real time. That means Lily can lean in when intrigued, look skeptical when confused, or even give an exaggerated eye-roll when unimpressed.  

“We realized that if we animated every possible reaction manually, the file size would be too much,” Jasmine says. “So we thought, ‘Why not break it down into modular pieces and let the State Machine handle the transitions?’ That way, Lily could ‘think’ and react without us having to animate every single frame.”

The Rive State Machine allowed the team to:

• Transition seamlessly between emotions: Subtle head bobs, facial expressions, and idle movements ensure Lily feels present and reactive rather than robotic. 

• Randomized gestures for variety: The team created eight different head animations and eight body animations that dynamically combine, generating over 64 variations of neutral movements. This prevented repetitive idle loops.

• Reduce file size while increasing expressiveness: Because the Rive State Machine handles transitions dynamically, Duolingo avoided bloated, pre-animated sequences, keeping performance smooth and efficient. 

• Adapt to user input in real time: Whether a learner speaks clearly, mumbles, or pauses, Lily’s expressions adjust accordingly, reinforcing meaning through visual cues. 

The Rive State Machine drives Lily's mouth positions, facial expressions, camera movements, and all other animations. “The State Machine is ridiculously powerful,” Jasmine tells us. 

Collaborating in Rive 

Bringing Lily to life required tight collaboration between designers, animators, and engineers. The team met twice a week to sync on updates, making sure animation inputs were named logically for developers and that AI-driven responses matched Lily’s animations in real-time. 

“We call the handoffs “Riv Deliv,” Jasmine says. “Every Monday, I send the team the latest Rive file. It’s always a work in progress, but that’s the beauty of it. We can prototype fast and iterate without breaking everything.”

Building a conversation partner

Lily’s AI-powered video calls play out like a real-life conversation:

1. The phone rings. Lily picks up, tilts her head, and greets the learner. 

2. She listens. Sometimes she leans in closer or furrows her brows in thought. 

3. If there’s a delay in processing, she reacts naturally by tilting her head or pondering, making the wait feel intentional rather than mechanical. 

4. If she’s confused, she might squint or shake her head. 

5. If she’s impressed, she might raise an eyebrow and nod approvingly. 

6. When the lesson ends, Lily signs off with her signature sarcasm before hanging up. 

“If she was too static, learners wouldn’t know how she reacted,” Jasmine explains. “The State Machine lets us create these organic transitions, so if there’s any processing delay, it looks like she’s pondering what you said. It’s subtle but crucial.”

Jasmine describes the process as building a “modular actor.” Instead of hundreds of fixed reactions, the animation team designed reusable motion components that could be combined in endless ways. 

“It’s kind of like a risograph,” Jasmine says. “You layer different parts, like eyes, brows, a mouth, and only when you put them all together do you see the full expression.” 

“In a traditional animation pipeline, you’d animate a character’s reaction based on a fixed script,” Jasmine explains. “But here, we don’t know what the learner is going to say. We needed a system that could mix and match expressions on the fly.” 

Making AI feel human

One of the biggest challenges was balancing expressiveness with Lily’s signature personality.

“She’s not the most emotive character,” Jasmine laughs. “She’s sarcastic, a little deadpan. But she still needs to feel engaged.” 

That meant carefully tweaking her facial expressions and body language to avoid anything too exaggerated. 

“If Lily opened her eyes too wide, people were like, ‘Why is she so happy? This feels weird,” Jasmine recalls. “So we had to dial it back. She’s still Lily, just a version of her that actually wants to talk to you." 

This attention to detail and ability to quickly iterate paid off. During user testing, learners loved how expressive she was. 

“People kept saying how much they liked that she emotes,” Jasmine says. “It made the experience feel way more natural.”

Measuring technical impact

Beyond expressiveness, Rive significantly improved Duolingo’s production pipeline: 

• Nested artboards separated Lily’s head and body for independent movement, enabling more fluid animation.

• Event-triggered expressions allowed AI-driven responses to sync with animations. 

• Performance gains: The final Rive file is under a megabyte despite the complexity. 

“I still can’t believe how tiny our Rive files are,” Jasmine says. “We’re getting all this interactivity in a fraction of the space. It’s kind of wild.”

Dreaming up future applications

Duolingo has already seen positive feedback from learners who find AI-powered conversations less intimidating than real-life speaking practice. With Rive, the team can continue evolving the feature by adding new expressions, refining interactions, and expanding modular animation approaches. 

“Honestly, I don’t even know how we would have done this without Rive,” she says. “The State Machine made everything possible — transitions, animations, blending. It was just… smooth.”

Duolingo doubled down on Rive when they started hiring creative technologists, specialists who combine animation and engineering expertise to optimize Rive files. Learn more about Duolingo’s approach to hiring creative technologists. 

Inspiring animators new to Rive

Jasmine draws inspiration from legendary Warner Bros. animator Chuck Jones, known for maximizing emotional impact with minimal animation. 

“Jones figured out how to do the most with the fewest frames, like Bugs Bunny’s eyebrow wiggle instead of a full reaction. Rive lets us do the same thing. We don’t need 50 separate animations when we can blend just a few perfectly.”

For animators keen on building interactive experiences in Rive, Jasmine shares this advice: 

1. Start small: Learn the foundations of the State Machine before tackling complex projects. 

2. Stay organized: Label animations, inputs, and transitions to simplify debugging. 

3. Think modularly: Instead of animating everything from scratch, build reusable motion components that can be layered dynamically. 

4. Experiment and iterate: “I break things every day,” Jasmine laughs. “That’s part of the process.