1. How easily and successfully do users—who are new to the writing system and new to Duolingo—use Duolingo's “learn a new writing system” tool?
2. What worked well for users while learning a new writing system?
3. What obstacles or frustrations did users experience while learning a new writing system?

We sent out a screener looking for participants interested in learning a new language with an unfamiliar writing system. We focused of people interested in Japanese and Korean because there was the most interest in those languages.

User interviews are a fast way to learn about users and what they think, feel, and need. My team conducted 10 60-minute, in-depth user interviews (I conducted two) using this moderator's guide. While 5-8 interviews would have been sufficient, all five researchers wanted to conduct interviews.

We chose to do all of our research on the desktop website because casual Duolingo users reported that they preferred to use desktop (the mobile app is limited unless users pay for premium).

We had participants complete three tasks:

1. Navigate to Duolingo’s homepage and explain what it communicates to them
2. Navigate to and explain the Characters page
3. Complete two lessons on either Japanese or Korean Characters

All ten participants completed the tasks successfully and without any assistance. They reported that the interface was easy to use, clean, and intuitive (easy to jump in & get started). The auditory features helped them remember the pronunciation of the characters they were studying.

The Korean langauge had a "build character" exercise, and participants assumed they should drag & drop instead of clicking on components that automatically filled the blue section of the character. We recommend adding a more thorough explanation that Korean characters are made of building blocks. Additionally, it might be helpful to include an animation of a few characters being built.

Participants on both languages were unsure how the "matching" questions worked - most people assumed they would drag & drop instead of just clicking on one question from each column. Some people didn't notice there were two columns because they are so close to each other. We recommended making a clearer distinction between the left and right column, and drawing a line connecting the two selected buttons.

Most participants did not notice Duolingo displayed the correct answer after they got a question wrong. No participant commented on the fact that questions they got wrong were repeated until they got the answer right, and when asked, participants were unsure as to when they would get repeat questions. We recommend that the correct answer be displayed more prominately and that Duolingo communicates that people will be asked the same question over and over until they get the answer right (e.g., “Don’t worry. We’ll try this question again.”). When those questions are displayed again, they could be flagged with the note “Let’s try this one more time”.

Participants were unsatisfied that they were not quizzed on their ability to write characters.

“[Duolingo] didn't have a feature to learn how to write it. [I] don't think people who are new to the language will be able to write it from this lesson. Maybe only be able to recognize the character and the pronunciation...”

We recommend that Duolingo continue their roll out the beta feature “trace a character” which we saw during one of our user interviews.

Users wanted to control the content of their lessons and understand the purpose of the content. For example:

• Choose which characters to learn next
• Want to see simple vocabulary and / or grammar integrated
• Want to understand how lessons play a part in learning the language overall

From prior experience, the moderators knew that characters were picked so that users would learn enough characters to start learning actual words. We recommend that Duolingo convey that, so users don't worry about which characters they are learning, and know that more engaging content is coming sooner. It would be beneficial if there was some kind of lesson plan or syllabus that explained why repetition is helpful, what they’ll be learning and when, and pushed people to use the “tips” page, which explains the pronunciation rules and gives more explanation for lesson content.

Because we had no budget, we struggled to recruit diverse participants. Most of our participants had experience learning foreign language(s), and several had experience learning new writing systems.

Because we have no affiliation with Duolingo, we were not able to use any internal data. Not having stakeholders made it difficult to pick our direction and narrow our scope because we were making decisions based on our interests and guesses as to what might be impactful.

Originally, I wanted to run this study as a diary study in order to understand when users figure out certain features of Duolingo and how they feel when things click. Unfortunately, due to the scope of the course, and the lack of budget, we only did moderated interviews.

We only did a cursory exploration of the streak on Duolingo, but I think that could be an entire study in and of itself.