Here’s how it went.
I took transcripts of my spoken responses and pasted them into ChatGPT. Then I said, “Okay, evaluate my language proficiency.”
One important note—if you try this yourself, make sure to tell ChatGPT that your transcripts are from spoken responses. That’s key. A writing sample is usually more polished and doesn’t reflect the natural pauses, hesitations, or filler words (like “um” or “uh”) that show up in real conversation. Speaking and writing are different, and you want an evaluation that reflects your true speaking ability.
When I did this, ChatGPT evaluated my proficiency level as Intermediate High.
I’ve been experimenting with using ChatGPT as a language proficiency evaluation tool for a couple of years now. What I’ve found is that the margin of error tends to be about one level. I discovered this by testing it with samples that had already been officially scored—specifically AP World Language responses that are publicly available online.
I input both the sample and the official scoring rubric into ChatGPT and asked it to evaluate. In most cases, the result was either a perfect match with the College Board’s score or just one level off. That margin of error is consistent with what human raters experience too. In fact, when humans evaluate AP responses, there are often two raters—and if their scores differ by more than one level, a third rater steps in. So a margin of error of one? Totally normal and acceptable.
After that initial test, I evaluated one of my own samples again—and this time, I got Advanced Low, which still falls within that same margin of error.
Next, I plan to test my Spanish proficiency using the same method—hopefully soon!
To help others do this too, I created a self-evaluation workbook with proficiency tasks at the Novice, Intermediate, and Advanced levels. You can use it with any language. Here’s how it works:
- Choose a task based on your target level.
- Record and transcribe your spoken response.
- Ask ChatGPT to evaluate it using a specific rubric.
- Let it give you feedback and corrections—it’s actually great at offering targeted suggestions for improvement.
- Practice again and keep refining your skills until you’re consistently performing at your desired level.
I’ll share some of my favorite activities that keep middle and high school language learners excited, engaged, and actually acquiring the language—while making your planning (and life) so much easier.