a language by passively turning pages in a textbook.
You actually progress when the language talks again to you.
Once you see photos, hear actual sentences, attempt to converse, and get suggestions, the whole lot lastly clicks in your head.
Previously, you wanted a instructor by your facet always to get that type of suggestions.
Right this moment, generative AI can play that position in your cellphone or laptop, like an AI language tutor you need to use any time.
Once I began studying Mandarin ten years in the past, I noticed many foreigners struggling to be understood by locals in on a regular basis conversations due to poor pronunciation.
It satisfied me that with out good pronunciation, a wealthy vocabulary is ineffective.
I nonetheless bear in mind sitting in my condo in Shanghai, repeating the identical sentence time and again, with out anybody to appropriate me.
Years later, after I found generative AI, I remembered the engineer in China who was fighting grammar books and tones.
I needed to construct instruments that may have helped me up to now.
As a startup founder, I don’t have a lot free time, so I wanted a approach to construct and check new instruments rapidly.
That’s the reason I turned to n8n to construct assistants that may have made my Chinese language apply a lot simpler.
On this article, I’ll present how I take advantage of n8n and multimodal AI to construct a “examine companions” for language studying that:
- Right my pronunciation utilizing Textual content-to-speech capabilities
- Create workout routines to check vocabulary lists
- Generate photos as an instance phrases or contexts for flash-card fashion apply
Collectively, they present how AI and low-code platforms like n8n can help anybody studying a posh language.
Even with every day utilization, all of those collectively value lower than 1 euro per 30 days.
AI For Pronunciation And Oral Comprehension
My identify is Samir, a provide chain skilled who struggled with Mandarin throughout his six-year keep in China.
Let me introduce you to Yin, the AI-powered Language coach I developed final week.
It is a internet software I designed to help my Chinese language studying journey after greater than 5 years with out practising.
It consists of three options:
- Pronunciation Workout routines
- A number of Alternative Questions (MCQ)
- Flash Playing cards
I’ll use every function to exhibit how I take advantage of multimodal AI to enhance my studying comprehension, listening, and pronunciation in Mandarin.
Why is pronunciation in Mandarin so Vital?
Let me share an actual story from China to spotlight the significance of utilizing the proper tone in Mandarin.
Sooner or later, I used to be invited to a job interview on the largest Chinese language categorical firm, valued at billions.
All the dialog was in Chinese language.
I had rigorously ready my sentences, highlighting how I used information science to enhance warehouse operations.
At one level, I needed to say: “I take advantage of information science to enhance selecting productiveness within the warehouse.”
The verb “selecting” means taking items from cabinets or racks in a warehouse.
In Chinese language, my colleagues used the verb 拣货 (jiǎn huò) to explain this course of.
However as an alternative of claiming jiǎn huò, I mentioned jiàn huò.
Which is a very totally different phrase that you just positively don’t need to use in a job interview.
To maintain it well mannered right here, let’s say jiàn huò is a impolite phrase.
The supervisor burst out laughing.
I didn’t perceive why till I debriefed with the headhunter later and repeated the sentence for her.
That second taught me that pronunciation in Chinese language isn’t nearly sounding pure.
You possibly can know hundreds of phrases, but when your tone is incorrect, individuals gained’t perceive you.
Because of this the primary function of my app is an AI pronunciation coach.
Utilizing Speech-to-Textual content Recognition to Practise
Utilizing speech-to-text and reasoning, the app listens to what I say, compares it with the goal sentence, and offers suggestions on which tones or syllables have been off.
The main focus right here is on enhancing my pronunciation of logistics and provide chain phrases (my discipline of experience).
For every phrase, we’ve:
- The phrase in Simplified Mandarin Characters: 合同
- The sentence used to practise my pronunciation: 我们需要在发货前签署这份运输合同。
- The English translation: We have to signal this transport contract earlier than delivery the products.
For freshmen, we will even add phonetics (Mandarin pinyin) utilizing the toggle.
The right way to apply pronunciation?
I simply should press the mic button on the backside to document my sentence.
The recording is routinely despatched to the backend for evaluation that compares my pronunciation with the proper one.
Just a few seconds later, I obtained my suggestions.
The suggestions is sort of detailed; it focuses on the phrases that you just mispronounced.
It’s practically like having a private instructor correcting me in actual time, besides this one by no means will get drained.
After all, this gained’t change a terrific instructor in a one-on-one lesson, however it might show you how to to practise after courses.
Once I began studying Mandarin, I used to spend evenings (after work) alone, repeating easy sentences to familiarise myself with the nuances of tones.
I didn’t have a suggestions loop on the time; this instrument would have been very useful.
How does it work?
Textual content-to-speech and reasoning capabilities of GenAI
The backend is an easy n8n workflow linked to the frontend through a webhook.
The text-to-speech capabilities are used to transcribe the audio file despatched by the entrance finish into phonetics (pinyin).
The output of this Gemini audio transcription node consists of the phonetics:
[
{
"content": {
"parts": [
{
"text": "zuò pǐn huò zǒnggòng fàng zài èrshí ge tuōpán shàng.n"
}
],
"position": "mannequin"
},
"finishReason": "STOP",
"avgLogprobs": -0.16858814502584524
}
]This pinyin is then despatched to the AI node Pronounciation Evaluation together with the goal pronunciation.
On this instance, I mispronounced the penultimate phrase.
That is exactly what the agent talked about in his suggestions.
This exhibits how we will use text-to-speech capabilities, mixed with the reasoning of generative AI fashions, to enhance our pronunciation.
This may be tailored to any language.
What about picture era and speetch-to-text?
Generative AI for Content material Era
In the event you observe the consumer interface of the appliance, you discover that every phrase has:
- An illustrative Picture
- A sentence for the context
- Audio transcription obtainable through the microphone icons
This content material is generated utilizing AI fashions to supply quite a lot of instructing supplies for the second function: flashcards.
Textual content-to-Speech Options
A good way to practise pronunciation is to pay attention and repeat.
Due to this fact, earlier than recording my sentence, I can discover ways to pronounce the phrase utilizing this primary speech-to-text function.
For this, I take advantage of Google’s Textual content-to-Speech API as it’s fairly handy and free.
from gtts import gTTS
def generate_speech(textual content: str, lang: str):
filename = f"{uuid4().hex}.mp3"
filepath = f"./information/gtts/{filename}"
tts = gTTS(textual content=textual content, lang=lang)
tts.save(filepath)With a few traces of code, you may generate the text-to-speech of any phrase utilizing the right language code.
That is precisely what I used within the instrument to generate flashcards that I offered on In the direction of Information Science three years in the past.
The thought on the time was to enhance my listening comprehension by including audio to the flashcard solutions.
What about lengthy sentences?
The issue with Google Textual content-to-speech is the robotic voice.
Thankfully, we’ve eleven labs.
The workflow above is linked to the app through webhook.
The Eleven labs node that takes the output of the AI Agent Generate Instance to generate the audio model of the sentence.
The consumer can now take heed to the sentence pronounced “like” a local speaker.
What’s remaining? Questions and illustrations …
Educating materials era
As defined within the earlier part, the sentences are additionally generated utilizing AI.
The AI Agent node, powered by Gemini, takes the phrase to check as enter and makes use of the system immediate under to generate a sentence.
You're a Chinese language language tutor for professionals.
Given a Chinese language phrase, you MUST return a JSON object with EXACTLY these keys:
- "sentence": a brief Chinese language sentence utilizing the phrase in a enterprise or
daily-life context
- "pinyin": the pinyin of the complete sentence
- "english": the English translation of the sentence
Return ONLY legitimate JSON. No explanations, no backticks, no additional textual content.
Instance:
{
"sentence": "我去仓库检查货物。",
"pinyin": "Wǒ qù cāngkù jiǎnchá huòwù.",
"english": "I am going to the warehouse to examine the products."
}That ensures a virtually infinite number of workout routines.
And the cherry on the cake is the picture generated with Gemini’s Nano Banana to assist us join a phrase to its context.
After studying hundreds of Chinese language characters, I observed that photos assist with memorising new phrases.
That is exactly what I take advantage of within the flashcards function.
The n8n backend supplies to the front-end:
- The phrase in Chinese language that you just need to be taught with pinyin and English translation
- An instance sentence and its translation generated by GPT
- An illustrative picture generated by Gemini
The entrance finish then manages the card-flipping mechanism.
If you wish to recreate this resolution tailor-made to your wants, I’ve shared an identical workflow on my GitHub.
Do you want multiple-choices questions? Gen AI might help!
Generate Workout routines from a vocabulary checklist
For the final function, we generate multiple-choice inquiries to be taught the identical vocabulary checklist.
We ask Gemini to generate questions from the vocabulary checklist, utilizing multiple-choice choices with just one appropriate reply.
[
{
"output": {
"question": "Which of the following is the correct Chinese translation for 'Variable Pricing'? Please answer with A, B, C, or D.",
"options": {
"A": "仓库",
"B": "可变定价",
"C": "卡车司机",
"D": "投标"
},
"correct": "B",
"right_feedback": "Great job! 可变定价 (kě biàn dìng jià) means Variable Pricing.",
"wrong_feedback": "Oops! The correct answer is B: 可变定价 (kě biàn dìng jià), which means Variable Pricing."
}
}
]The front-end makes use of this output to supply the questions with tailored suggestions.
The backend of this function relies on an n8n workflow that I additionally shared on my GitHub: AI-Powered Language Instructor utilizing GPT.
Conclusion
I developed this app to experiment with how AI may improve my studying capabilities.
After practically 5 years with out talking Chinese language, this multimodal AI assistant has confirmed to be a terrific assist.
All the backend is constructed on n8n for fast prototyping and seamless integration.
You aren’t conversant in n8n and need to be taught?
I’ve a whole tutorial, designed for freshmen, on my YouTube channel that can information you from occasion creation to credential setup.
After this tutorial, it is possible for you to to make use of any of the workflows shared in my repository.
As I don’t have time to decide to in-person Chinese language courses, I can have an assistant who will adapt to my schedule.
Can we do higher?
On the “roadmap” of this small facet undertaking, I’ve:
- Including advanced grammar workout routines that might be accomplished orally (combining studying comprehension, grammar and pronunciation)
- Implementing a writing module that may appropriate my calligraphy utilizing picture processing
Relying on my availability, I’ll purpose to ship it by Q1 2026.
About Me
Let’s join on LinkedIn and Twitter; I’m a Provide Chain Engineer utilizing information analytics to enhance logistics operations and cut back prices.
For consulting or recommendation on analytics and sustainable provide chain transformation, please contact me through Logigreen Consulting.
