Case study · Food & beverage · Saudi Arabia
Ordering coffee should be simple. Say what you want, answer a question, get your drink.
Teaching a kiosk to take orders in Najdi Arabic
How Grootan built a live AI voice-ordering kiosk for a Saudi coffee chain that understands everyday Saudi Arabic, follows the menu and replies within two seconds.
Chapter 01 / The ask
Order the way you would at the counter. In Najdi.
A coffee chain in Saudi Arabia wanted a self-service kiosk that customers could simply talk to. In English, or in Najdi, the everyday Arabic of central Saudi Arabia. It had to feel as natural as ordering from a barista.
That sounds like a speech problem. It is really three problems at once: hearing a dialect most services handle poorly, answering fast enough that the pause doesn't feel broken, and following a menu where every drink has its own questions.
“I'd like a hot medium Spanish latte with cream and a double shot.”
- Spanish latte1
- HotMediumDouble shotCream
How many sugars?
One sentence fills four details. The only correct next question is sugar. Illustration based on the real order flow.
Reply budget
At a counter, a longer pause feels like the machine has frozen.
The dialect
Everyday Saudi Arabic, often with English drink names in the middle of a sentence.
The menu logic
Hot drinks ask about size, sugar, espresso and flavour. Cold drinks skip sugar. Never ask twice.
Chapter 02 / First attempt
A quick prototype, and what it got wrong.
We started with managed cloud services to learn how far they could take us. The kiosk detected when the customer stopped speaking, uploaded the recording, and sent it through a chain of services.
Replies were quick, because almost every one was a video that already existed. Understanding was the weak point. When we ran 167 everyday ordering phrases through the translation step, this is what came back:
“Add cream”
→System understood“Dear guest”
“Add more ice”
→System understood“Snow supply”
“I want a large”
→System understood“My father is big”
The Najdi word for “I want” reads as “my father” in standard Arabic, and our sample order starts with the same word. Intent matching also broke on orders with several items, and pre-recorded replies meant the kiosk could say little beyond what had been filmed. The prototype did its job: it showed exactly which parts had to change.
Chapter 03 / Rebuild
Stream everything. Read Najdi directly.
Two lessons shaped the rebuild. Replies had to be generated live, so the kiosk could answer anything. And Najdi had to be understood directly, without translation. That turned two seconds into a budget shared by every step, from the microphone to the first spoken word.
Listening in a busy café
An always-open microphone also hears the coffee machine and the next customer. We clean the audio in four layers before it is transcribed.
- 1Face detection unmutes the mic only when someone is at the kiosk
- 2A noise filter on the kiosk removes machine noise
- 3Background voice cancellation keeps only the main speaker
- 4Tuned speech detection ignores brief chatter
Hearing Najdi correctly
One speech service averaged 78.1% on 120 test phrases, too low for orders that must be exact. We evaluated three routes.
A language model now read the Arabic directly and returned the order as structured data, so the screen could show each item as it was understood. We chose a cloned Najdi voice from about 40 Arabic voices. About 80% of orders came out right. Then we hit a wall.
Every drink has its own question sequence. Each fix we tried made the kiosk slower or less reliable.
The full menu and rules sent every turn. Replies took about 3.5 seconds, with extra cost even with caching.
Even after cutting the tools from about 17 to 4, each look-up added a round trip.
Faster, but it skipped required questions or asked ones that didn't apply.
Chapter 04 / Breakthrough
Stop telling the model the menu. Teach it.
We kept the real-time pipeline and replaced only the model. Instead of a large model reading the rules on every turn, we trained a small one that carries the rules in its own weights, and hosted it ourselves.
Pick the fastest model that can learn
We benchmarked three sizes of Qwen3 on the same GPU with a menu-length prompt. Time to the first word decided it.
Relative time to first token. Shorter is faster. An 8B trial on more data trained only slightly better and answered more slowly, so we kept the 4B.
Write the menu down once
The whole menu became one specification: six categories, the question order, allowed values, defaults and the steps each item skips.
# from the menu specification
- category: hot
flow: [size, sugar_count, espresso, flavor]
defaults:
sugar: white
milk: full_fat_milk
subcategories:
- name: espresso_based
items:
- name: spanish_latte
sizes: [m, l]
skip: []
- name: cortado
sizes: [one_size]
skip: [espresso]Build the training data from the spec
There were no real customer conversations to learn from. So we generated them: 11,419 partial orders walked from the specification, turned into conversations by another model, checked by a review pass (273 dropped), with the Najdi versions reviewed by native speakers. Edge cases were written by hand.
- Orders, English and Najdi 20,177
- Several items in one sentence 3,071
- Unknown items 1,575
- Login, language switch, menu, cart 3,059
- Questions and suggestions 915
An automated check blocks any malformed example from reaching training. Early datasets with missing quantities had held accuracy back.
LoRA fine-tune of Qwen3 4B, training only on the replies the model must produce.
vLLM on a single NVIDIA L4 GPU in Google Cloud's Dammam region, inside Saudi Arabia.
The backend checks every reply against the same specification. A failed reply gets one correction and one more try.
Chapter 05 / One order, end to end
Under two seconds, from last word to first reply.
Here is what happens after the customer finishes asking for that Spanish latte. The voice and the screen work from the same streamed reply, so neither waits for the other.
The turn detector decides the sentence is complete and the transcript is finalised.
Qwen3 4B writes the order as structured data, with the spoken reply first.
Voice: speech generation starts on the reply text.
Screen: the rest streams in, is checked and updates the cart.
…and is asked about sugar. Under 2 s for 9 in 10 replies.
Chapter 06 / Where it landed
From a demo that misheard to a kiosk customers use.
The kiosk is live with the coffee chain. In testing on more than 1,000 orders, 90% came out fully correct, down to the size and every customisation.
V1 · Prototype
Managed services
mostly pre-recorded clips
V2 · Real time
Long prompt
generated live
V3 · Live
Small model that knows the menu
for 9 in 10 replies, generated live
Final figures from testing on more than 1,000 orders, from the test set and from the team ordering at the kiosk. Earlier figures are rough results from testing at the time.
Customers stay in control
The order appears on screen as it is taken. If an item is misheard, a tap fixes it, and the tap joins the same conversation as speech. Those taps catch most of the orders the model alone gets wrong.
Honest trade-offs
Speech recognition, voice and face login still run on third-party services. Small menu changes go into the prompt as a note; larger ones are batched into a retrain that takes two to three days. A loud voice right next to the customer can still get through.
Chapter 07 / Lessons
Like an experienced barista, the kiosk now knows the menu by heart.
A good voice interface needs more than accurate speech recognition. It has to understand the dialect, follow the business rules and answer fast. Getting all three took moving the rules out of the prompt and into a small model built for the job.
- Let failures pick the next fix. Every version was judged on test orders, and what it got wrong decided what changed.
- Write the rules down. One precise specification replaced the training data we didn't have and became the reference for every live check.
- Small can beat big. A 4B model trained on the rules was faster and more accurate than a larger model reading them.
Work with us
Planning voice ordering or an AI agent that speaks your customers' dialect?
Talk to the team that built this kiosk.