EMNLP 2026 · Industry Track

Cloud and On-Premises Deployment of Uzbek Legal RAG via Targeted Retriever Fine-Tuning

Tatul Danielyan · Mariam Avetisyan · Hrant Davtyan

arXiv Model Metric AI Lab Tuzuk.AI

Abstract

We report on building and operating a retrieval-augmented legal assistant for Uzbek that must run in two regimes: a managed cloud service that maximises answer quality within a per-token cost ceiling, and an on-premises deployment for clients whose legal data may not leave their infrastructure, restricting us to open-weight models on limited local hardware under latency constraints.

Because no evaluation existed for this setting, we build two domain benchmarks — a retrieval benchmark of 178 expert-annotated legal queries with gold provision spans, and an end-to-end benchmark of 504 expert-curated question–answer pairs scored by an LLM judge validated against human judgments and against an independent-family judge. Applying them under each regime, we find the open-versus-proprietary gap is small and cheaply closed by fine-tuning. We therefore train UTE-1, a state-of-the-art text embedder among open models for Uzbek, and release our benchmarks, evaluation code and the fine-tuned embedder.

The transferable lesson: in low-resource legal RAG, fine-tune the retriever once and reuse it; rent or swap the generator.

91.6%

UTE-1 top-5 retrieval — state of the art among open embedders

>+4%

over its base model, closing about half the gap to the proprietary leader

6.6×

cheaper than the frontier tier, and scoring higher

Sovereign

the whole stack runs on client hardware — no legal data leaves the building

Embedders

Top-5 accuracy on our 178-query in-domain legal benchmark · nDCG@5 on Uzbek MTEB

Open weights Proprietary API
# Embedder In-domain top-5 MTEB[UZ]

Generators, end to end

All 504 QA items, scored by a family-independent LLM judge

Open weights Proprietary API
# Generator Retriever Overall Compl. Ctx. acc. Halluc.-free Legal compr.

Deployed stacks: gemini-embedding-001 + gemini-3-flash in the cloud, UTE-1 + qwen3.6-35b-a3b on-premises. 

@inproceedings{danielyan2026uzbeklegalrag,
  title     = {Cloud and On-Premises Deployment of Uzbek Legal RAG
               via Targeted Retriever Fine-Tuning},
  author    = {Danielyan, Tatul and Avetisyan, Mariam and Davtyan, Hrant},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods
               in Natural Language Processing: Industry Track},
  year      = {2026},
  publisher = {Association for Computational Linguistics}
}