Case Study

11 days, 73% improvement:

How SNOMED International proved that AI agents can translate industry specific terms with speed, precision and accuracy

SNOMED International is the developer of the world’s leading clinical terminology (SNOMED CT), used to achieve interoperability between healthcare actors globally.

At present, nations from outside the anglophone world that adopt SNOMED CT need to “localise” it; the principal task being to translate it into the local languages. Translation is an expensive, manual and time-consuming process.

Working with Veratai on a short proof-of-concept (just 11 days!), SNOMED was able to demonstrate that an AI agent could greatly increase the accuracy of machine translations when compared to existing machine translation benchmarks. 

This finding paves the way for SNOMED to develop a translation support service, reducing localisation costs for members and increasing future adoption.

“Veratai’s deep expertise in agentic AI was instrumental in demonstrating that we can significantly improve clinical translation with high precision using AI.

Their support helped us achieve a 75% correct rate in a very short period of time, nearly doubling the accuracy of existing tools and validating a scalable future for our stakeholders' translation efforts.”

— Rory Davidson,
Chief Digital & Information Officer, SNOMED International

Before:

expensive manual translation

  • Translation required a large specialist team of bilingual, clinically trained specialist.

  • Full translation of the terminology costs several million €.

  • Existing automated services (Google Translate, DeepL and LLMs such as GPT-4, Claude and Sonnet) produced a low-quality translation, requiring significant human intervention to use.

  • Translation guidelines evolve, requiring reviews of existing work to standardise the content.

After:

proof that automatic translation is viable

  • In human evaluations, 76% of translations using Veratai’s AI agent met the standard for clinical usage, up from 44% using an off-the-shelf service – a 73% increase.

  • Agentic translations conform much better to local translation guidelines.

  • The agent offers a natural way to automatically align existing translations when guidelines change, something not possible using legacy machine translation services.

  • Additional opportunities to improve the agent have been identified; we believe that the future agentic translation process will exceed 90% of translations meeting the standard.

Bespoke Agent Design

Our Method for helping SNOMED achieve results

  1. Start with the human process: Veratai analysed and understood the current process followed by human translators.

  2. Discover the right data: We prepared training and test data, including:

    • Paired translations

    • Reference materials in the target language

    • A translation style guide

    • Relevant academic works

    • A corpus of relevant medical texts in the target language, which could be cross-checked to verify the translations

  3. Develop the agent: Veratai built a bespoke agent, giving the LLM flexibility to select from different translation resources and iterate on the translations, in just 3 days.

  4. Evaluate the results: With TEHIK, the Health and Welfare Information Systems Centre in Estonia as a partner, localisation specialists “blind evaluated” agentic and traditional machine translations for 100 SNOMED concepts.

From confusion to confidence

SNOMED has established that an Agentic AI approach to translating clinical concepts is a promising route to localising the terminology more cheaply and efficiently. 

They are now designing a Localisation Support Service with the aim of developing the agentic translation capability and turning into a repeatable service that can be provided to existing and prospective member countries.