APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

Infomaniak Fixes TranslateGemma 4B to Run on Standard Production Servers

Infomaniak released a vLLM-compatible build of Google's TranslateGemma 4B translation model. Custom language-code fields are replaced by a delimiter format: <<<source>>>...<<<target>>>...<<<text>>>... Covers 55 languages, accepts text or 896x896 images for OCR-plus-translate in one call.

Infomaniak Fixes TranslateGemma 4B to Run on Standard Production Servers
Source
AlphaSignal
Published
Author
AlphaSignal
Read
2 min read

Takeaways

  • Infomaniak released a vLLM-compatible build of Google's TranslateGemma 4B translation model.
  • Custom language-code fields are replaced by a delimiter format: <<<source>>>...<<<target>>>...<<<text>>>...
  • Covers 55 languages, accepts text or 896x896 images for OCR-plus-translate in one call.
  • WMT24++ scores: MetricX 5.32, COMET 81.6 for the 4B variant.
  • RoPE config flattened and EOS token fixed so generation terminates correctly under vLLM and SGLang.
  • Fine-tuned from Gemma 3 with 4.3B SFT tokens plus 10.2M RL tokens, under the Gemma license.

Google’s TranslateGemma has been awkward to run on production inference servers because its custom chat template expects source_lang_code and target_lang_code fields inside each message. vLLM’s OpenAI-compatible API does not accept those custom content fields. Infomaniak’s AI team has published a compatible 4B checkpoint that moves the language metadata into ordinary message text.

The conversion lets applications serve translation through familiar vLLM and SGLang endpoints without wrapping the Transformers pipeline. Existing OpenAI-compatible clients can submit requests after formatting the source language, target language, and text into one string.

Compatibility without new weights

The upstream TranslateGemma 4B model is a Gemma 3 derivative fine-tuned for translation. Infomaniak retained its weights and changed three pieces of serving metadata:

  • Prompt serialization: Source and target language codes now appear inside the message string, separated by fixed delimiters, instead of custom JSON fields.
  • RoPE metadata: The upstream Transformers configuration defines separate rotary position embedding settings for full and sliding attention. Rotary embeddings encode token positions for the attention layers. Infomaniak flattened those settings into one linear-scaling block that vLLM can parse.
  • Stopping token: The end-of-sequence token changed from <end_of_turn> to <eos> , allowing the server to stop generation on the model’s expected token.

One string carries the request

Each translation request places the source code, target code, and input text in the user message’s content field. A Czech-to-German request uses this format:

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01huggingface.co
  2. 02huggingface.co
  3. 03arxiv.org
  4. 04huggingface.co