Internationalization (i18n) & Localization (l10n) Guide
This document explains how the TomoTree multilingual system works, how to mark new UI strings for translation, how to extract them to the template, and how to add a completely new language.
Table of Contents
1. Architecture overview
TomoTree uses a custom JSON-based i18n system instead of the standard
Python gettext binary (.mo) approach. The reasons are:
No binary compilation step (
msgfmt);xgettext/msgmergeare only needed when extracting new strings (see §5), not to run the app.JSON files are human-readable and easy to diff in version control.
Works identically on every platform without gettext installed.
The translation pipeline looks like this:
Python source files Translation template Language-specific files
st.button(_("Save")) --> locale/messages.pot --> locale/cs/LC_MESSAGES/messages.po
locale/de/LC_MESSAGES/messages.po
locale/es/LC_MESSAGES/messages.po
...
po_to_json.py
↓
locale/cs/messages.json ← loaded at runtime
locale/de/messages.json
...
The core module is src/tomotree/i18n/__init__.py. A single global dict
_current_translations maps English source strings → translated strings. The
function _(message) does a dict look-up and returns the original string
unchanged when no translation exists (safe fallback to English).
2. Directory layout
src/tomotree/i18n/
├── __init__.py # Core module: _(), init_i18n(), set_language()
├── po_to_json.py # Compiler: converts .po files → .json
├── build_translations.py # Legacy compiler (generates .mo, not used at runtime)
└── locale/
├── messages.pot # Master template — all source strings, no translations
├── cs/
│ ├── LC_MESSAGES/
│ │ └── messages.po # Czech translations (edit this file)
│ └── messages.json # Generated — do NOT edit by hand
├── de/
│ ├── LC_MESSAGES/messages.po
│ └── messages.json
├── es/
│ ├── LC_MESSAGES/messages.po
│ └── messages.json
├── fr/
│ ├── LC_MESSAGES/messages.po
│ └── messages.json
├── it/
│ ├── LC_MESSAGES/messages.po
│ └── messages.json
├── uk/
│ ├── LC_MESSAGES/messages.po
│ └── messages.json
└── zh/
├── LC_MESSAGES/messages.po
└── messages.json
Rule: Only ever edit the .po files by hand. The .json files are
generated and must never be edited directly.
3. How it works at runtime
Initialization
src/tomotree/streamlit_app/app.py calls init_i18n() on every run:
from tomotree.i18n import init_i18n, _
init_i18n() # reads st.session_state.language, loads JSON, renders flag buttons
init_i18n() injects flag buttons into the Streamlit sidebar. When a user
clicks a flag, set_language(code) loads the corresponding messages.json
into _current_translations and triggers st.rerun().
Translation look-up
Every call to _("Some string") does:
return _current_translations.get(message, message)
If the string is missing from the translation file (or English is selected), the original English string is returned unmodified — no exception is raised.
Plural forms
from tomotree.i18n import ngettext
label = ngettext("1 node selected", "{n} nodes selected", n).format(n=n)
ngettext chooses the singular or plural key and then delegates to _().
Complex plural rules (Czech, Ukrainian, etc.) are declared in the .po file
header (Plural-Forms:) and are informational only — the runtime uses a
simple count == 1 rule; full plural inflection must be handled in the
msgstr value itself.
4. Marking strings for translation in Python code
Import the function
Every Streamlit module that shows user-visible text must import _:
from tomotree.i18n import _
Basic usage
Wrap any user-visible string literal with _(...):
# Before
st.button("Save")
# After
st.button(_("Save"))
The string passed to _() must be a plain string literal — it serves as
both the look-up key and the English fallback.
Variable interpolation — use .format(), not f-strings
_() receives a key at import time, so f-strings evaluated before look-up
would produce a key that is unique per call and can never match a translation.
# WRONG — the f-string is evaluated first, key never matches
st.error(_(f"File {filename} not found"))
# CORRECT — translate the template, then format
st.error(_("File {filename} not found").format(filename=filename))
Multiline strings
st.info(_(
"This tool allows you to select areas in the tomogram for\n"
"focused analysis.\n"
"Use at most 12 intervals."
))
The key in the .po file is the concatenated string exactly as Python sees it.
Loop variable clash
If a loop uses _ as the loop variable in a file that already imports _ as
the translation function, rename the loop variable:
# WRONG — shadows the translation function
for _ in items:
...
# CORRECT
for _item in items:
...
5. Extracting new strings to the template (.pot)
The file locale/messages.pot is the master template. It contains all
msgid entries used in the source code, with empty msgstr "" values.
Automated extraction with make (recommended)
With GNU gettext tools installed (xgettext, msgmerge), run from the
repository root:
make pot # xgettext over all src/**/*.py → src/tomotree/i18n/locale/messages.pot
make po # msgmerge messages.pot into every locale/<lang>/LC_MESSAGES/messages.po
# ... translate the new/empty msgstr entries in the .po files ...
make json # python src/tomotree/i18n/po_to_json.py
make pot runs:
find src/ -name "*.py" > /tmp/py_files.txt
xgettext --files-from=/tmp/py_files.txt --language=Python \
--keyword=_ --keyword=N_ \
--output=src/tomotree/i18n/locale/messages.pot --from-code=UTF-8
make po runs msgmerge --update <lang>/LC_MESSAGES/messages.po messages.pot
for each language (cs de es fr it uk zh); existing translations are kept.
Note:
gettexttools are not a Python dependency and are not required for running the app — only for extracting strings.
Manual extraction (fallback)
Without gettext tools you can add the entries by hand:
Open
locale/messages.pot.Append at the end:
msgid "Your new string" msgstr ""
Repeat for each
locale/<lang>/LC_MESSAGES/messages.po, but fill in the translatedmsgstr.
6. Updating existing language files (.po)
.po file structure
# Language name translation for TomoTree
# …
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Language: de\n"
"Language-Team: German\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
msgid "Save"
msgstr "Speichern"
msgid "File uploaded successfully: {filename}"
msgstr "Datei erfolgreich hochgeladen: {filename}"
Rules:
msgidmust exactly match the string passed to_()in the Python source, including whitespace, newlines (\n), and emoji.msgstris the translated string. An emptymsgstr ""means the English fallback will be used at runtime.Lines starting with
#are comments and are ignored by the compiler.Section comments like
# --- admin.py ---are conventional grouping markers; they have no functional effect.
Adding a new string to an existing language
Find (or create) the appropriate section in
locale/<lang>/LC_MESSAGES/messages.po.Add:
msgid "Exact English source string" msgstr "Translation in target language"
Run the compiler (see §8).
7. Adding a brand-new language
Follow these steps to add, for example, Polish (pl).
Step 1 — Create the directory & PO file
mkdir -p src/tomotree/i18n/locale/pl/LC_MESSAGES
Copy the template and fill in translations:
cp src/tomotree/i18n/locale/messages.pot \
src/tomotree/i18n/locale/pl/LC_MESSAGES/messages.po
Edit the file header:
# Polish translation for TomoTree
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
"Language: pl\n"
"Language-Team: Polish\n"
"Plural-Forms: nplurals=3; plural=(n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);\n"
Then translate all msgstr "" entries.
Plural-Forms reference for common languages:
Language
Plural-Forms
English, German, Spanish, French
nplurals=2; plural=(n != 1);Czech, Slovak
nplurals=3; plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2;Ukrainian, Russian
`nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10
Polish
`nplurals=3; plural=(n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10
Step 2 — Register the language in __init__.py
Edit src/tomotree/i18n/__init__.py and add to both dicts:
def get_available_languages() -> dict:
return {
"en": "English",
"cs": "Česky (Czech)",
# … existing entries …
"pl": "Polski (Polish)", # ← add here
}
_LANGUAGE_FLAGS = {
"en": "🇬🇧",
# … existing entries …
"pl": "🇵🇱", # ← add here
}
Step 3 — Register the language in po_to_json.py
Edit src/tomotree/i18n/po_to_json.py and add the language code to the list:
languages = ["cs", "it", "es", "de", "fr", "uk", "zh", "pl"] # ← add "pl"
Also add a msgmerge line for the new language to the po target in the
Makefile, so make po keeps it in sync.
Step 4 — Generate the JSON file
make json # or: python src/tomotree/i18n/po_to_json.py
Expected output (the count depends on the current template):
✅ pl: messages.po → messages.json (534 strings)
Step 5 — Verify
python -c "
from tomotree.i18n import _load_json_translations
t = _load_json_translations('pl')
print(len(t), 'strings loaded')
print('Save =>', t.get('Save', 'MISSING'))
"
8. Compiling translations (.po → .json)
After editing any .po file you must regenerate the corresponding
.json file. Run from the repository root:
make json # or: python src/tomotree/i18n/po_to_json.py
This processes every language listed in the languages list inside that script
and writes locale/<lang>/messages.json.
The script output reports how many strings were converted per language and whether any errors occurred:
🔄 Converting .po files to JSON format...
✅ cs: messages.po → messages.json (534 strings)
✅ it: messages.po → messages.json (455 strings)
...
✅ zh: messages.po → messages.json (455 strings)
📊 Summary: 7 converted, 0 errors
✅ All .po files converted successfully!
Important: The
.jsonfiles are committed to version control alongside the.pofiles so that the app can work without runningpo_to_json.pyat deploy time (the Docker image build regenerates them anyway). Always commit both files together.
9. Currently supported languages
Code |
Language |
Flag |
Plural-Forms |
|---|---|---|---|
|
English |
🇬🇧 |
fallback (no file) |
|
Czech |
🇨🇿 |
3 forms |
|
Italian |
🇮🇹 |
2 forms |
|
Spanish |
🇪🇸 |
2 forms |
|
German |
🇩🇪 |
2 forms |
|
French |
🇫🇷 |
2 forms |
|
Ukrainian |
🇺🇦 |
3 forms |
|
Chinese (Simplified) |
🇨🇳 |
1 form |
The number of strings grows with the UI. Czech is kept in sync with
messages.pot; the other languages may lag behind — run make po and translate
the new entries (untranslated strings fall back to English).
10. Common pitfalls
Problem |
Cause |
Fix |
|---|---|---|
Translation not shown |
|
Run |
|
Whitespace, newline, or emoji difference between |
Copy the exact string from source; use |
f-string not translating |
|
Use |
|
|
Rename loop variable to |
New language not appearing |
Language not added to |
Edit both dicts in |
New language not compiled |
Language code not in |
Add code to the list and re-run |
|
Header |
Check that the first |