See www.zabbix.com for the official Zabbix site.

Docs/Synchronising translations

From Zabbix.org
Jump to: navigation, search

Updating Zabbix translations

Zabbix translations are managed by gettext and Pootle and the resulting po files are stored in SVN. During a translation lifetime, there are two main synchronisation operations :

  • syncing code changes into Pootle
  • syncing translation changes from Pootle to po files in SVN

Code changes involve:

  • added translatable strings
  • removed translatable strings
  • changed string location - that is, for each string, gettext stores files and lines where the string appears. Even if no strings change, some could be moved a line up or down. Translation updates should be synced into Pootle anyway - Pootle and other po tools display file and line information to the translator to aid in understanding the translatable string better.

Translation changes in Pootle are simpler - they involve changed and added translations.

Syncing changes involves syncing Pootle changes into code first, and code changes, if any, back into the code. Functional pre-requisites to perform this are :

  • shell access to zabbix.org (as it hosts Pootle)
  • commit access to SVN
  • a separate SVN checkout to perform the sync

Syncing process

The following are simple suggested steps to perform translation syncing. The three main steps are :

  • check and commit translation changes in the code
  • sync translation changes from Pootle to code
  • sync code changes to Pootle

Note that it is highly suggested to follow this exact process, otherwise translated strings might be lost (overwritten by old strings or empty strings).

All of these operations must be performed for all translated branch one by one. It is suggested to start with the oldest translated branch like 2.0 and proceed with all other branches every time syncing is performed.

Checking and committing translation changes in the code

To work with translation, a separate SVN checkout (usually on your workstation) is required. Following commands do that in an optimal way:

svn co svn://svn.zabbix.com --depth empty svn.zabbix.com
cd svn.zabbix.com
svn up branches --depth empty
svn up trunk branches/{2.2,2.4,3.0}

As code is changed, translation strings can be added, removed and moved around (changing line information, for example). To check translatable string changes, in the separate SVN checkout one may change to the frontends/php/locale directory and run :

svn revert -R . # we might have uncommitted changes from previous syncing attempts. Be careful not to run this in any directory with useful uncommitted changes
./update_po.sh  # this will update the po files to match the latest code changes
lang=pt_BR; svn diff "$lang" | grep -v "^[-+]#: \|^[-+]\"POT-Creation-Date\|[-+]\{3\} $lang/" | grep "^[-+]"
# The previous line will display any changed translation strings - added or removed. If there are no translation changes, the output will be empty.
# pt_BR is used as an example language, but normally the output should be the same for all languages.
Translatable string changes found

If translatable string changes are found, more work is required.
Note that for changed strings in a stable branch it must be reviewed that this change was really critical and unavoidable. If the change is not critical, syncing process on this branch must stop and the change must be reverted to avoid constantly changing stable branches and breaking translations.

For any removed and new strings check that they are listed in "changed strings" in an issue. All changes must be listed in issues and the listed strings there must match the actually performed changes exactly.

If there is any difference (different string, string not listed or an extra string is spotted), the subissue must be reopened noting the difference and another developer must verify the subissue, always. Syncing process for this branch must be stopped.
Failing to stop here has a chance of putting a curse on the person doing so and their relatives.
If all the string changes perfectly match suggested changes in translatable string subissues, any changed and new strings must be checked for spelling and language issues.

When done, commit the new po files, referencing issue ZBX-1357 - possibly like this :

svn commit -m "A.F....... [ZBX-1357] regenerate po files"
No translatable string changes found

If no translatable string changes have been found, run svn status. If any po files are listed as changed, most likely it was line changes for the translatable strings. Check a couple files with svn diff to be sure.

When done, commit the new po files, referencing issue ZBX-1357 - possibly like this :

svn commit -m "A.F....... [ZBX-1357] regenerate po files"
No changes found

If there were no changes to po files at all, great - we are done with this phase.

Note: Note that this process must be performed before each release, to ensure that the release is correct. Changes must be committed even if only the line numbers have changed. Otherwise users of external tools will have incorrect information when working with the release.

Checking and committing translation changes from Pootle

With the po files in SVN up to date with the actual code, one should check whether translators have spent their time and made some updates in Pootle. On zabbix.org, there's a convenience script to aid with setting up Pootle environment and running required Pootle commands. It also refreshes po files in /srv/www/htdocs/pootle/po/Zabbix-{version} from Pootle database. It has a not-too-userfriendly name of synct1.sh . Possible invocation examples :

ver=2.0; cd /srv/www/htdocs/pootle/po/Zabbix-$ver; /data/scripts/pootle/synct1.sh $ver; svn status
ver=trunk; cd /srv/www/htdocs/pootle/po/Zabbix-$ver; /data/scripts/pootle/synct1.sh $ver; svn status

Note that the above are examples for different branches - one branch should be processed at a time.

If there's no output and the po files for this branch were regenerated in the previous section, proceed to "Syncing code changes into Pootle". If there's any output, the listed locales/languages have changed and must be synced to SVN. Consider using 'svn diff' on some of the changed files and checking for obvious problems.
Back in the SVN checkout subfolder frontends/php/locale, sync the raw po files from Pootle, possibly like this :

rsync -av --exclude='*/.svn*' user@pootle_host:/srv/www/htdocs/pootle/po/Zabbix-2.0/* .
rsync -av --exclude='*/.svn*' user@pootle_host:/srv/www/htdocs/pootle/po/Zabbix-trunk/* .

Now run ./update_po.sh again.
Note - this is important, as Pootle version is likely out of date. Failing to run the update script would commit old version in SVN, overwriting the regeneration step before.
As part of ./update_po.sh , po file validity is checked. If errors are displayed for any language, it must be excluded from further syncing and zabbix-translators mailing list should be informed about the problem. If there are errors in a few strings and the errors are not trivially fixable, the broken strings may be removed to allow resuming syncing. The translator list should be notified in such cases as well.
For the changed languages that had no errors displayed, compare the language list to the latest ChangeLog file entry about translation changes. Add any missing languages or add a new ChangeLog entry if one is not there for the next release.

Remember to skip translations with errors for the next step and inform zabbix-translators about the problems.
Now, commit the changes in one go for the changed translations and the ChangeLog with something like this :

svn commit ChangeLog frontends/php/locale/{ja,it,sk} -m "A.F....... [ZBX-1357] updated Japanese, Italian and Slovak translations; thanks to Zabbix translators"

Note that if a language was already mentioned in the ChangeLog, it should not go in this commit message, but should be included in the actual locale reference (like 'ja' above).
Note that one must primarily rely on the changed translation list from svn status command on zabbix.org - svn status on the checkout could list all/more languages simply because they most likely only have a different timestamp, caused by not performing these steps quickly enough.

If this is not trunk, do not forget to make appropriate ChangeLog updates to ChangeLog in all more recent branches. In this case svn log message can be like this:

.D........ [ZBX-1357] update changelog entry

With the changes committed, update the "whatsnew" page in the manual for this version, "Updated translations" section.

Note that po files are never to be accepted directly - they should be uploaded in Pootle by the translator instead.
Note that the synct1.sh script makes sure all of the changes in Pootle are synced to the po files on disk - failing to run it and relying on svn status only will often give incorrect information.

Syncing code changes into Pootle

With the changes committed to SVN, return to zabbix.org and sync the SVN changes into Pootle. Again, there's a convenience script and a few examples of the potential invocation are :

ver=2.0; cd /srv/www/htdocs/pootle/po/Zabbix-$ver; /data/scripts/pootle/synct2.sh $ver
ver=trunk; cd /srv/www/htdocs/pootle/po/Zabbix-$ver; /data/scripts/pootle/synct2.sh $ver

Note that this command takes a long time to complete.

Now proceed to the next branch.

Adding new language

On a workstation where svn checkout is created

To add a new language, execute this command, passing new language 2-character code as a parameter:

cd frontends/php/locales
add_new_language.sh ka

NOTE: this command will also add svn:ignore for frontend.mo file in parent LC_MESSAGES folder, which is required.

NOTE: on different distros the list of available locales can be seen using different commands. Installed locales can be seen by localectl list-locales. In Debian to see all possible and generate required locales a command dpkg-reconfigure locales exists.

Open and, if required, edit generated frontend.po file - make sure that:

and it corresponds with https://www.zabbix.org/pootle/admin/languages.html. Also useful page http://www.gnu.org/software/gettext/manual/gettext.html

  • UTF-8 is set in this line "Content-Type: text/plain; charset=UTF-8\n"
  • Set correctly "Project-Id-Version: Zabbix 2.5\n"

NOTE: This should be revised - it seems like tools used don't regenerate plural forms once they have been picked up incorrectly, so maybe header should be created before generating po file itself. I.e. if plural formula is incorrect in the generated file - the file is not suitable to be used. In this case it's better to generate po file for another language (which has identical plural formula) and then fix generated file headers (language name etc) manually.

Add a new line to include/locales.inc.php The php source file contains some links on how to find out correct language code in case Zabbix frontend is hosted on a Windows server. Initially the new language should not be displayed to frontend users.

To commit, new po file should added to checkout:

svn add frontend.po

New ChangeLog entry is not required for new language, as actually nothing visible is changed for frontend users.

Sync with SVN with message like:

A.F....... [ZBX-1357] add initial <language> translation

On Pootle host

To update single version for single language, this command may be used:

ver=trunk; lang=ka; cd /srv/www/htdocs/pootle/po/Zabbix-$ver; /data/scripts/pootle/synct3-single-lang.sh $ver $lang

On https://www.zabbix.org/pootle/projects/Zabbix-trunk/admin.html add the new language. Visit Overview page to make sure new strings have appeared (first time page loading may take some time).

Add permissions for required users of new language: On a <lang>/Zabbix-trunk/admin_permissions.html Pootle page select a user, mark all options and, keeping pressed Ctrl, unselect 1st and 3rd options in a permissions list (Can administrate ..., Can commit ...), Save changes. Done.

Perform full synchronisation, as new translatable string (language name) has been added.

Enabling/disabling existing language

As stated on translators main page, if a translation drops below 75% completion, should be hidden from the language selection. A translation is added back to the dropdown when it reaches 100% completion. Changes are performed in include/locales.inc.php file

ChangeLog entry should be added too.

Sync with SVN with message like:

A.F....... [ZBX-1357] enabled Czech translation to be displayed by default

Tricks

Merge two po files

There is a way to merge new po file with the existing one. It can be used in for different cases. For example when some versions are fully translated but an intermediate version is not and translator wants to complete the intermediate version. An example command:

msgcat -o merged.po --no-wrap --sort-output --no-location --use-first trunk.po 20.po

First .po file used in the command line will be used in first order, so if the same translated string appears in both .po files but translated differently, a string from first listed file will be taken. So, in the example, if trunk.po file contains more recent/correct translations, it should be listed first.

Check the merged.po file header and contents, fix any issues found (for example "Project-Id-Version" header worth to be adjusted for consistency). If all is well, move the merged file in place and regenerate any comments from code, string location information etc:

mv merged.po frontend.po
./update_po.sh

Printed results of the regeneration may be used for a conclusion how the .po files merging helped to increase translation level for the intermediate version.

Translating with standalone tools

Standalone tools may be used for offline translation. You need to download a po file from Pootle and after editing upload it back to Pootle.

  • Virtaal is a standalone translation tool
  • Poedit is another standalone translation tool.
  • KBabel or Lokalize are KDE based standalone tools (Lokalize obsoletes KBabel)