How to localize MUSHclient messages into other languages

Posted by Nick Gammon on Tue 12 Jun 2007 03:47 AM — 34 posts, 211,158 views.

Australia Forum Administrator #0
Version 4.09 introduces a preliminary attempt at Internationalization (i18n).

I encourage anyone that wants to see a localized version for their language to try to follow the steps described below, and make a version that displays some messages (at least) in their own language.

This version is by no means complete - it was done fairly quickly with a view to being a proof of concept. It is possible that some things were inadvertently changed for the worse. Perhaps some existing messages will not be displayed correctly. If that happens, please advise the exact thing you were doing, and if possible compare to an earlier version to find what the message used to say.

Things NOT changed

This version does not have any changes to the "resource" part of the executable, in particular menus and dialog boxes will still be in English, and there is no provision for changing them. That should be possible to do by editing the resource files (which are publicly available) and making new versions marked as being appropriate for a new locale. These (when available) can be merged into the existing executable, in future releases.

[Edit] See below - version 4.10 now allows this.



How to localize

  1. You should find - as a subdirectory to where MUSHclient is installed - a directory called "locale". In that will be a file "Localize_template.lua".
  2. Copy (or rename) that file to be xx.lua, where xx is your locale identifier. (eg. EN for English, ES for Spanish, DE for German, FR for French).
  3. Check that MUSHclient has correctly deduced your locale by starting it up, and going to File -> Global Preferences -> General. Near the bottom corner is a new field "Locale code". Make sure that this agrees with the code you have chosen for the localization file (Eg. EN, ES, DE, FR). If not, change it. :)
  4. Edit the localization file with a text editor that supports UTF-8 encoding. For example, Crimson Editor: http://www.crimsoneditor.com/

    If you are using Crimson Editor make sure that the Document Encoding Type is set to UTF-8 Encoding (w/o BOM). See the Document menu -> Encoding Type.
  5. Review the English messages, and make translations where you feel is appropriate.

    For example, right near the start of that file are these lines:

    
    -- DDV_validation.cpp:41
      ["This field may not be blank"] =
        "",
    


    You can put in a translation by changing the empty string to show the translated version, like this (I did this with Google translate, so bear with me if it came out as nonsense):

    
    -- DDV_validation.cpp:41
      ["This field may not be blank"] =
        "Dieses fangen kann möglicherweise nicht leer sein auf.",
    

  6. Save the changed xx.lua file.
  7. Close MUSHclient if necessary (completely, not just the current world).
  8. Restart MUSHclient. At this stage it should notice the xx.lua file and process it. If there are syntax errors in it you will get a message now.
  9. If you made the suggested change above, you can confirm it worked by going to the Connection menu -> Quick Connect. Hit <enter> without entering a TCP/IP address, which will trigger off the message above ("This field may not be blank").
  10. You should now see the translated message instead.
  11. You can now re-edit the xx.lua file and make further translations. Remember to save it, and restart MUSHclient, in order to test them.




Different sections in the localization file

The localization file is currently split into four different sections. Each consists of a Lua table, the key of each item of which is the original message (in English) inside MUSHclient.

The four sections are:

  1. messages - these are "static" messages (error or advisory messages). They are static in the sense that they don't change at runtime, and thus can be translated as a unit. For example, the message "This field may not be blank" will always be exactly that.

    Thus, the translation in the "messages" section is simply a string, which is the translated message.
  2. formatted - these vary at runtime. For example, the message "Line number must be in range 1 to %i". In this case the upper limit varies, and thus the message cannot be translated into a static string.

    For formatted message there will be a "replacement" sequence indicated by %x where x is a formatting character. Consult the string.format documentation inside MUSHclient for possible replacement sequences. For example, %s is a string, %i is an integer, and %f is a floating-point number.

    Because some work may be needed to get a good translation, each message in this section calls a function, and the responsibility of that function is to do the translation, incorporating the variable fields.

    Let us assume that we are translating into Spanish for the moment, and that a good translation would be "La línea número debe estar en la gama 1 a %i."

    First, we locate the templated message:

    
    -- GoToLineDlg.cpp:45
      ["Line number must be in range 1 to %i"] =
        function (a)
         
          return ""
        end,  -- function
    


    The xx.lua file already has a function definition, and it has worked out that we have one variable, which it has named 'a'.

    So, to translate it into Spanish we could do this:

    
    -- GoToLineDlg.cpp:45
      ["Line number must be in range 1 to %i"] =
        function (a)
         
          return string.format (
            "La línea número debe estar en la gama 1 a %i.", a)
            
        end,  -- function
    


    We use string.format, which is a Lua function that lets us imbed things into a string, and supply the variable (a) as an argument.

    Having made this change, and restarted MUSHclient, we can go into the inbuilt notepad, press Ctrl+G to "go to line", and enter a line out of range (eg. 1000). You should then see the translated message appear, with the correct range incorporated into the message.

    Some messages may require more work, for example something like "loaded %i alias%s". Here we have a number (eg. 10), and if the number is not 1, then the second variable (%s) would have "es" in it, so the message would read "loaded 10 aliases".

    To translate that into another language, you would have to look at the variable which has the number in it, and take appropriate steps to "pluralize" the noun.

    If one of these translation functions has a run-time error (for example, concatenating a string with nil), then a dialog box will appear at runtime, at the moment that the message would have appeared, describing the nature of the error. Then, since the function has failed, the original message, in English, will be displayed.
  3. times - these are for formatted date/time strings. You may need to look at the source code to see the context . For example:

    
    -- doc.cpp:6767
      ["--- Connected on %A, %B %d, %Y, %#I:%M %p ---"] =
        "",
    


    For the "times" messages the fields starting with "%" are date/time fields. See the documention for os.date for an explanation.

    You may want to translate any English words (eg. "Connected on") plus revamp the order in which the date/time fields appear.
  4. headings - these are generally one-word fields which are used in the various "list" dialogs (like the list of plugins). By changing the translations here you can change the column headings. For example you might change "Author" to "Autore" (Italian).

    Thus, a changed entry might look like this:

    
    -- PluginsDlg.cpp:105
      ["Author"] =
        "Autore",
    




Notes

  • Any entries you don't change will continue to appear in English. Thus, if you don't mind a bit of a mixture of English and your chosen language, only translate the messages that appear often enough that you wish they were translated.
  • Since the entire localization file is a Lua source file, feel free to use comments, make special functions to handle things that are done often (like making words plural), and generally organizing it the way you want to.
  • You have access to the standard Lua libraries, plus MUSHclient extension libraries "rex", "bit", "utils", and "bc".
  • You can use the information about the source file, and line number, to read the MUSHclient source, and find out more about the context in which a particular message appears.
  • Some messages are displayed in the output window. If you are using Unicode characters then make sure that you have UTF-8 enabled in the world configuration "Output" section, and that you have chosen a font that displays Unicode characters.
  • Feel free to post messages announcing your intention to translate into a certain language. You may find other people interested in collaborating with you - for example, to each do a part of the file.
  • Please do not spend weeks of work translating just yet - the format of the file may be fine-tuned yet, depending on comments we receive from the initial attempts. Also, the file is very likely to be expanded with new entries.
  • Do not change the "key" text - the stuff inside the square brackets. If you do so, it will no longer match, and become ineffectual.
  • Any messages that are not in the table at all, or return an empty string, will default to the original English message being displayed.
  • There are probably parts of MUSHclient that have not yet been converted to use the translation process. Please advise any such messages that you think should appear in the localization file.

Amended on Wed 13 Jun 2007 02:18 AM by Nick Gammon
Australia Forum Administrator #1
If you want to experiment with localizing, you can download version 4.09 (1.91 Mb) from here:

http://www.gammon.com.au/files/mushclient/mushclient409.exe

This should be regarded as an experimental version - don't bother downloading it unless you are going to play with localizing it, and don't mind if things go wrong.

The sumcheck for this file is: 0c4b0c6c2828ccb3e2a0dff6c1bf9eb0
Amended on Tue 12 Jun 2007 04:37 AM by Nick Gammon
China #2
Maybe the message in the resource part is more important than the other ones for us to use the MushClient.I am eager for the version which "resource" part of the executable can be localized in an easier way. But I know I must take pains before the weeks of hard work on that.

Thanks Nick for his effort in the past week.
Australia Forum Administrator #3
OK, I see, but I thought from the screenshot that you posted that the resource part was basically solved.

I'll do some more research into how that might be achieved along similar lines to the text messages, however resources are compiled into some sort of binary form by the resource compiler.

I can imagine it won't be too easy to simply automate it by reading strings from a file. In some of those dialogs the various check boxes are squeezed fairly tightly together, and if the wording became longer it might be chopped off.
China #4
I have asked the man who localized the resource part about the localizing method . The answer is editing the binary code of mushclient.exe by some tools manually. But since the frequence of the version updating is so high that he can not localize the up to date version in time, the latest localized version is 4.01 now.

Maybe a binary editing tools is needed ,by which we can search the text to localize in the mushclient.exe , get its localized text from a dictionary like text file and exchange them automatically. It seems that the key to solve the problem is the algorithm of compiling the text message into binary form.
Amended on Tue 12 Jun 2007 10:34 PM by Hairui
Australia Forum Administrator #5
I am investigating separating out the resources into a separate DLL.

If successful, then the program can load the appropriate DLL at runtime, which will give the localized menus, dialogs etc.

This DLL would not need to change for mere code changes in the client (like bugfixes, or changes to scripting). For more major changes (like a new dialog box), then hopefully you could copy the new dialog from the English version, customise it, and paste into the localized version.
Australia Forum Administrator #6
Separate resource file

MUSHclient version 4.10 now separates out its resources into a separate DLL. This allows you to localize the resources separately from the program code.

Effectively, this would let you have a different GUI interface for any language you choose, with the core functionality staying the same.

This is intended to work in conjunction with the string localizing described above.

The resources are things like menus, dialog boxes, and some strings.

What version 4.10 (onwards) does is look in the locale subdirectory (as described above for the string localizing), and expects to find a file xx.dll. This is the resource file for the current locale.

For example, if the locale is English, you should have in the locale directory:


en.lua - string localization
en.dll - resource localization


MUSHclient cannot start if it does not find the localized dll, as all the menus, dialogs etc. are in it.

However, if the locale is other than English, it first tries the requested locale (Eg. DE.dll) and if that fails, retries looking for EN.dll. That gives a fallback position, in case you accidentally change the locale to something other than EN, but have not installed a different DLL.

Version 4.10 is available (1.90 Mb) from:

http://www.gammon.com.au/files/mushclient/mushclient410.exe

Its sumcheck is: 91be3a11ce1df9aa2e45f86528cb6cb9

This download includes the en.dll for the English localization.




How to localize the resources

I believe there are resource editors around that let you edit DLLs and simply change the resources. If you can find one, simply make a copy of en.dll (eg. as de.dll). Then edit the copy, making such changes as you see fit.

Alternatively, you can download the source files used to build the resource dll. These are available (269 Kb) from:

http://www.gammon.com.au/files/mushclient/resources/mushclient_resources_4.10.zip

The sumcheck for this file is: 78851ba0b216161e1042a9dc684be89e

Inside that file is a Visual Studio project, which if you build it, should generate the resources file for version 4.10.

[EDIT] Now available from:

https://github.com/nickgammon/mushclient_resources

You can then simply edit the resources using Visual Studio (or any other tool that works), and rebuild with the amended resources. You could try using Visual Studio Lite which is a free download from Microsoft.




Do I have to do that for every new release?

No, hopefully not. The resources part tends to change slowly, as they are basically dialogs and menus. An amended resource file should continue to work with future versions, unless such version incorporates major changes, like amendments to dialog boxes, or new ones.

If you work with the Visual Studio project mentioned above, the resources are in text form, and you could do a "diff" to compare a new release with an earlier one. That would quickly show which ones have changed, and thus only the changed ones would need to be amended (relocalized).




I am interested in getting any feedback about how well this works. It is possible that something important got omitted in the process of splitting MUSHclient into two parts - code and resources, so this should be regarded as an experimental version.

If all goes well, then people are welcome to submit localized versions of the resource dll, so that they can be shared with other MUSHclient users.
Amended on Tue 16 Aug 2011 08:40 PM by Nick Gammon
USA #7
What kind of security is there for user-submitted resource DLLs? More specifically: can a DLL that is supposed to only contain localized resources also contain arbitrary code? I trust code that comes from you and some users but not necessarily precompiled DLLs submitted by users I don't know. How easy is it for users to submit localized resource DLLs that the end-user can compile themselves after checking source code?
Australia Forum Administrator #8
The DLL is loaded using LoadLibrary and then set as the application resource handle.

I'm not sure whether any arbitrary code could be activated in any way, although probably the LibMain function which I presume is called as part of the loading process could.

For the end-user to compile them they would need a compiler - not every MUD player would have one.

If people go down the route of downloading the resource script and amending that, they could resubmit that to me for compilation and inclusion on this site.

However the version submitted by an earlier poster, who submitted a screen shot of a customized MUSHclient executable, could presumably have added malicious code to it, if the person doing it wanted to. So those particular users are not any worse off.

There is a similar issue with the lua string localization. So far I haven't attempted to disable things like the io library in Lua, for that particular file. Perhaps I should.

You could argue that localizing strings doesn't require operating system calls, or file IO.
USA #9
Ah, ok. For DLLs it looks like the main issue is the LibMain function. Maybe a prerequisite of a new resource DLL being published on this site could be a check-over of the source by either yourself or designated community members? I would feel a lot more comfortable about localization DLLs if I had confidence that they'd been checked over by somebody I trust.

As for the Lua localization, I think you are right, sandboxing the file to remove e.g. os and io would be a good idea.

This localization effort is pretty cool. :-) I would be happy to contribute to a French localization if there is demand for one. (To be honest, though, I don't know any MUD terms in French... e.g. "MUD" itself! (DMU? Donjon a Multiples Utilisateurs?))
Australia Forum Administrator #10
Yes I agree I wouldn't want to publish a DLL on this site, without viewing its source. What the general MUD community does is its own affair. Perhaps a German MUD might produce its own localized file, so if you trust the MUD admins, you trust the file.

As for the French version, even if you were to try localizing a couple of dialog boxes, and a couple of error messages, just to confirm I have the general technique right, that would be helpful.
Australia Forum Administrator #11
New version released - 4.11

As expected, there were some teething problems with the process of separating the resources from the code. In particular, a couple of places where the resource file was manually interrogated for text, such as the license agreement, and the program credits, stopped working because it was looking in the wrong place.

Also, I have tightened up the Lua environment for the localization processing. I can't see any need for localization code to do things like delete or write files, for example.

I also noticed that with all the resources moved into the resource file, the program file didn't even have a nice icon any more. So, this has been put back.

The new version is now available (1.91 Mb):

http://www.gammon.com.au/files/mushclient/mushclient411.exe

Sumcheck for the MUSHclient installer is: 8c978231db5d7f89da953431bbb0cf05

The resource localization file (146 Kb) is here:

http://www.gammon.com.au/files/mushclient/resources/mushclient_resources_4.11.zip

Sumcheck for the localization file is: 43cc0de5f0fe094f4f156e6b56cd43e3

I don't think the resources have changed much. This file is smaller than the previous one because it no longer has a copy of en.dll in it. This ships with MUSHclient anyway.

See the release notes for a full list of changes:

http://www.gammon.com.au/scripts/showrelnote.php?version=4.11&productid=0

The source for version 4.11 is now available (1.90 Mb):

http://www.gammon.com.au/files/mushclient/src/mushclient_4.11_src.zip

Sumcheck for the source is: eea5868afdcd0d196637193ad2c4a664
Amended on Wed 13 Jun 2007 06:12 AM by Nick Gammon
Australia Forum Administrator #12
Quote:

(To be honest, though, I don't know any MUD terms in French... e.g. "MUD" itself! (DMU? Donjon a Multiples Utilisateurs?))


From Google Translate:

Cachot multiple d'utilisateur
USA #13
Quote:
Cachot multiple d'utilisateur
Haha -- now that just doesn't make any sense. :-)

An interesting note is that "donjon" in French does not actually mean "dungeon", rather it means "keep", as in the central fortified keep of a castle. I guess that "dungeon" in English came to mean what it means due to the fact that the dungeons are, indeed, usually in the keep. Or maybe it was just confusion during ol' William the Conqueror's stay in England.


I'll give a shot at localizing the resource files. However, I don't have MSVC on this system (I run Linux). Is it ok if I send back modified the resource file?
Australia Forum Administrator #14
Sure. Just edit the mushclient.rc file and make whatever changes you want.

I have a suspicion we will yet have troubles because MUSHclient is not a Unicode application.

I am not sure what will happen if you try to incorporate characters whose code is > 127.

Perhaps the resource file can be Unicode and the rest not - I don't know.
Australia Forum Administrator #15
I should point out that even if you use English, you can edit the resource file to make changes like make the world configuration dialog boxes larger, or laid out more to your taste.
Australia Forum Administrator #16
Utility to detect changes in a new release

To help localizers, this utility below, written in Lua, can be used to compare a new distribution of MUSHclient with an already localized file.


-- stuff already localized
locale = "en" -- change to suit you

dofile (locale .. ".lua")

-- make copy
original = { 
    messages = messages, 
    formatted = formatted, 
    times = times, 
    headings = headings 
    }

messages, formatted, times, headings = nil 

-- from distribution
dofile ("Localize_template.lua")

-- make copy
distribution = { 
    messages = messages, 
    formatted = formatted, 
    times = times, 
    headings = headings 
    }

messages, formatted, times, headings = nil 

function compare_table (name)
 local count = 0
 local old = original [name]
 local new = distribution [name]

 print ("Processing table", name)
 print ""

 -- new message is in distribution, but not in already localized file
  for k, v in pairs (new) do
    if not old [k] then
      count = count + 1
      print (string.format ("  New message: %q", k))
    end -- if not there
  end -- for
  
  
  print ("Found ", count, " new messages")  
  print ""

  count = 0
  
 -- old message is in already localized file, but not in distribution
  for k, v in pairs (old) do
    if not new [k] then
      count = count + 1
      print (string.format ("  Deleted message: %q", k))
    end -- if not there
  end -- for
  
  print ("Found ", count, " deleted messages")  
  print ""
  
end -- compare_table

compare_table ("messages")
compare_table ("formatted")
compare_table ("times")
compare_table ("headings")



It is intended to be run in the "locale" directory. Here is some example output:


Processing table	messages

  New message: "You must specify a font file."
  New message: "You must specify some text to insert."
Found 	2	 new messages

Found 	0	 deleted messages

Processing table	formatted

  New message: "Chat sessions for %s"
  New message: "Saturation: %5.3f"
Found 	2	 new messages

Found 	0	 deleted messages

Processing table	times

  New message: "%A, %#d %B %Y, %#I:%M %p"
Found 	1	 new messages

Found 	0	 deleted messages

Processing table	headings

  New message: "When"
Found 	1	 new messages

  Deleted message: "Action"
Found 	1	 deleted messages



What it does is load up both files (with dofile) and compare the messages tables, to see if entries have been added or deleted.

If there is a new message in the distribution, then that will appear as "new message" as shown above. That alerts you to the need to find it (by using a 'find' in your editor), and copy and paste the message into your localized file - in the appropriate section - and then translate it.

Multi-line messages will not appear exactly the same as in the message file. In the message file line breaks will be indicated by \n, however this program will simply show them as \ followed by a line break. For example:


In en.lua:  "This is a 2-line\nmessage"

In program output: "This is a 2-line\
message"


The simple solution here (apart from amending this program) is to simply search for the contents of the first line, and check the rest matches manually.

If a message appears to be deleted, but another very similar one has been added, then what has probably happened is that a spelling or punctuation mistake has been corrected, and that you can continue to use the old translation, once the "key" of the message has been adjusted.
Amended on Sat 16 Jun 2007 05:00 AM by Nick Gammon
Australia Forum Administrator #17
There is quite a bit of discussion about gettext here:

http://www.gnu.org/software/gettext/manual/gettext.html

MUSHclient does not use gettext internally - personally I found the utility a bit obscure in places, however the comments in that document were very helpful in developing an alternative. It makes good background reading.

I did however use part of the gettext suite, namely xgettext, which scans source code for strings to be localized.

Based on the comments in the gettext manual, the MUSHclient source was changed slightly in spots, in particular places where strings were built up "piecemeal". For example:


CString s = "You need ";
s += atoi (a);
s += " to ";
s += aoi (b);
s += " piece";
if (b != 1)
 s += "s";


This sort of code does not lend itself to being translated very well, because translating the individual words will probably not result in a readable result. Thus it would be revamped as:


CString s = TFormat ("You need %i to %i piece%s", 
                    a, b, b == 1 ? "" : "s");


The formatted version at least gives the translator a whole sentence to work with, which is much more reasonable.
Australia Forum Administrator #18
How to detect which messages are used the most

Reading the gettext documentation gave me another idea. There are something like 750 messages to be localized. It would be nice to initially translate the common ones, and then work back to the less common ones.

We can do this fairly readily, because the translation file is in fact a Lua script, and we can make use of metatables to catch accesses to the translation items.

As an example, I appended the following to my en.lua file:



-- make copy of original tables
orig_messages = messages
orig_formatted = formatted
orig_times = times
orig_headings = headings

-- empty them out so __index is triggered
-- save original tables so we can look them up eventually

messages  = { _orig = orig_messages }
formatted = { _orig = orig_formatted }
times     = { _orig = orig_times }
headings  = { _orig = orig_headings }

counts = {} -- keep counts here

-- metatable for messages, titles, headings
mt_static = {
  -- called to access an entry
  __index=
    function (t, name)
      local s = rawget (t._orig, name)
      if s == nil or #s == 0 then
        counts [name] = (counts [name] or 0) + 1
      end -- not translated yet
      return s
    end;
  }
  
-- metatable for formatted messages
mt_formatted = {
  -- called to access an entry
  __index=
    function (t, name)
      local f = rawget (t._orig, name)
      -- no function? not translated then
      if f == nil then
        counts [name] = (counts [name] or 0) + 1
        return nil
      end
      assert (type (f) == "function")
    
      -- return a function, that will count if the original function
      -- returns an empty string
      return function (...)
         local s = f (...)  -- call original function
         if type (s) ~= "string" or #s == 0 then
           counts [name] = (counts [name] or 0) + 1
         end -- not translated
         return s  -- return translated value
         end -- function 
     end;
  }
  
-- apply the metatables
setmetatable (messages,   mt_static)
setmetatable (times,      mt_static)
setmetatable (headings,   mt_static)
setmetatable (formatted,  mt_formatted)

-- the user will call world.TranslateDebug to invoke this
function Debug ()
   
  -- for sorting
  local t = {}
  
  -- build into table which can be sorted
  for k, v in pairs (counts) do
    table.insert (t, k)
  end -- for

  -- clear out notepad, make heading
  utils.appendtonotepad ("translation", "Translation counts\n\n", true)

  -- sort into descending order
  table.sort (t, function (a, b)
                  return counts [a] > counts [b]
                 end)
                    
  -- display results
  for k, v in ipairs (t) do
    utils.appendtonotepad ("translation", string.format ("%4i: %q \n", counts [v], v))
  end -- for
  
end -- Debug




[EDIT] Amended 20th June 2007 - to fix problem where it didn't correctly look up the original message.

What this does is replace the translation tables by empty ones (after making a copy of them). Then by adding a metatable to the new, empty, tables, we will get a trigger to the __index function each time we attempt to locate a message to be translated. The __index function simply counts the number of times this item has been accessed, and then returns the saved, original, value.

Finally we make a Debug function that can be called by the (new) script function TranslateDebug. Now if we type into the command window:


/TranslateDebug ()


We see something like this:


Translation counts

  12: "Ready" 
   2: "Alias: %s" 
   2: "Clipboard converted for use with the Forum, %i change%s made" 
   2: "&Flip To World	Ctrl+Alt+Space" 
   2: "&Send To World	Shift+Ctrl+S" 
   2: "The connection to %s is not open. Attempt to reconnect?" 
   2: "Connecting to %s, port %d" 
   2: "--- Connected on %A, %B %d, %Y, %#I:%M %p ---" 
   1: "Trigger: %s" 
   1: "For information and assistance about MUSHclient visit our forum at:" 
   1: "Go to forum" 
   1: "Execution of line %i column %i" 
   1: "--- Disconnected on %A, %B %d, %Y, %#I:%M %p ---" 
   1: "Opening world \"%s\"" 
   1: "World: %s" 
   1: "No active world" 
   1: "Script error" 
   1: "--- Connected for %i day%s, %i hour%s, %i minute%s, %i second%s. ---" 
   1: "The \"%s\" server has closed the connection" 
   1: "Immediate execution" 
   1: "Written by Nick Gammon." 


Thus we see that in this (short) session, the above messages needed to be translated. In particular "Ready" was the most often used, thus we might translate that first.

The script shown above has enough complexity so that it doesn't count messages that are already translated. Thus, as you translate some, they will drop off the list, and the next most frequent ones can be translated.
Amended on Tue 19 Jun 2007 10:53 PM by Nick Gammon
Australia Forum Administrator #19
Locale codes

I have been searching around for a definitive list of locale codes (eg EN=English, FR=French etc.).

So far I have found this one:

http://www.loc.gov/standards/iso639-2/php/code_list.php

Anyone who is planning to localize MUSHclient as described earlier will probably want to use a code from that list.
#20
And you aren't using the default Windows locale codes (ie EN-US = 1033)?

http://www.microsoft.com/globaldev/reference/winxp/xp-lcid.mspx
Australia Forum Administrator #21
Well, I thought English was English.

I mean, if the messages comes out in 1033, I imagine someone who speaks 3081 or 4105 or 5129 will understand them.

Another example of Windows making things too complicated, I think.
Australia Forum Administrator #22
Just to elaborate on that answer. I think if someone speaks English then they will understand a message like:


"This variable name is already in the list of variables."


I don't think that message needs to be translated into:

  • English_United_States
  • English_United_Kingdom
  • English_Australian
  • English_Canadian
  • English_New_Zealand
  • English_Ireland
  • English_South_Africa
  • English_Jamaica
  • English_Caribbean
  • English_Belize
  • English_Trinidad
  • English_Zimbabwe
  • English_Philippines


Even if it was, I can't offhand think of what the different translations would be.

And, if we did start using locale codes, and the file was renamed 1033.lua (English_United_States), and someone from England started using it, the program would find the locale to be 2057 (English_United_Kingdom), and not process the file.

So, we have actually made things harder for people doing that.
USA #23
Usually things have default region codes, so if you "don't" have the correct code for 2057, it defaults to 1033, in most programs. In fact, 1033 is probably the default-default, so if you lack "any" support for other languages installed, then it would automatically drop to 1033, same way fonts go from its specific name, to its general family, to the system default, if each one in the chain is missing and can't be loaded.

Mind you, it occurs to me after posting that you might actually want to treat them as "families", so it some fool deletes 1033, you try to set it to 2057 and there is some "other" English set available, it will grab that instead, being the only "available" one to use in that family. Either way, if you wanted to allow for that granularity, instead of just jumping strait to the "default" English file, you would need to have some sort of table showing what the "closest" set of possible alternatives where, so you could load one that is available, if the one requested wasn't there.
Amended on Sun 17 Jun 2007 01:37 AM by Shadowfyr
#24
I'm in Australia and all the XP localization settings are EN-AU (except the keyboard is EN-US) and Office and the localized MS Games that I have all installed a 1033.dll for their resources.

*shrug*

You are going to need a translation table from machine locale to mushclient locale anyway and there's presumably going to be an option somewhere to change it if mushclient's first guess is wrong.
Australia Forum Administrator #25
Well, that is a Microsoft game. :)

I am calling GetLocaleInfo to get the abbreviated language name (eg. EN), based on the current locale.

However bearing in mind this may not be what the user actually wants to use (what happens if they speak multiple languages?) - then there is a configuration option to override that.

In Lua scripting, if you type:


/print (os.setlocale (""))


You see:


English_Australia.1252


Now I'm not sure where 1252 comes into it (I am in Australia too), as it isn't 1033.

From that table you suggested, this is the line for Australian English:


English_Australian / 0c09 / 3081 / 0c09:00000409


Now 1033 in decimal is 0409 in hex. But that entry (0c09:00000409) is listed as "Valid Locale ID:InputLocale combinations".

Thus 1033 (0x409) is really the input locale. I was under the impression the input locale was the keyboard locale, not the locale for displaying messages. Anyway you can see how confusing it is if you start looking at Microsoft's table.

To quote from Microsoft's documentation:


Input locales, implemented in Win9x, NT4, Windows 2000, and Windows XP are pairings of an input language with an input method (which might be a particular keyboard layout, an Input Method Editor, or speech-to-text converter, for example). Specifically, an input locale describes the language being entered, and how it is being entered.

Input locales are added on a per user basis. For each account it is possible to install multiple input locales and switch between them when entering text, allowing for the composition of multilanguage documents.

...

An Arabic user using Arabic Windows 2000 wants to type an email message in a mixture of Arabic and Russian. The user already has an Arabic input locale, and installs a second input locale for the Russian language (with an Arabic keyboard layout). When entering text, the user is able to switch between the Arabic input locale and the Russian input locale.


Thus, the input locale is really to help you enter text in different languages. However in my case I am trying to solve displaying of text in the desired language.
Amended on Sun 17 Jun 2007 09:08 PM by Nick Gammon
Australia Forum Administrator #26
The PennMUSH localization files seemed to be named using the countrycode_languagecode.po convention. See:

http://download.pennmush.org/Source/translations/1.8.3p2/

These are their file names:


da_DK.po
de_DE.po
eo.po
es_ES.po
fr_FR.po
hr_HR.po
hu_HU.po
id_ID.po
nl_NL.po
no_NO.po
pl_PL.po
pt_BR.po
pt_PT.po
ro_RO.po
ru_RU.po
sv_SE.po
zh_TW.po


I suppose there could be an argument for translating French slightly differently depending on which country you are in (or English for that matter), but in the case of a MUD client, I don't think that level of detail is needed. In a spell-checker, maybe.
Australia Forum Administrator #27
Version 4.12 has now been released, which fixes a few niggling problems with the earlier interim releases. See:

http://www.gammon.com.au/forum/?id=7967

The resource file en.dll has not changed at all from the earlier release, so if you have been localizing 4.11 resources, the same file can be used for 4.12.

http://www.gammon.com.au/files/mushclient/resources/mushclient_resources_4.11.zip

Some of the Lua code posted earlier in this thread is now distributed as part of the locale directory, to make it easier to do localization. There is also a locale_notes.txt file which explains the purpose of the various files.
#28
1252 is probably the codepage. Though I'm not completely sure why that would be specified in the locale name.
Germany #29
I have two issues with the localisation, may be they are not related to each other. I'm using MushClient 4.19 on an German WinXP.

1) Cannot load DE.dll: I've used this article and created a de.lua, but restarting MushClient does not create a DE.DLL. Also changing the paramter in Global Preferences dows not influence anything. (Personally I'm used to the english version and I don't want to change it.)

2) In a world file I have a timer, running every 30 second. when it is saved, it is second="30,00". But during loading there is an error message Line 682: Invalid character ',' in numeric attribute named 'second' (timer not loaded)
During reading he is expecting a '.', so if I change manually it to '.' or delete the, and the 0-decimals it works.
',' and '. are exchanged in German and French compared with the English localisation of it.
Australia Forum Administrator #30
As for DE.DLL it won't create it automatically. I suggest you copy EN.DLL to DE.DLL then it should work OK. You can then use a resource editor to edit DE.DLL to amend the wording of the menus if you want to.

For the timer issue, I thought that had been resolved. I will look into that.
Australia Forum Administrator #31
See:

http://www.gammon.com.au/scripts/showrelnote.php?version=4.18&productid=0

This version was supposed to fix this problem.
Australia Forum Administrator #32
Quote:

In a world file I have a timer, running every 30 second. when it is saved, it is second="30,00".


This was supposed to be have been fixed in version 4.18, which I did by temporarily changing the locale to "C" and back again. Evidently this didn't work. Version 4.20 fixes the problem by simply changing "," to "." when saving the world file. This will be consistent with the way world files are read, and allow for plugins to be shared between users who use different locales.
Germany #33
I have downloaded and tested it, and now it's working fine.

Thanks for the fast and excellent help.

StuDraKi.