Subscribe to WolfCoder's Kawaii Column        RSS Feed
***** 1 Votes

Unicode of Many Tounges

Icon 3 Comments
I actually did knew what I was getting into when I built LandTraveller with extra language support so it could display and receive Japanese, Chinese and Korean text, but that doesn't lessen the impact any! You all know what UNICODE is, and everyone thought they were made out of 16-bit character strings and sometimes extendable 8-bit UTF8 strings and that the platforms all have functions to properly handle these weird strings.


So there's many many encodings and every single one of the foundation class libraries I used seems to think there is one true encoding. This is annoying when you want to pick one format for yourself to use in your program. I had to write some additional helper functions here, configure a few things there, and I finally got the engine to work with itself nicely under Windows. Windows uses the encoding you'd expect.. 16-bit wchar_t for its -W series of functions and it has a long-established userbase among those East Asian countries I was trying to target in the first place. The program can mix text of all sorts so you can have people speaking Korean, Japanese and English all on the same multiplayer server.

So I port it to Linux which was easy because I used self-contained code and cross-platform libraries like I usually do.

But it wasn't easy because Linux goes ahead swears to me repeatedly wchar_t's one and true type has always been 32-bit... So much so that the supposed POSIX style wchar.h functions all behave differently in Linux than they do Windows. I had some amusement watching all the editable text boxes and text displays go berserk before I realized there's no way I can do a straight to Linux port without making a giant mess out of the code. Even if you use the -fshort-wchar option, it does not alter the behavior of wchar.h's functions. I also can't use two different UNICODE encodings on different platforms because these programs need to talk to each other interchangeably.

Thanks to both Linux and Windows being stubborn with their UNICODE support, it's going to be quite a ton of work if there is to be a native LandTraveller build for Linux. One option I'm already using is to just run in WINE. I'm the author of the game engine so I can just test against WINE to make sure it runs fine, you know. Another option is to try compiling with WINELIB since the application already gets past the compile process on Linux.. but it may not fix my UNICODE issues. Getting it to run on Linux was something I wanted out of principle, but in reality I would only gain 1.5% of the userbase right now. Still, I will find a fix for this at some point.

3 Comments On This Entry

Page 1 of 1

mimimintan Icon

29 September 2013 - 03:20 PM
Sounds like fun, in the masochistic sense :D

Keep on it, we're all waiting for this juicy gem you're cooking up. ;P
Best of luck to you.

shortbus Icon

30 September 2013 - 07:38 PM
cant wait for this game. really hope you release a build in the coming months

WolfCoder Icon

04 October 2013 - 08:25 AM
I have one scheduled for November 30th but I'm not sure I'll meet that deadline with something ready to be demoed.
Page 1 of 1

January 2017

151617 18 192021