LOCALIZATION

SDN++

Localizationpart of the SDN++ series

1st Edition, 07/08

Published by:Symbian Software Limited2-6 Boundary RowSouthwarkLondon SE1 8HPUKwww.symbian.com

Trademarks, copyright, disclaimer‘Symbian,’ ‘Symbian OS,’ and other associated Symbian marks are alltrademarks of Symbian Software Ltd. Symbian acknowledges the trademarkrights of all third parties referred to in this material. © Copyright SymbianSoftware Ltd 2008. All rights reserved. No part of this material may bereproduced without the express written permission of Symbian Software Ltd.Symbian Software Ltd makes no warranty or guarantee about the suitabilityor the accuracy of the information contained in this document. The informationcontained in this document is for general information purposes only andshould not be used or relied upon for any other purpose whatsoever.

Compiled by:Will Bamberg

Managing Editor:Ashlee Godwin

Reviewed by:Tim BandChris CooperKai DuanAntony EdwardsFreddie GjertsenJames HeginbottomJasdeep SawhneyRenota SchoeppJo Stichbury

An early draft of this booklet was reviewed by Chris Cooper, who passed away in early July2008. He will be sadly missed for his contribution to the Text and Internationalization team inSymbian, and to the character of the company itself.

3

Contents

INTRODUCTION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

TEXT SUPPORT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6

SCRIPT SUPPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

CONVERTING THE INPUTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

FONT SUPPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

LOCALES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

WHAT IS A LOCALE? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

LOCALE MODEL OVERVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

SUPPLIER INTERFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14

CLIENT INTERFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

SELECTED LOCALE COMPONENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16

LOCALE LIFECYCLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

CHECKLIST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23

GLOSSARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23

REFERENCES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

4

IntroductionSymbian smartphones are used in many different countries, currently selling through more than250 major network operators worldwide. There is no single type of ‘smartphone user’ and it isimportant for phone manufacturers to be able to tailor their devices for a number of differentlanguages, regions, and cultures. Besides the fundamental differences of language, scripts, andreading direction, there are a number of additional characteristics that must be varied, such asthe way names, dates, numbers, and currencies are formatted.

Symbian OS, and the application layers above it, can be readily adapted for those variations.For example, the Contacts application does not need to be changed when it ships in twophones, each built to use a different sort order for the contact names.

Localization is analogous to base porting. Symbian OS does not make assumptions about thespecific hardware it runs on, but runs unchanged on top of an adaptation layer that needs tobe reimplemented as part of the base port to a different hardware platform. Similarly, SymbianOS does not make assumptions about geography, culture, or language. It supplies a number ofadaptation mechanisms, and device creators supply the data and algorithms that vary fromone market to another.

What’s the difference between Internationalization and Localization?

Internationalization is the up-front design and implementation of software to allow it to rununchanged in various locales.

Localization is the adaptation of a software system for a specific region or language. Ittypically adds new components specific to the region in question.

It may help to think of them in terms of the development roles. The software developers insideSymbian, who design and implement the features of Symbian OS for supporting differentlocales, are internationalization engineers. Developers working inside device creationcompanies, who create phones for different regions, are localization engineers.

The terms internationalization and localization are frequently abbreviated to i18n (where 18stands for the number of letters between the i and the n), and L10n respectively. The capital Lon L10n helps to distinguish it from the lowercase i in i18n.

This booklet provides an overview of the tasks a device creator needs to consider whenlocalizing Symbian OS. It divides into two sections. The first section describes text processingissues that may arise when creating a new language variant. The second describes the supportthat the operating system provides for sets of user preferences, such as date format, that tendto vary from one language or geographic region to another, which are usually referred to aslocales.

Like all specialized technical domains, this one includes a significant amount of domain-specific terminology. Brief definitions of such terms are given where they are used, but thedefinitive reference is the Unicode glossary, at www.unicode.org/glossary.

5

Text SupportThis section considers issues of text support when creating a new language variant. Thefollowing must be guaranteed:

• Symbian OS has support for rendering and editing the script• all text input to Symbian OS is converted into the Unicode UTF-16 encoding used internally,meaning that:

- the appropriate character convertors are included to convert text entering thedevice from the outside world

- the appropriate Front-end Processors (FEPs) are included for scripts that requirespecialized input methods (FEPs are explained in more detail in the section ‘Front-end Processors,’ later in this booklet)

• suitable fonts are included in Symbian OS.

Figure 1 shows an extremely simplified view of the text processing components in Symbian OS.

As the section ‘Converting The Inputs’ describes, text enters the system either from direct userinput, or from the world outside the device (for example, in an email). Text entering the textprocessing subsystem must be in UTF-16. UTF-16 is a standard for encoding Unicode, definedin RFC 2781.

The Symbian OS text formatting code takes as input:• the UTF-16 encoded text• formatting instructions (for example, the font to use, the font size, and which effects toapply).

The Symbian OS text formatting code fetches the abstract character properties for the text(such as whether we are allowed to break a line at a given character) from the base Unicodesupport, and the physical dimensions of the text from the text rendering component. Then ituses all this information to calculate how to lay out the text.

The text rendering component maps groups of characters onto glyphs for the purposes ofmeasuring and drawing the text (a glyph is the representation in a font of a piece of text such asa character, including its dimensions and associated bitmap). The mapping between charactersand glyphs is not always straightforward, so the text rendering code needs to work out how agroup of characters resolves to a group of glyphs, and then retrieve them from the font.

6

Figure 1: Text processing subsystem.

Device creators must consider three aspects when localizing Symbian OS for a different writingsystem (script):

• basic script support: ensuring that the version of Symbian OS you are taking supports thescript in question

• converting the inputs: ensuring that text entering the text processing components has beenconverted into UTF-16 encoded Unicode

• font support: supplying the correct fonts.

Script Support

Some scripts are not supported by all releases of Symbian OS. For example, versions ofSymbian OS prior to v9.2 did not support any Indic writing systems.

The concept of ‘support’ for a script is really a kind of shorthand for supporting a number ofspecific use cases such as correct rendering and intuitive editing of the script.

7

This section gives an overview of some of the fundamental elements in supporting such usecases, but cannot pretend to be definitive. Even where one device creator has successfullyshipped devices localized to use a specific script, this is no guarantee that the support offeredby Symbian OS will be acceptable, as it is, to all device creators. Each may have subtlydifferent requirements, for example, how cursor movement in bidirectional text should work.But it should give a good approximation of what is possible with the different releases.

Unicode Character Database SupportUnicode defines a set of characters and assigns a code point and a corresponding collection ofproperties to each one. The properties include, for example, the character’s bidirectional classor line breaking properties. The complete set of code points and properties is called theUnicode Character Database (UCD) (www.unicode.org/ucd).

Symbian OS stores the UCD internally in a table. The text formatting code retrieves thecharacter properties from the table and uses this information to determine, for example, thedirectionality of the text or where line breaks are allowed.

So for the platform to support a script its internal copy of the UCD needs to contain codepoints for all the script’s characters.

Symbian OS releases from v9.1 to v9.3 contain the version of the UCD from Unicode 3.0.0, soany scripts added to the standard after 3.0.0 are not supported by those versions of theplatform. From v9.4, Symbian OS contains a few extra code points needed for the followingIndic scripts: Devanagari, Gujarati, Bengali, Gurmukhi, Tamil, and Kannada.

Most notably, Symbian OS does not yet support any code points outside the Basic MultilingualPlane: that is, code points above FFFF, which therefore need to be encoded using a ‘surrogatepair’ consisting of two 16-bit values. Scripts containing characters mapped to code pointsabove FFFF (also known as supplementary characters) are not currently supported.

Script-Specific Algorithm SupportApart from simply understanding the code points, many scripts have specific behavior whichrequires the text handling code to implement specific algorithms. Table 1 summarizes theSymbian OS releases in which support for such scripts was added:

Table 1: Support for writing systems added by Symbian OS version.

Symbian OSrelease

Script

v6.0 Japanese

v6.0 Chinese

v7.0s Arabic

v7.0s Hebrew

v8.0 Thai

v9.2 Vietnamese

v9.2 Devanagari

v9.4 Gujarati

v9.4 Kannada

8

Languages requiring bidirectional supportArabic and Hebrew are interesting because they are written right-to-left, so implementingsupport for them means supporting the Unicode Bidirectional Algorithm(www.unicode.org/reports/tr9).

Languages with complex shaping requirementsThai, Devanagari, Gujarati, and Kannada are interesting because the visual representation(shape and placement) of a given letter or diacritic depends on the syllable in which it occurs,so they require the rendering and formatting code to understand where the divisions arebetween syllables.

For Thai, which Symbian implements according to the WTT2.0 specification(www.inet.co.th/cyberclub/trin/thairef/wtt2/char-class.pdf), the rules are simple enough for it tobe practical for the font file to include different glyphs for the different variant forms of eachcharacter, and for the rendering code to select the appropriate glyph based on the syllable inwhich it appears. But for the other scripts the rules are sufficiently complex that this is nolonger very practical, and so the solution in these cases is to use OpenType fonts(partners.adobe.com/public/developer/opentype/index_spec.html) in combination with the opensource ICU Layout Engine (www.icu-project.org/userguide/layoutEngine.html) to shape theglyphs based on their context. The ICU Layout Engine was integrated into Symbian OS in v9.2.

Converting The Inputs

Internally, all text processing in Symbian OS assumes UTF-16 encoded Unicode text. So, anyinput to the text-processing subsystem must be in UTF-16 encoded Unicode. Input to the text-processing subsystem comes either from the user (for example, in the form of keystrokes), inwhich case it is converted using a Front-end Processor (FEP), or it comes from some othercomputer (for example, a mail server), in which case it is converted using a CharacterConvertor (Charconv) plug-in.

CharconvThe Character Conversion component, Charconv, is a plug-in framework for converting textbetween some other character encoding (or charset), referred to as the ‘foreign’ encoding, andthe native UTF-16 encoding. Each convertor implements conversion between a single foreigncharacter encoding and UTF-16, and is identified by UID. The UIDs are listed in the header filecharconv.h.

Symbian OS supplies:• A number of convertors built into the framework. These are convertors that the largemajority of language variants are expected to need.

• A group of additional convertors separately as ECOM plug-ins. These are convertors that areonly needed for certain variants.

Some convertors, such as those mapping Unicode, are universal, but most are intended toencode specific scripts or groups of scripts. Part of localizing the device for a new scriptinvolves ensuring that the device contains convertors for all the encodings in which the newlanguage is commonly encoded. This may mean simply ensuring that the correct Symbian-supplied plug-ins are included, but in some cases the device creator may need to writeadditional plug-ins.

Table 2 summarizes the convertors supplied by Symbian OS. A § symbol is used to indicate

9

the release in which each first appeared if it was not available in the first EKA2 release ofSymbian OS (v9.1). Each of the scripts shown is a plug-in, unless indicated by a ‡ symbol.

Table 2: Character convertors supplied by Symbian.

Name Target script

ISO 8859-6 Arabic

ISO 8859-13 Baltic

ISO 8859-14 Celtic

ISO 8859-2 Central European

GB2312 Chinese (Simplified)

GBK Chinese (Simplified)

HZ Chinese (Simplified)

Big5 Chinese (Traditional)

GB12345 Chinese (Traditional)

ISO 8859-5 Cyrillic

ISO 8859-7 Greek

ISO 8859-8 Hebrew

eucJP Japanese

ISO 2022-JP Japanese

ISO 2022-JP1 Japanese

JIS Japanese

JIS X 0201 Japanese

JIS X 0208 Japanese

JIS X 0212 Japanese

Name Target script

J5 DoCoMo Japanese encoding

for DoCoMo

Shift-JIS DoCoMoJapanese encoding

for DoCoMo

J5 KDDI (§ Symbian OS v9.3)

Japanese encoding for KDDI

Shift-JIS KDDI(§ Symbian OS v9.3)

Japanese encoding for KDDI

ISO 8859-10 Nordic

ISO 8859-4 Northern European

ISO 8859-3 South European

ISO 8859-9 Turkish

UTF-7 ‡ Universal (Unicode)

UTF-8 ‡ Universal (Unicode)

UCS-2 Universal (Unicode)

IMAP UTF-7 ‡Universal (Unicode)

for IMAP

Java UTF-8 ‡Universal (Unicode)

for Java

CP-1252 ‡Western European /

US

ISO 8859-1 ‡Western European /

US

ASCII ‡Western European /

US

SMS 7-bit ‡Western European /

US

ISO 8859-15Western European /

US

CP 850(§ Symbian OS v9.2)

Western European /US

10

For example, a device to ship in Russia would need the ISO 8859-5 Cyrillic plug-in, but thedevice creator would also need to write a Windows-1251 plug-in, as that encoding is morewidely used. Similarly, for a device to ship in Thailand the device creator would need to writean ISO 8859-11 convertor, as this is also not supplied by Symbian.

Also note that the Shift-JIS convertor needs special attention. Different operators use differentversions of the standard, and so two variants are produced: one for NTT DoCoMo and anotherfor KDDI. Both versions of the standard use the same IANA name, so the convertors use thesame UID and are not allowed to coexist in the same device. Documentation explaining how tobuild the correct Shift-JIS plug-in can be found in the Symbian OS codebase in the followinglocation: \syslibs\charconv\plugins\documentation\shift-jis_howto.doc.

We can distinguish two classes of Charconv plug-in:• Those where the foreign character set is small enough that the conversion is implementedas a lookup table: in this case the source code can be generated from specially formattedtext files.

• Those where the foreign character set is too large to implement the conversion as a lookup:then the conversion is defined by an algorithm (for example, UTF-8) and the source filemust be handwritten.

The Symbian Developer Library documentation contains detailed documentation on how toimplement a Charconv plug-in(www.symbian.com/developer/techlib/v9.3docs/doc_source/ToolsAndUtilities93). Documentationfor the tool used to generate mapping tables can be found in the Symbian OS codebase in thefollowing location: \syslibs\charconv\plugins\documentation\cnvtool.rtf.

Front-end ProcessorsFront-end Processors (FEPs) are plug-ins that convert user input into UTF-16 encoded input.Typical FEPs implement handwriting recognition systems or predictive text systems. Theirrelevance to script support is that certain scripts contain too many characters to fit on adevice’s keyboard, so other input techniques are needed. The most obvious example isChinese, in which the most common ways to implement text entry are Pinyin and handwritingrecognition.

When dealing with Arabic or Hebrew input, care should be taken to ensure the correct use ofBidirectional Ordering Control characters in the resulting input to the text processingsubsystem (www.unicode.org/reports/tr9). This is particularly important when mixing Right-to-Left and Left-to-Right text.

Font Support

The rendering code works out which sets of glyphs a set of Unicode characters maps to, andfetches the glyphs from the font in use.

In order for a device to support a specific script, certain glyphs must be supported. Symbianpublishes a font specification that documents these and the glyph codes that must be used toidentify them. It can be found in the Symbian OS codebase in the following location: \graphics\fonts\Documentation\Symbian_OS_Font_Specification.doc.

11

LocalesThis section describes the support Symbian OS provides for locales, and includes a descriptionof:

•what a locale is• how to provision a device with new locales•which locale elements need special attention• how the locale is initialized, and how changes to it propagate through the system.

Although the description here is specific to Symbian OS v9.4, it is largely valid for all versionsof Symbian OS from v9.1 onwards.

What is a Locale?

A locale is defined in Symbian OS as a set of data values that vary according to language andgeographic region, for example, the names of the days of the week, the currency symbol, andthe rules for sorting strings (collation).

The locale settings include the language ID itself, which is a TLanguage enumeration valueused to determine which localized resource file is loaded for an application when theapplication (or the application framework acting on its behalf ) callsBaflUtils::NearestLanguageFile().

Symbian OS also defines four locale ‘aspects,’ or groups of settings, and allows client code toload individual aspects in isolation from each other. The defined aspects are:

• language settings, such as the names of the days of the week• time and date settings, such as date format• region settings, such as the currency symbol• collation settings, which determine the sort order for strings containing text.

The reason for these distinctions is that it should be possible for certain subgroups of localesettings to vary independently of each other. For example, the desired language should beable to vary independently of the currently selected region.

Locale Model Overview

Symbian’s locale support implements the mapping between two interfaces, as can be seen inFigure 2:

• the client interface, which allows code on the device to retrieve and change locale settings • the supplier interface: the binary interface that must be implemented by suppliers of localeDLLs.

12

Figure 2: Locale model overview.

Localization FW

Localization DLL 1

Supplier interface

Localization DLL 2

Implement interface

Loads locale data using

ApplicationApplication

Application

Client interface

SettingsApplication

Loads locale data Retrieves locale settings

Licensee code

Symbian code

13

Supplier Interface

Locale DLLsEach locale is supplied by the device creator and is packaged as a polymorphic interface DLL.Each DLL must implement the binary interface defined in eloclu.def. This is animplementation of the static functions in the Locl class, declared in localise.h:

class Locl{

public:IMPORT_C static TLanguage Language();IMPORT_C static TBool UniCode();IMPORT_C static void LocaleData(SLocaleData *aLocale);IMPORT_C static const TLocaleText* CurrencySymbol();IMPORT_C static const TLocaleText* ShortDateFormatSpec();IMPORT_C static const TLocaleText* LongDateFormatSpec();IMPORT_C static const TLocaleText* TimeFormatSpec();IMPORT_C static const TFatUtilityFunctions* FatUtilityFunctions();IMPORT_C static const TLocaleText* const *DateSuffixTable();IMPORT_C static const TLocaleText* const *DayTable();IMPORT_C static const TLocaleText* const *DayAbbTable();IMPORT_C static const TLocaleText* const *MonthTable();IMPORT_C static const TLocaleText* const *MonthAbbTable();IMPORT_C static const TLocaleText* const *AmPmTable();IMPORT_C static const TLocaleText* const *MsgTable();IMPORT_C static const LCharSet *CharSet();IMPORT_C static const TUint8 *TypeTable();IMPORT_C static const TLocaleText* UpperTable();IMPORT_C static const TLocaleText* LowerTable();IMPORT_C static const TLocaleText* FoldTable();IMPORT_C static const TLocaleText* CollTable();};

The DLL is conventionally named ‘elocl.XYZ’ where ‘XYZ’ is taken from the TLanguageenumeration, defined in e32lang.h. Thus, the locale for UK English is named elocl.01and that for Arabic is named elocl.37.

If TLanguage does not contain a value for the locale you want to create, post a request onthe SDN++ discussion forum for Symbian APIs(developer.symbian.com/forum/forum.jspa?forumID=8), stating the locale that you need.

Symbian’s LOCE32 component contains some reference locale DLLs. The simplest way to createa locale is to copy and adapt one of these. The documentation of LOCE32 dates from SymbianOS v6, but is still quite useful, and may be found in Symbian’s codebase under\base\loce32\ongoing\doc.

The Default LocaleWhenever Symbian OS boots up, the initialiselocale executable runs. This reads aconfiguration file implemented as a central repository keyspace, and uses it to populate thesystem locale. Subsequently, initialiselocale monitors the current system locale andwhen it changes it writes the new values into the central repository so the values persistacross future re-boots.

14

Device creators that wish to make use of initialiselocale need to supply a copy of thecentral repository keyspace that contains the initial default values for the locale. The UID forthis keyspace is 0x1020E4D3, and a sample keyspace is provided along with theinitialiselocale component, which can be found in the\syslibs\bafl\initlocale\data\ directory of the Symbian OS codebase.

The keyspace contains:• four string values, one for each aspect, each of which contains the name of a locale DLLfrom which to load that aspect

• a series of customizable individual settings.

Client Interface

Retrieving locale dataClient code can retrieve locale settings using the TExtendedLocale class. To use this classlink against EUser.lib and include e32std.h. To initialize a locale object to the currentsystem settings the client needs to execute code like this:

TExtendedLocale locale;locale.LoadSystemSettings();

The client can then retrieve the values:

TPtrC currencySymbol = locale.GetCurrencySymbol();

Setting locale dataWhile many applications will want to retrieve locale settings, it is expected that only a settingsapplication provided by the UI vendor will need to change the settings. Individual settings canbe changed manually with code such as this:

TExtendedLocale locale;locale.LoadSystemSettings();locale.SetCurrencySymbol(currencySymbol);locale.SaveSystemSettings();

Setting all locale values from a locale DLLMore probably, the settings application will want to set all the settings from a locale DLL. Thefollowing code can be used to load all the settings from a given locale DLL:

TExtendedLocale locale;locale.LoadSystemSettings();locale.LoadLocale(dllName);locale.SaveSystemSettings();

Setting all locale values in a single aspect from a locale DLLThe following code can be used to load a single aspect:

TExtendedLocale locale;locale.LoadSystemSettings();locale.LoadLocaleAspect(ELocaleLanguageSettings), dllName);locale.SaveSystemSettings();

15

Note that not all locale settings belong to an aspect. FatUtilityFunctions is used onlyby the file server, and the collection of settings in SLocaleData is not part of any aspect,and can only be loaded by loading the entire locale.

Publishing the new valuesNote that without the call to SaveSystemSettings()in the examples above, the localesettings are not published to the system and remain local to this locale object. This distinctionenables an application to use a collection of locale settings which differs from the systemsettings.

Detecting changes to the system localeTo detect changes to the system locale you can use RChangeNotifier, declared ine32std.h, and look for its asynchronous Logon(TRequestStatus& aStatus) methodto complete with a status containing the EChangesLocale bit flag (defined ine32const.h).

Selected Locale Components

Collation SettingsThe supplier interface function Locl::CharSet() returns the collation settings to use forthe current locale.

Symbian OS supports the Unicode Collation Algorithm. The rules for collation defined byversion 2.1.9 of the standard are encoded as constant static data in collate.h, and theseare referenced by the default implementation of Locl::CharSet().

It is possible that a specific locale will require different sorting to that specified in thestandard. This is most likely to be where the same characters are used in different languagesand the different languages expect different sorting for them.

If you need different sorting to that specified in the standard, then you need to generate acustom collation table using the COLTAB tool and build that into your locale. This process isdocumented in Symbian’s codebase under \base\loce32\ongoing\coltab.

FAT Utility Functions

FAT Utility Functions definitionThe supplier interface function Locl::FatUtilityFunctions() returns a pointer to anobject of type TFatUtilityFunctions (declared in localise.h). An instance of thisstructure corresponds to a specific local character encoding, and is a struct containing threefunction pointers:

TConvertFromUnicodeL Converts from UTF-16 to the local character set.

TConvertToUnicodeL Converts from the local character set to UTF-16.

TIsLegalShortNameCharacter Returns true if the character is legal in the local character set in question.

16

FAT Utility Functions purposeThe purpose of this element is to enable Symbian OS to interact with FAT file systems in avariety of regions.

For example, a Japanese user will want to use Japanese names for files and directories on theirSD card, and this means the file system plug-in will need to convert file and directory namesfrom the UTF-16 encoding used internally into an encoding conventional in Japan.

This is exactly analogous to the use of Charconv explained previously: •When names originate in Symbian OS and act as input to the FAT file system, they need tobe converted from Unicode to the local encoding.

•When names are being read from the FAT device for use in Symbian OS they need to beconverted to UTF-16 from the local encoding.

So a command like MkDirL() will take a 16-bit name as an argument, and the FAT filesystem plug-in will convert this to an 8-bit legal short name before writing the directory entry:

Figure 3: MkDirL() interaction with locale settings.

Correspondingly, commands to read directory entries convert the names read from the diskinto UTF-16 encoded Unicode, using ConvertToUnicodeL().

Relationship to the localeWe have seen that the character encoding to be used for FAT file and directory names has aclose relationship with the locale in which the device is being used.

But the relationship is not straightforward: in particular, we do not necessarily want thecharacter encoding to change whenever the UI language or even the locale change, becausethis could invalidate, or at least change the names of, existing files on FAT file systems.

The solution currently in place is:•On boot, the conversion functions are NULL (which is interpreted effectively as ASCII).• The first time a locale with a non-NULL pointer is loaded, that is set as the FAT Utilityfunctions pointer, and its mapping is used subsequently.

17

• This pointer does not change until the next reboot, whether or not new locales containingdifferent pointers are loaded.

Implementing Fat Utility FunctionsSymbian OS supplies implementations of mapping functions for a variety of characterencodings in the component FATCharsetConv. However, these implementations aresupplied as standalone plug-in DLLs, so the device creator needs to extract the functionimplementations and package them as part of the relevant locale DLLs.

The process of packaging the functions inside locale DLLs is documented in detail in thedocument ‘SGL.TS0019.219 – FatCharSetConv Integration HowTo-2.doc,’which may be found in Symbian’s codebase under \base\documentation\. It isanticipated that in future releases of the OS the file server will load FATCharsetConv plug-insdirectly, and the device creator will no longer have to go through this step.

Locale Lifecycle

The locale lifecycle involves the interaction of several different system components. Note thatthe lifecycle described here is the default one provided by Symbian: several of the systemcomponents, including initialiselocale and those involved in controlling the bootsequence, may be replaced by a device creator.

BootLocale setup during boot is controlled by the EStart component and passes through threedistinct phases.

First phaseIn the first phase EStart sets the locale values to some hard coded defaults:

Figure 4: Boot, first phase: hard coded defaults.

Note that in this phase the locale properties are set directly, bypassing TExtendedLocale.After this phase EStart can mount local drives.

18

Second phaseIn the second phase, EStart initializes the HAL data, and then uses the ELanguageIndexHAL setting to load the initial locale:

Figure 5: Boot, second phase: locale loaded from HAL settings.

HalSettings uses the file HAL.DAT to initialize the HAL settings. Once they are initializedEStart retrieves the language setting and uses that to construct the name of a locale DLL toload (for instance, ‘ELOCL.37’ if the language index returned by HAL is 37, for Arabic). It thenloads the locale and sets it as the current system locale.

Now EStart can mount removable drives.

Third phaseIn the third phase EStart runs SysStart, which runs initialiselocale to read thepersisted locale settings from the central repository keyspace and publish them to the system.This is shown in Figure 6.

19

Figure 6 Boot, third phase: locale initialized from initialiselocale.

20

Finally, initialiselocale requests notification of any changes to the locale settings:

Figure 7: Boot, final phase: initialiselocale requests update notifications.

Persisting ChangesWhen a client changes any of the currently selected locale values either individually, or byloading a locale aspect, or by loading an entire locale, the change notifier is triggered:

Figure 8: Changing system settings.

21

This causes initialiselocale to persist the new locale values to the central repository:

Figure 9: Persisting locale settings.

So at the next boot, these values will be installed as the system settings.

Resetting the LocaleResetting the locale to the factory settings simply uses the central repository’s resetimplementation, so the persisted copy of keyspace 0x1020E4D3 will be replaced with theoriginal copy.

The device creator should take care to set the keyspace metadata controlling the resetbehavior, otherwise restoring the locale settings to the factory default will not functioncorrectly.

Backup and RestoreIf the persisted settings change through backup and restore, they will take effect after the nextreboot. Note that initialiselocale does not listen to restore notifications.

22

ChecklistThis section summarizes the main questions you need to ask when preparing a newlocalization of a Symbian OS-based device.

GlossaryThe most comprehensive resource available for this topic is the Unicode glossary, which can befound at www.unicode.org/glossary.

Localization Checklist �

1 Does Symbian have basic support for the writing system (script)?

2Which character encodings are commonly used to encode thelanguage?

2aDoes Symbian already supply convertors (Charconv plug-ins) for allthese encodings, or will you need to create some?

3Does the writing system impose any special input requirements,such as Pinyin or bidirectional control codes?

4 Do you have fonts that conform to Symbian’s font specification?

5 Which locales will you need to create?

5a Will any of the locales need custom collation methods?

5bWill any of the locales need conversion functions for FAT filesystems?

5cWill any of the locales need language identifiers not already definedin TLanguage?

5d Which locale will be the default?

23

ReferencesUnicode Character Databasewww.unicode.org/ucd

Unicode Bidirectional Algorithmwww.unicode.org/reports/tr9

WTT 2.0 Thai Input and Output Methodswww.inet.co.th/cyberclub/trin/thairef/wtt2/char-class.pdf

OpenTypepartners.adobe.com/public/developer/opentype/index_spec.html

ICU Layout Enginewww.icu-project.org/userguide/layoutEngine.html

How to create a Charconv plug-in (in the Symbian OS v9.3 Symbian Developer Library)www.symbian.com/developer/techlib/v9.3docs/doc_source/ToolsAndUtilities93

24

25

New from

Games on Symbian OS: A Handbook for Mobile DevelopmentThis book forms part of the TechnologySeries from Symbian Press. It describesthe key aspects of the mobile gamesmarketplace, with particular emphasis oncreating games for smartphones based onSymbian OS v9.x.

Symbian Press: developer.symbian.com/press

Quick Recipes on Symbian OSThis book aims to make it easier todevelop applications by describing anumber of common programming tasksand providing clear explanations of how tocomplete them. The recipes are divided bytechnology, including graphics, multimedia,location-based services, networking,telephony, connectivity, and messaging.

26

from

Symbian OS C++ for Mobile Phones, Volume 3 This book will help you to become aneffective Symbian OS developer, and willgive you a deep understanding of thefundamental principles upon whichSymbian OS is based.

Developing Software for Symbian OS, Second Edition This second edition of Developing Softwarefor Symbian OS helps software developersnew to Symbian OS to create smartphoneapplications. The original book has beenupdated for Symbian OS v9 and nowincludes a new chapter on applicationsigning and platform security, and updatesthroughout for Symbian OS v9 and changesto the development environment.

27

Also from

The Symbian OS Architecture Sourcebook

Mobile Python

This book conducts a rapid tour of thearchitecture of Symbian OS and providesan introduction to the key ideas of objectorientation (OO) in software, with adetailed exploration of the architecture ofSymbian OS.

Mobile Python is a practical hands-onbook that introduces the popular opensource programming language Pythonto the mobile space. It teaches how toprogram your own powerful - and fun -applications easily on Nokiasmartphones based on Symbian OS andthe S60 platform.

Symbian Press: developer.symbian.com/press

28

Also from

For all Symbian C++ developers:Symbian OS Communications Programming, 2nd Editionby Iain Campbell

S60 Programming - A Tutorial Guideby Coulton & Edwards

Symbian OS Explainedby Jo Stichbury

Symbian OS Internalsby Jane Sales

Symbian OS Platform Securityby Craig Heath

Smartphone Operating System Concepts with Symbian OS by Mike Jipping

Accredited Symbian Developer Primerby Jo Stichbury & Mark Jacobs

29

Also from

Published Booklets

Coding StandardsCoding TipsPerformance TipsEssential UIQ - Getting StartedGetting to Market Getting StartedQuick Recipes TasterJava ME on Symbian OSP.I.P.SCarbide.c++ v1.3Data Sharing TipsEssential S60 - Developers’ Guide

Translated Booklets

Chinese SpanishJapanese RussianKorean Persian

30

Notes

31

Notes

LOCALIZATION

SDN++

Localization is the adaptation of a software system for a specificregion or language.

This booklet provides an overview of the tasks a device creator needsto consider when localizing the operating system. The first sectiondescribes text processing issues that may arise when creating a newlanguage variant. The second describes the support that Symbian OSprovides for sets of user preferences, such as date format, that tend tovary from one language or geographic region to another, which areusually referred to as locales.

Localization is part of the SDN++ series, designed to provideinformation in a handy format to SDN++ developers. For furtherinformation about the booklets, or to make suggestions for futuretitles, please contact books@symbian.com.

SDN++ Why? What? Where? How?

Symbian PressSymbian Press publishes books designed to communicateauthoritative, timely, relevant and practical informationabout Symbian OS and related technologies. Informationabout the Symbian Press series can be found atdeveloper.symbian.com/books