diff --git a/docs/README.md b/docs/README.md index af5040039f2..543d4b9894c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,7 +4,7 @@ CLDR main page: [https://www.unicode.org/cldr](unicode.org/cldr) ## TR35 -- [Unicode Technical Standard #35 UNICODE LOCALE DATA MARKUP LANGUAGE (LDML)](ldml/tr35.html) +- [Unicode Technical Standard #35: Unicode Common Locale Data & Rules (CLDR)](ldml/tr35.html) ## RFC diff --git a/docs/ldml/tr35-acknowledgments.md b/docs/ldml/tr35-acknowledgments.md index 74900e62d7f..da0e03bbfcc 100644 --- a/docs/ldml/tr35-acknowledgments.md +++ b/docs/ldml/tr35-acknowledgments.md @@ -4,13 +4,16 @@ title: Acknowledgments --- ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Appendix B: Acknowledgments +# Unicode Common Locale Data & Rules (CLDR)
Appendix B: Acknowledgments For the full header, table of contents, and status, see [Part 1: Core](tr35.md). ### _Summary_ -This is a partial document, describing only the acknowledgments of contributors to LDML. For the other parts of the LDML see the [main LDML document](tr35.md). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers acknowledgments of contributors to CLDR. + +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ## Contents of Appendix B, Acknowledgments @@ -20,16 +23,16 @@ This is a partial document, describing only the acknowledgments of contributors Special thanks to the following people for their continuing overall contributions to the CLDR project, and for their specific contributions in the following areas. These descriptions only touch on the many contributions that they have made. -* Mark Davis for creating the initial version of LDML, and adding to and maintaining this specification, and for his work on the LDML code and tests, much of the supplemental data and overall structure, and transforms and keyboards. +* Mark Davis for creating the initial version of CLDR, and adding to and maintaining this specification, and for his work on the CLDR code and tests, much of the supplemental data and overall structure, and transforms and keyboards. * John Emmons for the POSIX conversion tool and metazones. -* Deborah Goldsmith for her contributions to LDML architecture and this specification. +* Deborah Goldsmith for her contributions to CLDR architecture and this specification. * Chris Hansten for coordinating and managing data submissions and vetting. * Erkki Kolehmainen and his team for their work on Finnish. * Steven R. Loomis for development of the survey tool and database management. * Peter Nugent for his contributions to the POSIX tool and from Open Office, and for coordinating and managing data submissions and vetting. * George Rhoten for his work on currencies. * Roozbeh Pournader (روزبه پورنادر) for his work on South Asian countries. -* Ram Viswanadha (రఘురామ్ విశ్వనాధ) for all of his work on LDML code and data integration, and for coordinating and managing data submissions and vetting. +* Ram Viswanadha (రఘురామ్ విశ్వనాధ) for all of his work on CLDR code and data integration, and for coordinating and managing data submissions and vetting. * Vladimir Weinstein (Владимир Вајнштајн) for his work on collation. * Yoshito Umaoka (馬岡 由人) for his work on the timezone architecture. * Rick McGowan for his work gathering language, script and region data. diff --git a/docs/ldml/tr35-collation.md b/docs/ldml/tr35-collation.md index 7e0e4b16303..ff037b06799 100644 --- a/docs/ldml/tr35-collation.md +++ b/docs/ldml/tr35-collation.md @@ -8,13 +8,14 @@ editors: ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 5: Collation +# Unicode Common Locale Data & Rules (CLDR)
Part 5: Collation ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers collation (sorting, searching, and grouping). -This is a partial document, describing only those parts of the LDML that are relevant for collation (sorting, searching & grouping). For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -30,7 +31,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -526,7 +527,8 @@ The format for this file uses the CLDR collation syntax, see _[Collation Tailori ``` -This element of the LDML format contains one or more `collation` elements, distinguished by type. Each `collation` contains elements with parametric settings, or rules that specify a certain sort order, as a tailoring of the root order, or both. +This element of the CLDR XML format contains one or more `collation` elements, distinguished by type. +Each `collation` contains elements with parametric settings, or rules that specify a certain sort order, as a tailoring of the root order, or both. > 👉 **Note**: CLDR collation tailoring data should follow the [CLDR Collation Guidelines](https://cldr.unicode.org/index/cldr-spec/collation-guidelines). @@ -544,7 +546,7 @@ A collation type name that starts with "private-", for example, "private-kana", > 👉 **Note**: There is an on-line demonstration of collation at [[LocaleExplorer](tr35.md#LocaleExplorer)] that uses the same rule syntax. (Pick the locale and scroll to "Collation Rules", near the end.) -> 👉 **Note**: In CLDR 23 and before, LDML collation files used an XML format. Starting with CLDR 24, the XML collation syntax is deprecated and no longer used. See the _[CLDR 23 version of this document](tr35-collation.md#Collation_Tailorings)_ for details about the XML collation syntax. +> 👉 **Note**: In CLDR 23 and before, CLDR collation files used a fine-grained XML format. Starting with CLDR 24, the fine-grained XML collation syntax is deprecated and no longer used. See the _[CLDR 23 version of this document](tr35-collation.md#Collation_Tailorings)_ for details about the XML collation syntax. #### Collation Type Fallback @@ -632,7 +634,7 @@ If a setting is not present, the CLDR default (or the default for the locale, if identic[strength I]
(identical) kanoignore[alternate non-ignorable]
- Sets alternate handling for variable weights, as described in [UCA], where "shifted" causes certain characters to be ignored in comparison. The default for LDML is different than it is in the UCA. In LDML, the default for alternate handling is non-ignorable, while in UCA it is shifted. In addition, in LDML only whitespace and punctuation are variable by default. + Sets alternate handling for variable weights, as described in [UCA], where "shifted" causes certain characters to be ignored in comparison. The default for CLDR is different than it is in the UCA. In CLDR, the default for alternate handling is non-ignorable, while in UCA it is shifted. In addition, in CLDR only whitespace and punctuation are variable by default. shifted[alternate shifted]
(UCA default) n/an/a
(blanked) @@ -680,7 +682,7 @@ If a setting is not present, the CLDR default (or the default for the locale, if #### Common settings combinations -Some commonly used parametric collation settings are available via combinations of LDML settings attributes: +Some commonly used parametric collation settings are available via combinations of CLDR settings attributes: * “Ignore accents”: **strength=primary** * “Ignore accents” but take case into account: **strength=primary caseLevel=on** @@ -1030,7 +1032,9 @@ The **case first** parameter controls whether to swap the order of upper and low Importantly, the case parameters have no effect in many instances. For example, they have no effect on the comparison of two non-ignorable characters with different primary weights, or with different secondary weights if the strength = **secondary (or higher).** -When either the **case level** or **case first** parameters are set, the following describes the derivation of the modified collation elements. It assumes the original levels for the code point are [p.s.t] (primary, secondary, tertiary). This derivation may change in future versions of LDML, to track the case characteristics more closely. +When either the **case level** or **case first** parameters are set, the following describes the derivation of the modified collation elements. +It assumes the original levels for the code point are [p.s.t] (primary, secondary, tertiary). +This derivation may change in future versions of CLDR, to track the case characteristics more closely. #### Untailored Characters diff --git a/docs/ldml/tr35-dates.md b/docs/ldml/tr35-dates.md index 40e945994d7..97bc8c2194b 100644 --- a/docs/ldml/tr35-dates.md +++ b/docs/ldml/tr35-dates.md @@ -7,13 +7,14 @@ editors: ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 4: Dates +# Unicode Common Locale Data & Rules (CLDR)
Part 4: Dates ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers date, time, and time zone formatting. -This is a partial document, describing only those parts of the LDML that are relevant for date, time, and time zone formatting. For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -29,7 +30,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -126,7 +127,7 @@ The LDML specification is divided into the following parts: ``` -The LDML top-level `` element contains information regarding the format and parsing of dates and times, the formatting of date/time intervals, and the naming of various calendar elements. +The CLDR top-level `` element contains information regarding the format and parsing of dates and times, the formatting of date/time intervals, and the naming of various calendar elements. * The `` element is described in [Calendar Elements](#Calendar_Elements). * The `` element is described in [Calendar Fields](#Calendar_Fields). @@ -1838,7 +1839,7 @@ This element is for data that is used to format a time zone’s generic location … ``` -This information was previously specified by the LDML `` element under each locale’s `` element. However, that approach had inheritance issues, and the data is not really locale-specific anyway. +This information was previously specified by the CLDR `` element under each locale’s `` element. However, that approach had inheritance issues, and the data is not really locale-specific anyway. ## Using Time Zone Names diff --git a/docs/ldml/tr35-general.md b/docs/ldml/tr35-general.md index 6bfc554338b..347959132ee 100644 --- a/docs/ldml/tr35-general.md +++ b/docs/ldml/tr35-general.md @@ -8,13 +8,14 @@ editors: ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 2: General +# Unicode Common Locale Data & Rules (CLDR)
Part 2: General ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers certain general parts of CLDR, including display names & transforms. -This is a partial document, describing general parts of the LDML: display names & transforms, etc. For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -30,7 +31,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -658,7 +659,7 @@ _Examples for zh.xml:_ When determining the character repertoire needed to support a language, a reasonable initial set would include at least the characters in the main and punctuation exemplar sets, along with the digits and common symbols associated with the numberSystems supported for the locale (see _[Numbering Systems](tr35-numbers.md#Numbering_Systems)_). -The _index_ characters are a set of characters for use as a UI "index", that is, a list of clickable characters (or character sequences) that allow the user to see a segment of a larger "target" list. For details see the [Unicode LDML: Collation](tr35-collation.md#Collation_Indexes) document. The index set may only contain characters whose lowercase versions are in the main and auxiliary exemplar sets, though for cased languages the index exemplars are typically in uppercase. Characters from the auxiliary exemplar set may be necessary in the index set if it needs to properly handle items such as names which may require characters not included in the main exemplar set. +The _index_ characters are a set of characters for use as a UI "index", that is, a list of clickable characters (or character sequences) that allow the user to see a segment of a larger "target" list. For details see the [Unicode CLDR: Collation](tr35-collation.md#Collation_Indexes) document. The index set may only contain characters whose lowercase versions are in the main and auxiliary exemplar sets, though for cased languages the index exemplars are typically in uppercase. Characters from the auxiliary exemplar set may be necessary in the index set if it needs to properly handle items such as names which may require characters not included in the main exemplar set. Here is a sample of the XML structure: @@ -693,11 +694,11 @@ The ordering of the characters in the set is irrelevant, but for readability in ### ~~Mapping~~ -**This element has been deprecated.** For information on its structure and how it was intended to specify locale-specific preferred encodings for various purposes (e-mail, web), see the [Mapping](tr35-general.md#Character_Mapping) section from the CLDR 27 version of the LDML Specification. +**This element has been deprecated.** For information on its structure and how it was intended to specify locale-specific preferred encodings for various purposes (e-mail, web), see the [Mapping](tr35-general.md#Character_Mapping) section from the CLDR 27 version of the CLDR specification. ### ~~Index Labels~~ -**This element and its subelements have been deprecated.** For information on its structure and how it was intended to provide data for a compressed display of index exemplar characters where space is limited, see the [Index Labels](tr35-general.md#IndexLabels) section from the CLDR 27 version of the LDML Specification. +**This element and its subelements have been deprecated.** For information on its structure and how it was intended to provide data for a compressed display of index exemplar characters where space is limited, see the [Index Labels](tr35-general.md#IndexLabels) section from the CLDR 27 version of the CLDR specification. ```xml @@ -882,7 +883,7 @@ The `paperSize` attribute gives the height and width of paper used for normal bu For both `measurementSystem` entries and `paperSize` entries, later entries for specific territories such as "US" will override the value assigned to that territory by earlier entries for more inclusive territories such as "001". -The measurement information was formerly in the main LDML file, and had a somewhat different format. +The measurement information was formerly in the main CLDR XML file, and had a somewhat different format. Again, for finer-grained detail about specific units for various usages, see **Part 6: Supplemental:** _[Preferred Units for Specific Usages](tr35-info.md#Preferred_Units_For_Usage)_. @@ -892,7 +893,7 @@ Again, for finer-grained detail about specific units for various usages, see **P ``` -The `measurement` element is deprecated in the main LDML files, because the data is more appropriately organized as connected to territories, not to linguistic data. Instead, the `measurementData` element in the supplemental data file should be used. +The `measurement` element is deprecated in the main CLDR XML files, because the data is more appropriately organized as connected to territories, not to linguistic data. Instead, the `measurementData` element in the supplemental data file should be used. ## Unit Elements @@ -1225,7 +1226,7 @@ For example, `kilowatt-hour-per-3-meter-5-second` has the equivalent normalized The simple_unit structure does not allow for any two simple_units to overlap. That is, there are no cases where simple_unit1 consists of X-Y and simple_unit2 consists of Y-Z. -This was not true in previous versions of LDML: cup-metric overlapped with metric-ton. +This was not true in previous versions of CLDR: cup-metric overlapped with metric-ton. That meant that the unit identifiers for the product_unit of cup and metric-ton and the product_unit of cup-metric and ton were ambiguous. The constraint that the identifiers can't overlap also means that parsing of multiple-subtag simple units is simpler. @@ -1904,7 +1905,7 @@ Note that the script and region codes are cased iff they are in the main subtag, ### Inheritance -The CLDR transforms are built using the following locale inheritance. While this inheritance is not required of LDML implementations, the transforms supplied with CLDR may not otherwise behave as expected without some changes. +The CLDR transforms are built using the following locale inheritance. While this inheritance is not required of CLDR implementations, the transforms supplied with CLDR may not otherwise behave as expected without some changes. For either the source or the target, the fallback starts from the maximized locale ID (using the likely-subtags data). It also uses the country for lookup before the base language is reached, and root is never accessed: instead the script(s) associated with the language are used. Where there are multiple scripts, the maximized script is tried first, and then the other scripts associated with the language (from supplemental data). @@ -3037,7 +3038,7 @@ To find a localized subfamily name such as “Extraleicht Schmal” for a font c ## Grammatical Features -LDML supplies grammatical information that can be used to distinguish localized forms on a per-locale basis. The current data is part of an initial phase; the longer term plan is to add structure to permit localized forms based on these features, starting with measurement units such as the dative form in Serbian of “kilometer”. That will allow unit values to be inserted as placeholders into messages and adopt the right forms for grammatical agreement. +CLDR supplies grammatical information that can be used to distinguish localized forms on a per-locale basis. The current data is part of an initial phase; the longer term plan is to add structure to permit localized forms based on these features, starting with measurement units such as the dative form in Serbian of “kilometer”. That will allow unit values to be inserted as placeholders into messages and adopt the right forms for grammatical agreement. The current data includes the following: diff --git a/docs/ldml/tr35-info.md b/docs/ldml/tr35-info.md index 316628a661b..21247cb4116 100644 --- a/docs/ldml/tr35-info.md +++ b/docs/ldml/tr35-info.md @@ -8,16 +8,17 @@ editors: ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 6: Supplemental +# Unicode Common Locale Data & Rules (CLDR)
Part 6: Supplemental For the full header, summary, and status, see [Part 1: Core](tr35.md). ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers supplemental data. -This is a partial document, describing only those parts of the LDML that are relevant for supplemental data. For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +For a general introduction see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -33,7 +34,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -1397,7 +1398,7 @@ The following is the algorithm for computing the preferred output unit from an i 3. Let ranked units be the result of a lookup of **region** in the category-usage preferences. There may be both region values and [containment regions](https://www.unicode.org/cldr/charts/latest/supplemental/territory_containment_un_m_49.html). 1. If the lookup fails, set **region** to its containing region and repeat. (This will always terminate because region `001` is always present.) * For example, CH (Switzerland) ⊂ 155 (Western Europe) ⊂ 150 (Europe) ⊂ 001 (World). - * This loop can be optimized to only include containing regions that occur in the data (eg, only 001 in LDML 45). + * This loop can be optimized to only include containing regions that occur in the data (eg, only 001 in CLDR 45). 4. If there is a **USM**, and the corresponding Fallback Region is different than **region**, and any of the ranked units don't match **USM**, then let ranked units be the result of a lookup of the Fallback Region in the category-usage preferences. #### Search the ranked units diff --git a/docs/ldml/tr35-keyboards.md b/docs/ldml/tr35-keyboards.md index 8a548bc836b..f4b5d5d731a 100644 --- a/docs/ldml/tr35-keyboards.md +++ b/docs/ldml/tr35-keyboards.md @@ -8,13 +8,14 @@ editors: ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 7: Keyboards +# Unicode Common Locale Data & Rules (CLDR)
Part 7: Keyboards ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers keyboards. -This is a partial document, describing keyboards. For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -32,7 +33,7 @@ See also [Compatibility Notice](#compatibility-notice). ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -165,7 +166,7 @@ Some goals of this format are: 2. Provide definitive platform-independent definitions for new keyboard layouts. * For example, a new French standard keyboard layout would have a single definition which would be usable across all implementations. 3. Allow platforms to be able to use CLDR keyboard data for the character-emitting keys (non-frame) aspects of keyboard layouts. -4. Deprecate & archive existing LDML platform-specific layouts so they are not part of future releases. +4. Deprecate & archive existing CLDR platform-specific layouts so they are not part of future releases. @@ -193,7 +194,7 @@ Note that in parts of this document, the format `@x` is used to indicate the _at ### Compatibility Notice > A major rewrite of this specification, called "Keyboard 3.0", was introduced in CLDR v45. -> The changes required were too extensive to maintain compatibility. For this reason, the `ldmlKeyboard3.dtd` DTD is _not_ compatible with DTDs from prior versions of CLDR such as v43 and prior. +> The changes required were too extensive to maintain compatibility. For this reason, the `CLDRKeyboard3.dtd` DTD is _not_ compatible with DTDs from prior versions of CLDR such as v43 and prior. > > To process earlier XML files, use the data and specification from v43.1, found at > @@ -241,7 +242,7 @@ Keyboard implementations will typically consist of two parts: **Keyboard:** A particular arrangement of keys for the inputting of text, such as a hardware keyboard or a touch keyboard. -**Keyboard author:** The person or group of people designing and producing a particular keyboard layout designed to support one or more languages. In the context of this specification, that author may be editing the LDML XML file directly or by means of software tools. +**Keyboard author:** The person or group of people designing and producing a particular keyboard layout designed to support one or more languages. In the context of this specification, that author may be editing the CLDR XML file directly or by means of software tools. **Keyboard layout:** A layout is the overall keyboard configuration for a particular locale. Within a keyboard layout, there is a single base map, one or more key maps and zero or more transforms. @@ -306,7 +307,7 @@ Attribute values escaped in this manner are annotated with the ` -### Changes in LDML Version 49 (Differences from Version 48.0) +### Changes in CLDR Version 49 (Differences from Version 48.0) +### General + +* Replaced “Common Locale Data Repository (CLDR)” by “Unicode Common Locale Data & Rules (CLDR)” +and the title of the specification “UTS #35: Unicode Locale Data Markup Language (LDML)” by “UTS #35: Unicode Common Locale Data & Rules (CLDR)”. + * The former names and the acronym “LDML” led to many misunderstandings and have been changed for clarity. + * Note that all references under the former names and the LDML acronym remain valid; such references on the Unicode site will be updated over time. + ### Locales * [`typeValues`](tr35-general.md#type-values) Added typeValues for On/Off translations in menus and locale display names diff --git a/docs/ldml/tr35-numbers.md b/docs/ldml/tr35-numbers.md index 68c3c73cdbd..0d93ab69b0f 100644 --- a/docs/ldml/tr35-numbers.md +++ b/docs/ldml/tr35-numbers.md @@ -8,13 +8,15 @@ editors: ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 3: Numbers +# Unicode Common Locale Data & Rules (CLDR)
Part 3: Numbers ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). -This is a partial document, describing only those parts of the LDML that are relevant for number and currency formatting. For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers number and currency formatting. + +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -31,7 +33,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -1069,7 +1071,7 @@ Notice that the currency code is completely independent of the end-user's langua Thus logically speaking, once a currency amount is entered into a system, it should be logically accompanied by a currency code in all processing. This currency code is independent of whatever the user's original locale was. Only in badly-designed software is the currency code (or equivalent) not present, so that the software has to "guess" at the currency code based on the user's locale. -> **Note:** The number of decimal places **and** the rounding for each currency is not locale-specific data, and is not contained in the Locale Data Markup Language format. Those values override whatever is given in the currency `numberFormat`. For more information, see _[Supplemental Currency Data](#Supplemental_Currency_Data)_. +> **Note:** The number of decimal places **and** the rounding for each currency is not locale-specific data, and is not contained in the CLDR format. Those values override whatever is given in the currency `numberFormat`. For more information, see _[Supplemental Currency Data](#Supplemental_Currency_Data)_. For background information on currency names, see [[CurrencyInfo](tr35.md#CurrencyInfo)]. diff --git a/docs/ldml/tr35-personNames.md b/docs/ldml/tr35-personNames.md index e6fd40a06ca..1788bdc85d0 100644 --- a/docs/ldml/tr35-personNames.md +++ b/docs/ldml/tr35-personNames.md @@ -11,13 +11,14 @@ editors: --- ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML)
Part 8: Person Names +# Unicode Common Locale Data & Rules (CLDR)
Part 8: Person Names ### _Summary_ -This document describes parts of an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part covers person name structures and formatting. -This is a partial document, describing only those parts of the LDML that are relevant for person names (name structure, formats, sorting). For the other parts of the LDML see the [main LDML document](tr35.md) and the links above. +For the full header and a general introduction, see [Unicode Common Locale Data & Rules (CLDR): Part 1](tr35.md). ### _Status_ @@ -33,7 +34,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -117,7 +118,7 @@ There is a wide variety in the way that people’s names appear in different lan * The ordering of name fields can be different across languages, as well as the spacing (or lack thereof) and punctuation. * Name formatting needs to be adapted to different circumstances, such as a need to be presented shorter or longer; formal or informal context; or when talking about someone, or talking to someone, or as a monogram (JFK). -This document provides the [LDML](tr35.md) specification for formatting of personal names, using data, structure, and examples. +This document provides the [CLDR](tr35.md) specification for formatting of personal names, using data, structure, and examples. The CLDR functionality is targeted at formatting names for typical usage on computers (e.g. contact names, automated greetings, etc.), rather than being designed for special circumstances or protocol, such addressing royalty. However, the structure may be enhanced in the future when it becomes clear that additional features are needed for some languages. @@ -215,7 +216,8 @@ Sections below specify the precise manner in which a pattern is selected, and ho ## XML Structure -Person name formatting data is stored as LDML with schema defined as follows. Each element has a brief description of the usage, but the exact algorithms for using these elements are provided in [Formatting Process](#formatting-process). +Person name formatting data is stored as CLDR XML with a schema defined as follows: +Each element has a brief description of the usage, but the exact algorithms for using these elements are provided in [Formatting Process](#formatting-process). ### personNames Element @@ -224,7 +226,7 @@ Person name formatting data is stored as LDML with schema defined as follows. Ea ``` -The LDML top-level `` element contains information regarding the formatting of person names, and the formatting of person names in specific contexts for a specific locale. +The CLDR top-level `` element contains information regarding the formatting of person names, and the formatting of person names in specific contexts for a specific locale. ### personName Element @@ -367,7 +369,7 @@ A modifier is supplied, _-informal_, which can be used to indicate which data el ## Person Name Attributes -A person name pattern may have any of four attributes: order, length, usage, and formality. LDML specifies that all the values for these attributes are unique. For example, because length=long is valid, usage=long cannot also be valid. That allows the pattern labels to be simple, because the attribute names can be skipped. That is, +A person name pattern may have any of four attributes: order, length, usage, and formality. CLDR specifies that all the values for these attributes are unique. For example, because length=long is valid, usage=long cannot also be valid. That allows the pattern labels to be simple, because the attribute names can be skipped. That is, > `{order=givenFirst, length=long, usage=referring, formality=formal}` @@ -1075,7 +1077,8 @@ The output is: ## Sample Name -The sampleName element is used for test names in the personNames LDML data for each locale to aid in testing and display in the CLDR Survey Tool. They are not intended to be used in production software as prompts or placeholders in a user interface and should not be displayed in a user interface. +The sampleName element is used for test names in the personNames CLDR XML data for each locale to aid in testing and display in the CLDR Survey Tool. +They are not intended to be used in production software as prompts or placeholders in a user interface and should not be displayed in a user interface. ### Syntax diff --git a/docs/ldml/tr35.md b/docs/ldml/tr35.md index 0eee6c401eb..dbedf671c2c 100644 --- a/docs/ldml/tr35.md +++ b/docs/ldml/tr35.md @@ -21,11 +21,25 @@ latest: https://www.unicode.org/reports/tr35 ## Unicode Technical Standard #35 -# Unicode Locale Data Markup Language (LDML) +# Unicode Common Locale Data & Rules (CLDR)
Part 1: Core ### _Summary_ -This document describes an XML format (_vocabulary_) for the exchange of structured locale data. This format is used in the [Unicode Common Locale Data Repository](https://www.unicode.org/cldr/). +[Unicode Common Locale Data & Rules (CLDR)](https://www.unicode.org/cldr/) provides structured data +for internationalization services in a wide variety of languages. +This specification determines how that data is properly interpreted, and also specifies (among other things): +* identifiers for locales, units of measurement, numbering systems, and calendars +* formatting of dates, times, timezones, numbers, currencies, and measurements +* sorting and searching data and rules + +The CLDR specification is presented in multiple Parts, each on its own web page. +This Part provides a general introduction and covers languages, locales, inheritance, and basic structure. + +[Unicode Common Locale Data & Rules (CLDR)](https://www.unicode.org/cldr/) was formerly known as “Common Locale Data Repository (CLDR)” +and this specification was formerly known as “Locale Data Markup Language (LDML)”. +The former names and the acronym “LDML” led to many misunderstandings and have been changed for clarity. +However, all references under the former names and the LDML acronym remain valid; +such references on the Unicode site will be updated over time, as will references to the LDML acronym in this specification. ### _Status_ @@ -42,7 +56,7 @@ Unicode Technical Reports are governed by the Unicode [Terms of Use](https://www ## Parts -The LDML specification is divided into the following parts: +The CLDR specification is divided into the following parts: * Part 1: [Core](tr35.md#Contents) (languages, locales, basic structure) * Part 2: [General](tr35-general.md#Contents) (display names & transforms, etc.) @@ -255,32 +269,39 @@ But there remain significant differences across systems and applications in the > **Note:** There are many different equally valid ways in which data can be judged to be "correct" for a particular locale. The goal for the common locale data is to make it as consistent as possible with existing locale data, and acceptable to users in that locale. -This document specifies an XML format for the communication of locale data: the Unicode Locale Data Markup Language (LDML). This provides a common format for systems to interchange locale data so that they can get the same results in the services provided by internationalization libraries. It also provides a standard format that can allow users to customize the behavior of a system. With it, for example, collation (sorting) rules can be exchanged, allowing two implementations to exchange a specification of tailored collation rules. Using the same specification, the two implementations will achieve the same results in comparing strings. Unicode LDML can also be used to let a user encapsulate specialized sorting behavior for a specific domain, or create a customized locale for a minority language. Unicode LDML is also used in the Unicode Common Locale Data Repository (CLDR). CLDR uses an open process for reconciling differences between the locale data used on different systems and validating the data, to produce with a useful, common, consistent base of locale data. +Common Locale Data & Rules (CLDR) provides a common format for systems to exchange locale data and get the same results from services provided by internationalization libraries. +The specification also provides a standard format that allows users to customize the behavior of a system. +With it, for example, collation (sorting) rules can be exchanged, allowing two implementations to share tailored collation rules. +Using the same specification, the two implementations will achieve the same results in comparing strings. +The CLDR specification can also be used to let a user encapsulate specialized sorting behavior for a specific domain, or create a customized locale for a minority language. + +CLDR uses an open process for reconciling differences between the locale data used on different systems and validating the data, to produce with a useful, common, consistent base of locale data. -For more information, see the Common Locale Data Repository project page [[LocaleProject](#localeProject)]. +For more information, see the Common Locale Data & Rules project page [[LocaleProject](#localeProject)]. -As LDML is an interchange format, it was designed for ease of maintenance and simplicity of transformation into other formats, above efficiency of run-time lookup and use. Implementations should consider converting LDML data into a more compact format prior to use. +The CLDR interchange format was designed for ease of maintenance and simplicity of transformation into other formats, above efficiency of run-time lookup and use. +Implementers should consider converting CLDR data into a more compact format prior to use. ### Conformance -There are many ways to use the Unicode LDML specification and the CLDR data. +There are many ways to use the Unicode CLDR specification and the CLDR data. The Unicode Consortium does not restrict the ways in which the format or data are used. -However, an implementation may also claim conformance to the LDML specification and/or to CLDR data, as follows: +However, an implementation may also claim conformance to the CLDR specification and/or to CLDR data, as follows: _**UAX35-C1.**_ An implementation that claims conformance to this specification shall: 1. Identify the sections of the specification that it conforms to. - * For example, an implementation might claim conformance to all LDML features except for _transforms_ and _segments_. - * The names of sections may change for clarity, so the associated links should be included in any reference — links into LDML will remain stable. -2. Interpret the relevant elements and attributes of LDML data in accordance with the descriptions in those sections. + * For example, an implementation might claim conformance to all CLDR features except for _transforms_ and _segments_. + * The names of sections may change for clarity, so the associated links should be included in any reference — links into the CLDR specification will remain stable. +2. Interpret the relevant elements and attributes of CLDR data in accordance with the descriptions in those sections. * For example, an implementation that claims conformance to the date format patterns must interpret the characters in such patterns according to [Date Field Symbol Table](tr35-dates.md#Date_Field_Symbol_Table). 3. Declare which types of CLDR data it uses. * For example, an implementation might declare that it only uses language names, and those with a _draft_ status of _contributed_ or _approved_. 4. Declare when it overrides CLDR data, or uses `alt` data * For example, for `//ldml/numbers/symbols/group` an implementation could use `alt="official"` data. -An implementation may also make a _general claim_ of conformance to the LDML specification and/or CLDR data. +An implementation may also make a _general claim_ of conformance to the CLDR specification and/or CLDR data. Such a claim is understood to claim conformance to all portions of this specification that are relevant to the operations performed by the implementation, except for those specifically declared as exceptions. For example, if an implementation making a _general claim_ of conformance performs date formatting, and does not declare date formatting as an exception, @@ -293,7 +314,7 @@ it is understood to be claiming conformance to date formatting as described in t ~~External specifications may also reference particular components of Unicode locale or language identifiers, such as:~~ -~~> _Field X can contain any Unicode region subtag values as given in Unicode Technical Standard #35: Unicode Locale Data Markup Language (LDML), excluding grouping codes._~~ +~~> _Field X can contain any Unicode region subtag values as given in Unicode Technical Standard #35: Unicode Common Locale Data & Rules (CLDR), excluding grouping codes._~~ NOTE: _**UAX35-C2.**_ is replaced by the following generalization. @@ -436,7 +457,7 @@ For example, the following is from a sample header: If an implementation overrides CLDR data, then various lines in the relevant test files may need to be modified correspondingly, or skipped. ### EBNF -The EBNF syntax used in LDML is a variant of the Extended Backus-Naur Form (EBNF) notation used in [W3C XML Notation](https://www.w3.org/TR/REC-xml/#sec-notation). The main differences are: +The EBNF syntax used in CLDR is a variant of the Extended Backus-Naur Form (EBNF) notation used in [W3C XML Notation](https://www.w3.org/TR/REC-xml/#sec-notation). The main differences are: 1. Bounded repetition following Perl regex syntax is allowed, such as `digit{3}` for 3 digits, `digit{3,5}` for 3 to 5 digits, and `digit{3,}` for 3 or more digits. 2. Whitespace inside bracketed enumerations and ranges is ignored. @@ -451,7 +472,7 @@ In the text, this is sometimes referred to as "EBNF (Perl-based)". ## What is a Locale? -Before diving into the XML structure, it is helpful to describe the model behind the structure. People do not have to subscribe to this model to use data in LDML, but they do need to understand it so that the data can be correctly translated into whatever model their implementation uses. +Before diving into the XML structure, it is helpful to describe the model behind the structure. People do not have to subscribe to this model to use data in CLDR, but they do need to understand it so that the data can be correctly translated into whatever model their implementation uses. The first issue is basic: _what is a locale?_ In this model, a locale is an identifier (id) that refers to a set of user preferences that tend to be shared across significant swaths of the world. Traditionally, the data associated with this id provides support for formatting and parsing of dates, times, numbers, and currencies; for measurement units, for sort-order (collation), plus translated names for time zones, languages, countries (regions), and scripts. The data can also include support for text boundaries (character, word, line, and sentence), text transformations (including transliterations), and other services. @@ -467,7 +488,7 @@ We will speak of data as being "in locale X". That does not imply that a locale ## Unicode Language and Locale Identifiers -Unicode LDML uses stable identifiers based on [[BCP47](#BCP47)] for distinguishing among languages, locales, regions, currencies, time zones, transforms, and so on. There are many systems for identifiers for these entities. The Unicode LDML identifiers may not match the identifiers used on a particular target system. If so, some process of identifier translation may be required when using LDML data. +Unicode CLDR uses stable identifiers based on [[BCP47](#BCP47)] for distinguishing among languages, locales, regions, currencies, time zones, transforms, and so on. There are many systems for identifiers for these entities. The Unicode CLDR identifiers may not match the identifiers used on a particular target system. If so, some process of identifier translation may be required when using CLDR data. The BCP 47 extensions (-u- and -t-) are described in _[Unicode BCP 47 U Extension](#u_Extension)_ and _[Unicode BCP 47 T Extension](#BCP47_T_Extension)_. @@ -561,7 +582,7 @@ As of the release of this specification, there were no other_extensions defined. As for terminology, the term _code_ may also be used instead of "subtag", and "territory" instead of "region". The primary language subtag is also called the _base language code_. For example, the base language code for "en-US" (American English) is "en" (English). The _type_ may also be referred to as a _value_ or _key-value_. -All identifier field values are case-insensitive. Although case distinctions do not carry any special meaning, an implementation of LDML should use the casing recommendations in [[BCP47](#BCP47)], especially when a Unicode locale identifier is used for locale data exchange in software protocols. +All identifier field values are case-insensitive. Although case distinctions do not carry any special meaning, an implementation of CLDR should use the casing recommendations in [[BCP47](#BCP47)], especially when a Unicode locale identifier is used for locale data exchange in software protocols. The identifiers can vary in case and in the separator characters. The "-" and "\_" separators are treated as equivalent, although "-" is preferred. @@ -635,7 +656,7 @@ Unicode language and locale identifiers inherit the design and the repertoire of * No irregular BCP 47 legacy language tags (marked as “Type: grandfathered” in BCP 47) are allowed (these are all deprecated in BCP 47) * A tag must not start with the subtag "x": thus a _privateuse_ (eg x-abc) can only be after a language subtag, like "und" * It allows for certain semantic additions and constraints: - * Certain codes that are private-use in BCP 47 and ISO are given semantics by LDML + * Certain codes that are private-use in BCP 47 and ISO are given semantics by CLDR * Each macrolanguage has an identified primary encompassed language, which is treated as an alias for the macrolanguage, and thus is replaced when canonicalizing (as allowed by BCP 47, see [Section 4.1.2](https://www.rfc-editor.org/rfc/rfc5646.html#section-4.1.2)) * It allows certain syntax for backwards compatibility (not BCP 47-compatible): * The "\_" character for field separator characters, as well as the "-" used in [[BCP47](#BCP47)] (however, the canonical form is with "-") @@ -821,7 +842,8 @@ Unicode identifiers give specific semantics to the following subtags. | `ZZ` | `ZZZ` | 999 | Unknown or Invalid Territory | used in APIs or as replacement for invalid code | private use | -The private use subtags listed as **excluded** in _[Private Use Codes](#Private_Use_Codes)_ will normally never be given specific semantics in Unicode identifiers, and are thus safe for use for other purposes by other applications. However, LDML may follow widespread industry practice in the use of some of these codes, such as for XK. +The private use subtags listed as **excluded** in _[Private Use Codes](#Private_Use_Codes)_ will normally never be given specific semantics in Unicode identifiers, and are thus safe for use for other purposes by other applications. +However, CLDR may follow widespread industry practice in the use of some of these codes, such as for XK. The CLDR provides data for normalizing territory/region codes, including mapping overlong codes like "eng-840" or "eng-USA" to the correct code "en-US". @@ -938,7 +960,7 @@ The Equivalents for Subsets are not as well defined, so the "Equivalents" are ma The special codes most frequently used are in the locale identifiers `zh-Hans`, `zh-Hant`, `ja-Jpan`, and `ko-Kore`: the first two are **Subsets**, and the last two are **Compounds**. -These are used, for example, in [Likely Subtags](#Likely_Subtags) in LDML. +These are used, for example, in [Likely Subtags](#Likely_Subtags) in CLDR. The **Equivalent** values in the **Subset** variants are only approximate, _and_ the variants are also visual variants. Thus `Hans` is a request for: @@ -1255,15 +1277,15 @@ The `va` key ID defines a special variant used for locales. Well-formed values m For more information on the allowed keys and types, see the specific elements below, and [U Extension Data Files](#Unicode_Locale_Extension_Data_Files). -Additional keys or types might be added in future versions. Implementations of LDML should be robust to handle any syntactically valid key or type values. +Additional keys or types might be added in future versions. Implementations of CLDR should be robust to handle any syntactically valid key or type values. #### Numbering System Data -LDML supports multiple numbering systems. The identifiers for those numbering systems are defined in the file **bcp47/number.xml**. For example, for the latest version of the data see [bcp47/number.xml](https://github.com/unicode-org/cldr/blob/main/common/bcp47/number.xml). +CLDR supports multiple numbering systems. The identifiers for those numbering systems are defined in the file **bcp47/number.xml**. For example, for the latest version of the data see [bcp47/number.xml](https://github.com/unicode-org/cldr/blob/main/common/bcp47/number.xml). Details about those numbering systems are defined in **supplemental/numberingSystems.xml**. For example, for the latest version of the data see [supplemental/numberingSystems.xml](https://github.com/unicode-org/cldr/blob/main/common/supplemental/numberingSystems.xml). -LDML makes certain stability guarantees on this data: +CLDR makes certain stability guarantees on this data: 1. Like other BCP 47 identifiers, once a numeric identifier is added to **bcp47/number.xml** or **numberingSystems.xml**, it will never be removed from either of those files. 2. If an identifier has type="numeric" in numberingSystems.xml, then @@ -1272,7 +1294,7 @@ LDML makes certain stability guarantees on this data: #### Time Zone Identifiers -LDML inherits time zone IDs from the tz database [[Olson](#Olson)]. Because these IDs from the tz database do not satisfy the BCP 47 language subtag syntax requirements, CLDR defines short identifiers for the use in the Unicode locale extension. The short identifiers are defined in the file **common/bcp47/timezone.xml**. +CLDR inherits time zone IDs from the tz database [[Olson](#Olson)]. Because these IDs from the tz database do not satisfy the BCP 47 language subtag syntax requirements, CLDR defines short identifiers for the use in the Unicode locale extension. The short identifiers are defined in the file **common/bcp47/timezone.xml**. The short identifiers use UN/LOCODE [[LOCODE](#LOCODE)] (excluding a space character) codes where possible. For example, the short identifier for "America/Los_Angeles" is "uslax" (the LOCODE for Los Angeles, US is "US LAX"). Identifiers of length not equal to 5 are used where there is no corresponding UN/LOCODE, such as "usnavajo" for "America/Shiprock", or "utcw01" for "Etc/GMT+1", so that they do not overlap with future UN/LOCODE. @@ -1598,17 +1620,17 @@ For information about the registration process, meaning, and usage of the 't' ex ### Compatibility with Older Identifiers -LDML version before 1.7.2 used slightly different syntax for variant subtags and locale extensions. Implementations of LDML may provide backward compatible identifier support as described in following sections. +CLDR version before 1.7.2 used slightly different syntax for variant subtags and locale extensions. Implementations of CLDR may provide backward compatible identifier support as described in following sections. #### Old Locale Extension Syntax -LDML 1.7 or older specification used different syntax for representing Unicode locale extensions. The previous definition of Unicode locale extensions had the following structure: +CLDR 1.7 or older specification used different syntax for representing Unicode locale extensions. The previous definition of Unicode locale extensions had the following structure: | | EBNF | | ----------------------------- | ---- | | `old_unicode_locale_extensions` | `= "@" old_key "=" old_type`
`(";" old_key "=" old_type)*` | -The new specification mandates keys to be two alphanumeric characters and types to be three to eight alphanumeric characters. As the result, new codes were assigned to all existing keys and some types. For example, a new key "co" replaced the previous key "collation", a new type "phonebk" replaced the previous type "phonebook". However, the existing collation type "big5han" already satisfied the new requirement, so no new type code was assigned to the type. All new keys and types introduced after LDML 1.7 satisfy the new requirement, so they do not have aliases dedicated for the old syntax, except time zone types. The conversion between old types and new types can be done regardless of key, with one known exception (old type "traditional" is mapped to new type "trad" for collation and "traditio" for numbering system), and this relationship will be maintained in the future versions unless otherwise noted. +The new specification mandates keys to be two alphanumeric characters and types to be three to eight alphanumeric characters. As the result, new codes were assigned to all existing keys and some types. For example, a new key "co" replaced the previous key "collation", a new type "phonebk" replaced the previous type "phonebook". However, the existing collation type "big5han" already satisfied the new requirement, so no new type code was assigned to the type. All new keys and types introduced after CLDR 1.7 satisfy the new requirement, so they do not have aliases dedicated for the old syntax, except time zone types. The conversion between old types and new types can be done regardless of key, with one known exception (old type "traditional" is mapped to new type "trad" for collation and "traditio" for numbering system), and this relationship will be maintained in the future versions unless otherwise noted. The new specification introduced a new field `attribute` in addition to key/type pairs in the Unicode locale extension. When it is necessary to map a new Unicode locale identifier with `attribute` field to a well-formed old locale identifier, a special key name _attribute_ with the value of entire `attribute` subtags in the new identifier is used. For example, a new identifier `ja-u-xxx-yyy-ca-japanese` is mapped to an old identifier `ja@attribute=xxx-yyy;calendar=japanese` . @@ -1616,7 +1638,7 @@ The chart below shows some example mappings between the new syntax and the old s ###### Table: Locale Extension Mappings -| Old (LDML 1.7 or older) | New | +| Old (CLDR 1.7 or older) | New | | ------------------------------------------ | ---------------------------- | | `de_DE@collation=phonebook` | `de_DE_u_co_phonebk` | | `zh_Hant_TW@collation=big5han` | `zh_Hant_TW_u_co_big5han` | @@ -1636,7 +1658,7 @@ Where the old API is supplied the bcp47 language code, or vice versa, the recomm #### Legacy Variants -Old LDML specification allowed codes other than registered [[BCP47](#BCP47)] variant subtags used in Unicode language and locale identifiers for representing variations of locale data. Unicode locale identifiers including such variant codes can be converted to the new [[BCP47](#BCP47)] compatible identifiers by following the descriptions below: +Old CLDR specification allowed codes other than registered [[BCP47](#BCP47)] variant subtags used in Unicode language and locale identifiers for representing variations of locale data. Unicode locale identifiers including such variant codes can be converted to the new [[BCP47](#BCP47)] compatible identifiers by following the descriptions below: ###### Table: Legacy Variant Mappings @@ -1664,7 +1686,7 @@ Examples: The locale id format generally follows the description in the _OpenI18N Locale Naming Guideline_ [[NamingGuideline](#NamingGuideline)], with some enhancements. The main differences from those guidelines are that the locale id: -1. does not include a charset (since the data in LDML format always provides a representation of all Unicode characters. The repository is stored in UTF-8, although that can be transcoded to other encodings as well.) +1. does not include a charset (since the data in CLDR formats always provides a representation of all Unicode characters. The repository is stored in UTF-8, although that can be transcoded to other encodings as well.) 2. adds the ability to have a variant, as in Java 3. adds the ability to discriminate the written language by script (or script variant). 4. is a superset of [[BCP47](#BCP47)] codes. @@ -1945,7 +1967,7 @@ The inherited item lookup must remain stable, because the resources are built wi Resource bundle lookup, on the other hand, is more flexible; changes in the view of the "best" match between the input request and the output bundle are more tolerant, when represent overall improvements for users. For more information, see _[A.1 Element fallback](#Fallback_Elements)_. -Where the LDML inheritance relationship does not match a target system, such as POSIX, the data logically should be fully resolved in converting to a format for use by that system, by adding _all_ inherited data to each locale data set. +Where the CLDR inheritance relationship does not match a target system, such as POSIX, the data logically should be fully resolved in converting to a format for use by that system, by adding _all_ inherited data to each locale data set. For a more complete description of how inheritance applies to data, and the use of keywords, see _[Inheritance](#Inheritance_and_Validity)_ . @@ -1953,7 +1975,7 @@ The locale data does not contain general character properties that are derived f **Warning:** If a locale has a different script than its parent (for example, sr_Latn), then special attention must be paid to make sure that all inheritance is covered. For example, auxiliary exemplar characters may need to be empty ("[]") to block inheritance. -**Empty Override:** There is one special value reserved in LDML to indicate that a child locale is to have no value for a path, even if the parent locale has a value for that path. That value is "∅∅∅". For example, if there is no phrase for "two days ago" in a language, that can be indicated with: +**Empty Override:** There is one special value reserved in CLDR to indicate that a child locale is to have no value for a path, even if the parent locale has a value for that path. That value is "∅∅∅". For example, if there is no phrase for "two days ago" in a language, that can be indicated with: ```xml @@ -2207,7 +2229,7 @@ Note that region-priority inheritance does not currently make use of parent loca ### Inheritance and Validity -The following describes in more detail how to determine the exact inheritance of elements, and the validity of a given element in LDML. +The following describes in more detail how to determine the exact inheritance of elements, and the validity of a given element in CLDR. #### Definitions @@ -2262,7 +2284,7 @@ An element chain A is an _extension_ of an element chain B if B is equivalent to 1. `//ldml/localeDisplayNames` 2. `//ldml/localeDisplayNames/languages/language[@type="ar"]` -An LDML file can be thought of as an ordered list of _element pairs_: , where the element chains are all the chains for the end-nodes. (This works because of restrictions on the structure of LDML, including that it does not allow mixed content.) The ordering is the ordering that the element chains are found in the file, and thus determined by the DTD. +A CLDR file can be thought of as an ordered list of _element pairs_: , where the element chains are all the chains for the end-nodes. (This works because of restrictions on the structure of CLDR, including that it does not allow mixed content.) The ordering is the ordering that the element chains are found in the file, and thus determined by the DTD. For example, some of those pairs would be the following. Notice that the first has the null string as element contents. @@ -2279,7 +2301,7 @@ For example, some of those pairs would be the following. Notice that the first h > > 1. <`//ldml/dates/calendars/calendar[@type='gregorian']/week/weekendStart[@day='sun'][@time='00:00']`,`""`> > > 2. <`//ldml/dates/calendars/calendar[@type='gregorian']/week/weekendStart`,`[@day='sun'][@time='00:00']`> -Two LDML element chains are _equivalent_ when they would be identical if all attributes and their values were removed — except for distinguishing attributes. Thus the following are equivalent: +Two CLDR element chains are _equivalent_ when they would be identical if all attributes and their values were removed — except for distinguishing attributes. Thus the following are equivalent: * `//ldml/localeDisplayNames/languages/language[@type="ar"]` * `//ldml/localeDisplayNames/languages/language[@type="ar"][@draft="unconfirmed"]` @@ -2294,7 +2316,7 @@ To produce fully resolved locale data file from CLDR for a locale ID L, you star 1. Let Result be initially L. 2. For each Li in the locale chain for L, starting at L and going up to root: - 1. Let Temp be a copy of the pairs in the LDML file for Li + 1. Let Temp be a copy of the pairs in the CLDR file for Li 2. Replace each alias in Temp by the resolved list of pairs it points to. 1. The resolved list of pairs is obtained by recursively applying this procedure. 2. That alias now blocks any inheritance from the parent. (See _[Common Elements](#Common_Elements)_ for an example.) @@ -2305,11 +2327,11 @@ To produce fully resolved locale data file from CLDR for a locale ID L, you star * When adding an element pair to a result, it has to go in the right order for it to be valid according to the DTD. * The identity element and its children are unaffected by resolution. -* The LDML data must be constructed so as to avoid circularity in step 2.2. +* The CLDR data must be constructed so as to avoid circularity in step 2.2. #### Valid Data -The attribute `draft="x"` in LDML means that the data has not been approved by the subcommittee. (For more information, see [Process](https://cldr.unicode.org/index/process)). However, some data that is not explicitly marked as `draft` may be implicitly `draft`, either because it inherits it from a parent, or from an enclosing element. +The attribute `draft="x"` in CLDR means that the data has not been approved by the subcommittee. (For more information, see [Process](https://cldr.unicode.org/index/process)). However, some data that is not explicitly marked as `draft` may be implicitly `draft`, either because it inherits it from a parent, or from an enclosing element. **Example 2.** Suppose that new locale data is added for af (Afrikaans). To indicate that all of the data is _unconfirmed_, the attribute can be added to the top level. @@ -2340,9 +2362,10 @@ Any data can be added to that file, and the status will all be `draft="unconfirm ``` -However, normally the draft attributes should be canonicalized, which means they are pushed down to leaf nodes as described in _[Canonical Form](#Canonical_Form)_. If an LDML file does have draft attributes that are not on leaf nodes, the file should be interpreted as if it were the canonicalized version of that file. +However, normally the draft attributes should be canonicalized, which means they are pushed down to leaf nodes as described in _[Canonical Form](#Canonical_Form)_. +If a CLDR file does have draft attributes that are not on leaf nodes, the file should be interpreted as if it were the canonicalized version of that file. -More formally, here is how to determine whether data for an element chain E is implicitly or explicitly draft, given a locale L. Sections 1, 2, and 4 are simply formalizations of what is in LDML already. Item 3 adds the new element. +More formally, here is how to determine whether data for an element chain E is implicitly or explicitly draft, given a locale L. Sections 1, 2, and 4 are simply formalizations of what is in CLDR already. Item 3 adds the new element. #### Checking for Draft Status @@ -2804,7 +2827,7 @@ The `paradigmLocales` also allow matching to macroregions. For example, desired= ## XML Format -There are two kinds of data that can be expressed in LDML: language-dependent data and supplementary data. In either case, data can be split across multiple files, which can be in multiple directory trees. +There are two kinds of data that can be expressed in CLDR: language-dependent data and supplementary data. In either case, data can be split across multiple files, which can be in multiple directory trees. For example, the language-dependent data for Japanese in CLDR is present in the following files: @@ -2859,7 +2882,7 @@ The XML structure is stable over releases. Elements and attributes may be deprec In general, all translatable text in this format is in element contents, while attributes are reserved for types and non-translated information (such as numbers or dates). The reason that attributes are not used for translatable text is that spaces are not preserved, and we cannot predict where spaces may be significant in translated material. -There are two kinds of elements in LDML: _rule_ elements and _structure_ elements. +There are two kinds of elements in CLDR: _rule_ elements and _structure_ elements. For structure elements, there are restrictions to allow for effective inheritance and processing: @@ -2870,7 +2893,7 @@ For structure elements, there are restrictions to allow for effective inheritanc To illustrate these restrictions, consider the below chunk of XML: ```xml - + {0} per day @@ -2879,7 +2902,7 @@ To illustrate these restrictions, consider the below chunk of XML: ``` -LDML is actually structured as below (from `en.xml`): +CLDR XML is actually structured as below (from `en.xml`): ```xml @@ -2904,7 +2927,10 @@ In particular, all elements allow for draft versions to coexist in the file at t There must be only one instance of these per parent, unless there are other distinguishing attributes (such as an `alt` element). -In general, LDML data should be in NFC format. Normalization forms are defined by [[UAX15](https://www.unicode.org/reports/tr41/#UAX15)]. However, certain elements may need to contain characters that are not in NFC, including exemplars, transforms, segmentations, and p/s/t/i/pc/sc/tc/ic rules in collation. These elements must not be normalized (either to NFC or NFD), or their meaning may be changed. Thus LDML documents must not be normalized as a whole. To prevent problems with normalization, no element value can start with a combining slash (U+0338 COMBINING LONG SOLIDUS OVERLAY). +In general, CLDR data should be in NFC format. Normalization forms are defined by [[UAX15](https://www.unicode.org/reports/tr41/#UAX15)]. +However, certain elements may need to contain characters that are not in NFC, including exemplars, transforms, segmentations, and p/s/t/i/pc/sc/tc/ic rules in collation. +These elements must not be normalized (either to NFC or NFD), or their meaning may be changed. +Thus CLDR documents must not be normalized as a whole. To prevent problems with normalization, no element value can start with a combining slash (U+0338 COMBINING LONG SOLIDUS OVERLAY). Lists, such as singleCountries are space-delimited. That means that they are separated by one or more XML whitespace characters: @@ -2941,7 +2967,10 @@ This element is designed to allow for arbitrary additional annotation and data t ##### Sample Special Elements -The elements in this section are _**not**_ part of the Locale Data Markup Language 1.0 specification. Instead, they are special elements used for application-specific data to be stored in the Common Locale Repository. They may change or be removed in future versions of this document, and are present here more as examples of how to extend the format. (Some of these items may move into a future version of the Locale Data Markup Language specification.) +The elements in this section are _**not**_ part of the Unicode Common Locale Data & Rules (CLDR) specification. +Instead, they are special elements used for application-specific data to be stored in CLDR's format. +They may change or be removed in future versions of this document, and are present here more as examples of how to extend the format. +(Some of these items may move into a future version of the CLDR specification.) * [https://www.unicode.org/cldr/dtd/1.1/ldmlICU.dtd](https://www.unicode.org/cldr/dtd/1.1/ldmlICU.dtd) * [https://www.unicode.org/cldr/dtd/1.1/ldmlOpenOffice.dtd](https://www.unicode.org/cldr/dtd/1.1/ldmlOpenOffice.dtd) @@ -3115,7 +3144,7 @@ Aliases must be resolved recursively. An alias may point to another path that re > xx-YY → xx // finds value here -It is an error to have a circular chain of aliases. That is, a collection of LDML XML documents must not have situations where a sequence of alias lookups (including inheritance and lateral inheritance) can be followed indefinitely without terminating. +It is an error to have a circular chain of aliases. That is, a collection of CLDR XML documents must not have situations where a sequence of alias lookups (including inheritance and lateral inheritance) can be followed indefinitely without terminating. #### Element displayName @@ -3258,7 +3287,7 @@ Note however that it may deviate from the syntax provided in [[UTS18](https://ww In particular, Section [RL2.6](https://www.unicode.org/reports/tr18/#RL2.6) _Wildcards in Property Values_ is not supported. However, that feature can be supported in clients such as ICU by implementing a “hook” as is done in the [online UnicodeSet utilities](https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5Cp%7Bname%3D%2FAPPLE%2F%7D). -A UnicodeSet may be cited in specifications outside of the domain of LDML. +A UnicodeSet may be cited in specifications outside of the domain of CLDR. In such a case, that specification may specify a subset or superset of the syntax provided here. ##### UnicodeSet syntax @@ -3565,7 +3594,7 @@ The [DTD Annotations](#DTD_Annotations) in are used to determine whether element ### Canonical Form -The following are restrictions on the format of LDML files to allow for easier parsing and comparison of files. +The following are restrictions on the format of CLDR XML files to allow for easier parsing and comparison of files. Peer elements have consistent order. That is, if the DTD or this specification requires the following order in an element `foo`: @@ -3591,7 +3620,7 @@ Note that there was one case that had to be corrected in order to make this true ``` -[XML](https://www.w3.org/TR/REC-xml/) files can have a wide variation in textual form, while representing precisely the same data. By putting the LDML files in the repository into a canonical form, this allows us to use the simple diff tools used widely (and in CVS) to detect differences when vetting changes, without those tools being confused. This is not a requirement on other uses of LDML; just simply a way to manage repository data more easily. +[XML](https://www.w3.org/TR/REC-xml/) files can have a wide variation in textual form, while representing precisely the same data. By putting the CLDR XML files in the repository into a canonical form, this allows us to use the simple diff tools used widely (and in CVS) to detect differences when vetting changes, without those tools being confused. This is not a requirement on other uses of CLDR XML; just simply a way to manage repository data more easily. #### Content @@ -3829,7 +3858,7 @@ Similarly, the subdivision translations represent a large body of data that may The [DTD Annotations](#DTD_Annotations) in are used to determine whether DTD items such as elements, attributes, or attribute values are deprecated. -Though such deprecated items are still valid LDML, they are strongly discouraged, and are no longer used in CLDR. +Though such deprecated items are still valid CLDR XML, they are strongly discouraged, and are no longer used in CLDR. The CLDR [DTD Deltas](https://www.unicode.org/cldr/charts/latest/supplemental/dtd_deltas.html) chart shows which DTD items have been deprecated in which version of CLDR. @@ -3881,7 +3910,7 @@ Instead use the basic collation syntax with the [`` element](tr35-collation. * The deprecated `` and `` are replaced by the `months` element with the context `type="format"` and the width `type="wide"` (for ...Names) and `type="narrow"` (for ...Abbr), respectively. * The deprecated `` and `` are replaced by the `days` element with the context `type="format"` and the width `type="wide"` (for ...Names) and `type="narrow"` (for ...Abbr), respectively. -* `` is deprecated in the main LDML files, because the data is more appropriately organized as connected to territories, not to linguistic data. Use the supplemental `` element instead. +* `` is deprecated in the main CLDR XML files, because the data is more appropriately organized as connected to territories, not to linguistic data. Use the supplemental `` element instead. * The standalone `` and `` are deprecated, and the data are instead included as part of the `` element * `` is deprecated as a subelement of `` instead, a `` element should be located just under a `` element. See [Calendar Fields](tr35-dates.md#Calendar_Fields). @@ -3936,7 +3965,7 @@ For the supplemental `` subelement ``, the `territory` a ## Annex B Links to Other Parts -The LDML specification is split into several [parts](#Parts) by topic, with one HTML document per part. The following tables provide redirects for links to specific topics. Please update your links and bookmarks. +The CLDR specification is split into several [parts](#Parts) by topic, with one HTML document per part. The following tables provide redirects for links to specific topics. Please update your links and bookmarks. Part 1 Links: Core (this document): No redirects needed. @@ -4279,7 +4308,7 @@ The above algorithm is a logical statement of the process, but would obviously n ## References -| Ancillary Information | To properly localize, parse, and format data requires ancillary information, which is not expressed in Locale Data Markup Language. Some of the formats for values used in Locale Data Markup Language are constructed according to external specifications. The sources for this data and/or formats include the following: | +| Ancillary Information | To properly localize, parse, and format data requires ancillary information, which is not expressed in CLDR. Some of the formats for values used in CLDR are constructed according to external specifications. The sources for this data and/or formats include the following: | | -------------------------------------------------------- | --- | | [Bugs] | CLDR Bug Reporting form
[https://cldr.unicode.org/index/bug-reports](https://cldr.unicode.org/index/bug-reports) | | [Charts] | The online code charts can be found at [https://www.unicode.org/charts/](https://www.unicode.org/charts/) An index to character names with links to the corresponding chart is found at [https://www.unicode.org/charts/charindex.html](https://www.unicode.org/charts/charindex.html) | @@ -4293,7 +4322,7 @@ The above algorithm is a logical statement of the process, but would obviously n | [Unicode] | The Unicode Consortium, _The Unicode Standard, Version 13.0.0_
(Mountain View, CA: The Unicode Consortium, 2020. ISBN 978-1-936213-26-9)
[https://www.unicode.org/versions/Unicode13.0.0/](https://www.unicode.org/versions/Unicode13.0.0/) | | [Versions] | Versions of the Unicode Standard
[https://www.unicode.org/versions/](https://www.unicode.org/versions/)
_For information on version numbering, and citing and referencing the Unicode Standard, the Unicode Character Database, and Unicode Technical Reports._ | | [XPath] | [https://www.w3.org/TR/xpath/](https://www.w3.org/TR/xpath/) | -| Other Standards | _Various standards define codes that are used as keys or values in Locale Data Markup Language. These include:_ | +| Other Standards | _Various standards define codes that are used as keys or values in CLDR. These include:_ | | [BCP47] | [https://www.rfc-editor.org/rfc/bcp/bcp47.txt](https://www.rfc-editor.org/rfc/bcp/bcp47.txt)
The Registry
[https://www.iana.org/assignments/language-subtag-registry](https://www.iana.org/assignments/language-subtag-registry) | | [ISO639] | ISO Language Codes
[https://www.loc.gov/standards/iso639-2/](https://www.loc.gov/standards/iso639-2/)
Actual List
[https://www.loc.gov/standards/iso639-2/langcodes.html](https://www.loc.gov/standards/iso639-2/langcodes.html) | | [ISO1000] | ISO 1000: SI units and recommendations for the use of their multiples and of certain other units, International Organization for Standardization, 1992.
[https://www.iso.org/iso/catalogue_detail?csnumber=5448](https://www.iso.org/iso/catalogue_detail?csnumber=5448) | @@ -4312,13 +4341,13 @@ The above algorithm is a logical statement of the process, but would obviously n | [Comparisons] | Comparisons between locale data from different sources
[https://www.unicode.org/cldr/charts/latest/by_type/index.html](https://www.unicode.org/cldr/charts/latest/by_type/index.html) | | [CurrencyInfo] | UNECE Currency Data
[https://www.iso.org/iso-4217-currency-codes.html](https://www.iso.org/iso-4217-currency-codes.html) | | [DataFormats] | CLDR Translation Guidelines
[https://cldr.unicode.org/translation](https://cldr.unicode.org/translation) | -| [Example] | A sample in Locale Data Markup Language
[https://www.unicode.org/cldr/dtd/1.1/ldml-example.xml](https://www.unicode.org/cldr/dtd/1.1/ldml-example.xml) | +| [Example] | A sample in CLDR
[https://www.unicode.org/cldr/dtd/1.1/ldml-example.xml](https://www.unicode.org/cldr/dtd/1.1/ldml-example.xml) | | [ICUCollation] | ICU rule syntax
[https://unicode-org.github.io/icu/userguide/collation/customization/](https://unicode-org.github.io/icu/userguide/collation/customization/) | | [ICUTransforms] | Transforms
[https://unicode-org.github.io/icu/userguide/transforms/](https://unicode-org.github.io/icu/userguide/transforms/)
Transforms Demo
[https://icu4c-demos.unicode.org/icu-bin/translit](https://icu4c-demos.unicode.org/icu-bin/translit) | | [ICUUnicodeSet] | ICU UnicodeSet
[https://unicode-org.github.io/icu/userguide/strings/unicodeset.html
](https://unicode-org.github.io/icu/userguide/strings/unicodeset.html)API
[https://unicode-org.github.io/icu-docs/apidoc/released/icu4j/com/ibm/icu/text/UnicodeSet.html](https://unicode-org.github.io/icu-docs/apidoc/released/icu4j/com/ibm/icu/text/UnicodeSet.html) | | [ITUE164] | International Telecommunication Union: List Of ITU Recommendation E.164 Assigned Country Codes
available at [https://www.itu.int/opb/publications.aspx?parent=T-SP&view=T-SP2](https://www.itu.int/opb/publications.aspx?parent=T-SP&view=T-SP2) | | [LocaleExplorer] | ICU Locale Explorer
[https://icu4c-demos.unicode.org/icu-bin/locexp](https://icu4c-demos.unicode.org/icu-bin/locexp) | -| [LocaleProject] | Common Locale Data Repository Project
[https://cldr.unicode.org](https://cldr.unicode.org) | +| [LocaleProject] | Common Locale Data & Rules Project
[https://cldr.unicode.org](https://cldr.unicode.org) | | [NamingGuideline] | OpenI18N Locale Naming Guideline
formerly at https://www.openi18n.org/docs/text/LocNameGuide-V10.txt | | [RBNF] | Rule-Based Number Format
[https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/classicu_1_1RuleBasedNumberFormat.html](https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/classicu_1_1RuleBasedNumberFormat.html) | | [RBBI] | Rule-Based Break Iterator
[https://unicode-org.github.io/icu/userguide/boundaryanalysis/](https://unicode-org.github.io/icu/userguide/boundaryanalysis/) | diff --git a/docs/site/cldr-tc.md b/docs/site/cldr-tc.md index 6795b53708a..ff98140678d 100644 --- a/docs/site/cldr-tc.md +++ b/docs/site/cldr-tc.md @@ -4,11 +4,11 @@ title: "CLDR Technical Committee (TC)" # CLDR Technical Committee (TC) -The CLDR Technical Committee is responsible for the Unicode Common Locale Data Repository data repository, +The CLDR Technical Committee is responsible for the Unicode Common Locale Data & Rules data repository, which encompasses: * the data repository itself, in XML format -* the specification for the data and structure [UTS 35 (LDML)][] +* the specification for the data and structure [UTS 35][] * the tooling used to gather locale data, test the data and structure, and provide test data for implementations * additional formats for the locale data (e.g., JSON) @@ -28,7 +28,7 @@ More detailed information about the procedures related to working groups are ava ### CLDR Conformance Testing Working Group -The [CLDR Conformance Testing WG][] is tasked with providing code and test data to verify consistency between implementations and standards using [UTS 35 (LDML)][] (the CLDR specification). +The [CLDR Conformance Testing WG][] is tasked with providing code and test data to verify consistency between implementations and standards using [UTS 35][] (the CLDR specification). Those implementations and standards include [ICU4C, ICU4J][], [ICU4X][], [ECMA-402][], and others. In addition to the development and maintenance of the web-based conformance scorecard, @@ -64,7 +64,7 @@ The WG also manages the process of validating new keyboards contained in the CLD ### CLDR MessageFormat Working Group The [CLDR MessageFormat WG][] was tasked with developing the specification for Message Format 2 (MF2) and ensuring implementations of MF2 in the ICU and ICU4X libraries. -These recommendations, once approved, are incorporated into the CLDR data and tooling, and the LDML specification [UTS 35 Part 9: MessageFormat][]. +These recommendations, once approved, are incorporated into the CLDR data, tooling, and specification [UTS 35 Part 9: MessageFormat][]. The main page for the Message Format WG is in the [MessageFormat Repository][]. @@ -83,7 +83,7 @@ but was renamed to disambiguate from the Unicode Infrastructure group since both ### CLDR Person Name Working Group The [Person Name WG][] is responsible for the development of recommendations for person name formatting structure and data. -These recommendations, once approved, are incorporated into the CLDR data and tooling, and the LDML specification [UTS 35 Part 8: Person Names][]. +These recommendations, once approved, are incorporated into the CLDR data, tooling, and specification [UTS 35 Part 8: Person Names][]. ### DDL Working Group @@ -119,4 +119,4 @@ including but not limited to evaluating requests of [new locale Core data reques [CLDR Conformance Testing WG]: /cldr-tc/conformance_wg [Unicode Technical Group Leadership]: https://www.unicode.org/consortium/techcommittees.html [Unicode® Technical Group Procedures]: https://www.unicode.org/consortium/tc-procedures.html -[UTS 35 (LDML)]: https://www.unicode.org/reports/tr35/ +[UTS 35]: https://www.unicode.org/reports/tr35/ diff --git a/docs/site/cldr-tc/person-name-wg.md b/docs/site/cldr-tc/person-name-wg.md index 2101418f7aa..53970094173 100644 --- a/docs/site/cldr-tc/person-name-wg.md +++ b/docs/site/cldr-tc/person-name-wg.md @@ -5,5 +5,5 @@ title: CLDR Person Name Working Group # CLDR Person Name Working Group The Person Name WG is responsible for the development of recommendations for person name formating structure and data. -These recommendations, once approved, are incorporated into the CLDR data and tooling, and the LDML specification [Part 8: Person Names](https://www.unicode.org/reports/tr35/tr35-personNames.html#contents-of-part-8-person-names). +These recommendations, once approved, are incorporated into the CLDR data and tooling, and the CLDR specification [Part 8: Person Names](https://www.unicode.org/reports/tr35/tr35-personNames.html#contents-of-part-8-person-names). There is a development version at [Part 8: Person Names (development)](https://www.unicode.org/reports/tr35/dev/tr35-personNames.html#contents-of-part-8-person-names). diff --git a/docs/site/ddl.md b/docs/site/ddl.md index 642f4244b65..67507fc2adc 100644 --- a/docs/site/ddl.md +++ b/docs/site/ddl.md @@ -4,7 +4,7 @@ title: 'DDL Working Group' Need help with your locale? Please visit our new help page: [DDL Help](/translation/ddl) -The Common Locale Data Repository (CLDR) is [widely used](/index), and the content has grown dramatically over the years with participation by organizations of all types and sizes, as well as many individual contributors. +The Common Locale Data & Rules (CLDR) is [widely used](/index), and the content has grown dramatically over the years with participation by organizations of all types and sizes, as well as many individual contributors. Contributors for _Digitally Disadvantaged Languages_ (DDL) face unique challenges. The CLDR-DDL Working Group has been formed to evaluate mechanisms to make it easier for contributors for DDLs to: diff --git a/docs/site/development/new-cldr-developers.md b/docs/site/development/new-cldr-developers.md index cc7c2b38e7c..a88ad2c491c 100644 --- a/docs/site/development/new-cldr-developers.md +++ b/docs/site/development/new-cldr-developers.md @@ -58,5 +58,5 @@ The table below points to documentation for various tasks. Other useful pages are under [CLDR Development Site](/development/cldr-development-site); you can also use the search box. -[UTS #35: Unicode Locale Data Markup Language (LDML)](https://www.unicode.org/reports/tr35/) is the specification of the XML format used for CLDR data, including the interpretation of the CLDR data. +[UTS #35: Unicode Common Locale Data & Rules (CLDR)](https://www.unicode.org/reports/tr35/) is the specification for CLDR data and conformant usage. diff --git a/docs/site/downloads/cldr-49.md b/docs/site/downloads/cldr-49.md index bc3783f2f95..6fcae6a065c 100644 --- a/docs/site/downloads/cldr-49.md +++ b/docs/site/downloads/cldr-49.md @@ -6,7 +6,7 @@ title: CLDR 49 Release Note | No. | Date | Rel. Note | Data | Charts | Spec | Delta | GitHub Tag | Delta DTD | CLDR JSON | |:---:|:----------:|:---------:|:------:|:--------:|:------------:|:---:|:----------:|:---------:|:---------:| -| 49 | 2026-10-~~XX~~ | [v49](/index/downloads/cldr-49) | [CLDR49](https://unicode.org/Public/cldr/49/) | [Charts49](https://unicode.org/cldr/charts/dev) | [LDML49](https://www.unicode.org/reports/tr35/49/tr35.html) | [Δ49](https://unicode-org.atlassian.net/issues/?jql=project%20%3D%20CLDR%20AND%20status%20%3D%20Done%20AND%20resolution%20%3D%20Fixed%20AND%20fixversion%20%3D%2049%20ORDER%20BY%20priority%20DESC) | [release-49-beta2](https://github.com/unicode-org/cldr/releases/tag/release-49-beta2) | [ΔDtd49](https://www.unicode.org/cldr/charts/dev/supplemental/dtd_deltas.html) | [49.0.0-BETA1](https://github.com/unicode-org/cldr-json/releases/tag/49.0.0-BETA1) | +| 49 | 2026-10-~~XX~~ | [v49](/index/downloads/cldr-49) | [CLDR49](https://unicode.org/Public/cldr/49/) | [Charts49](https://unicode.org/cldr/charts/dev) | [CLDR49](https://www.unicode.org/reports/tr35/49/tr35.html) | [Δ49](https://unicode-org.atlassian.net/issues/?jql=project%20%3D%20CLDR%20AND%20status%20%3D%20Done%20AND%20resolution%20%3D%20Fixed%20AND%20fixversion%20%3D%2049%20ORDER%20BY%20priority%20DESC) | [release-49-beta2](https://github.com/unicode-org/cldr/releases/tag/release-49-beta2) | [ΔDtd49](https://www.unicode.org/cldr/charts/dev/supplemental/dtd_deltas.html) | [49.0.0-BETA1](https://github.com/unicode-org/cldr-json/releases/tag/49.0.0-BETA1) | ## Overview @@ -74,8 +74,12 @@ For a full listing, see [Coverage Levels](https://unicode.org/cldr/charts/dev/su ## Specification Changes -The following are the most significant changes to the specification (LDML), aside from those covered under DTD changes below. +The following are the most significant changes to the CLDR specification, aside from those covered under DTD changes below. +- Replaced “Common Locale Data Repository (CLDR)” by “Unicode Common Locale Data & Rules (CLDR)”, +and the title of the specification “UTS #35: Unicode Locale Data Markup Language (LDML)” by “UTS #35: Unicode Common Locale Data & Rules (CLDR)”. + - The former names and the acronym “LDML” led to many misunderstandings and have been changed for clarity. + - Note that all references under the former names and the LDML acronym remain valid; such references on the Unicode site will be updated over time. - Clarified the process of selecting the best `dateFormatItem` when there is no exact match, and how to use `appendItems` to add missing fields. - This includes a clarification of what are date fields and what are time fields, and a note that `appendItems` for date and time fields should be appended before combining them. - In the Key/Type Description table, added a description which key/types use constructed values and a brief description of the typeValue element. @@ -264,7 +268,7 @@ See [UTS #61][]: Unicode Set Notation (currently in draft). ## Acknowledgments -Many people have made significant contributions to CLDR and LDML; +Many people have made significant contributions to CLDR; see the [Acknowledgments](/index/acknowledgments) page for a full listing. The Unicode [Terms of Use](https://unicode.org/copyright.html) apply to CLDR data; diff --git a/docs/site/index.md b/docs/site/index.md index 69c0c7aa77d..82b4b93a784 100644 --- a/docs/site/index.md +++ b/docs/site/index.md @@ -17,7 +17,7 @@ _To build and maintain the most trusted and comprehensive repository of [locale ## What is CLDR? -CLDR (Common Locale Data Repository) supplies key information and structures critical for programs and operating systems around the world to ensure that they feel natural, +CLDR (Common Locale Data & Rules) supplies key information and structures critical for programs and operating systems around the world to ensure that they feel natural, no matter which language users speak or where they live. For example, imagine looking at a list of files on your mobile phone. @@ -30,7 +30,7 @@ Over 100 languages are supported, with more added each release. CLDR consists of three main components: 1. A curated collection of structured data used by implementations -1. A specification, [UTS #35: Unicode Locale Data Markup Language (LDML)][], documenting the structure and usage of that data (via defined algorithms), including conformance requirements and guidelines +1. A specification, [UTS #35: Unicode Common Locale Data & Rules (CLDR)][], documenting the structure and usage of that data (via defined algorithms), including conformance requirements and guidelines 1. Code used to collect that data from language specialists, guide those specialists in supplying the data, verify the validity and consistency, and process it into different formats for use by software developers Formatting dates, numbers, currencies, and units of measurement is far more complicated across different languages and regions than most people recognize. @@ -64,7 +64,7 @@ There are other projects which consume [cldr-json] directly, see [here][cldr-jso Most developers will use CLDR indirectly, via a set of software libraries, such as [ICU](https://icu.unicode.org/), [Closure](https://github.com/google/closure-library), or [TwitterCLDR](https://blog.x.com/engineering/en_us/a/2012/twittercldr-improving-internationalization-support-in-ruby). These libraries typically compile the CLDR data into a format that is compact and easy for the library to load and use. -For those interested in the source CLDR data, it is available for each release in the XML format specified by [UTS #35: Unicode Locale Data Markup Language (LDML)][]. There are also tools that will convert to JSON and POSIX format. For more information, see [CLDR Releases/Downloads](index/downloads). +For those interested in the source CLDR data, it is available for each release in the XML format specified by [UTS #35: Unicode Common Locale Data & Rules (CLDR)][]. There are also tools that will convert to JSON and POSIX format. For more information, see [CLDR Releases/Downloads](index/downloads). ## How to Contribute? @@ -91,7 +91,7 @@ People may file [tickets](requesting_changes) with bug fixes or feature requests ## Who has contributed? -Many people have made significant contributions to CLDR and LDML; see the [Acknowledgments](index/acknowledgments) page for a full listing. +Many people have made significant contributions to CLDR; see the [Acknowledgments](index/acknowledgments) page for a full listing. ## What is the Schedule? @@ -108,4 +108,4 @@ The details for the current release are found in [Current CLDR Cycle](https://do [cldr-json]: /index/json-format-data [cldr-json-users]: https://github.com/unicode-org/cldr-json/blob/master/USERS.md#projects [locale data]: /index/cldr-spec/definitions#locale-data -[UTS #35: Unicode Locale Data Markup Language (LDML)]: https://www.unicode.org/reports/tr35/ +[UTS #35: Unicode Common Locale Data & Rules (CLDR)]: https://www.unicode.org/reports/tr35/ diff --git a/docs/site/index/acknowledgments.md b/docs/site/index/acknowledgments.md index 925e0f988d4..ba20a921bd0 100644 --- a/docs/site/index/acknowledgments.md +++ b/docs/site/index/acknowledgments.md @@ -4,9 +4,9 @@ title: Acknowledgments # Acknowledgments -Acknowledgments for special contributions to successive versions of CLDR and LDML are in the [Acknowledgments](http://www.unicode.org/reports/tr35/#Acknowledgments) section of [*UTS #35: Locale Data Markup Language (LDML)*](http://www.unicode.org/reports/tr35/). +Acknowledgments for special contributions to successive versions of CLDR are in the [Acknowledgments](http://www.unicode.org/reports/tr35/#Acknowledgments) section of [*UTS #35: Unicode Common Locale Data & Rules (CLDR)*](http://www.unicode.org/reports/tr35/). -Many other people have made significant contributions to CLDR and LDML, including the following: +Many other people have made significant contributions to CLDR, including the following: - A S Alam (ਅਮਨਪਰੀਤ ਸਿੰਘ ਆਲਮ) - Aamir Raz Soomro (عامر راز سومرو) diff --git a/docs/site/index/bcp47-extension.md b/docs/site/index/bcp47-extension.md index ca83d654500..1139e282c6f 100644 --- a/docs/site/index/bcp47-extension.md +++ b/docs/site/index/bcp47-extension.md @@ -10,11 +10,11 @@ title: Unicode Extensions for BCP 47 - The subtags available for use in the 't' extension provide language tag extensions that provide for additional information needed for identifying transformed content, or a request to transform content in a certain way. For example, the language tag "ja-Kana-t-it" can be used as a content tag indicates Japanese Katakana transformed from Italian. It can also be used as a request for a given transformation. -For more details on the valid subtags for these extensions, their syntax, and their meanings, see LDML Section 3.7 [*Unicode BCP 47 Extension Data*](https://www.unicode.org/reports/tr35/#Locale_Extension_Key_and_Type_Data). +For more details on the valid subtags for these extensions, their syntax, and their meanings, see UTS #35, Part 1, Section 3.7 [*Unicode BCP 47 Extension Data*](https://www.unicode.org/reports/tr35/#Locale_Extension_Key_and_Type_Data). ## Machine-Readable Files for Validity Testing -Beginning with CLDR version 1.7.2, machine-readable files are available listing the valid attributes, keys, and types for each successive version of [LDML](https://unicode.org/reports/tr35/). The most recently released version is always available at http://unicode.org/Public/cldr/latest/ in a file of the form cldr-common\*.zip (in older versions the file was of the form cldr-core\*.zip). Inside that file, the directory "common/bcp47/" contains the data files defining the valid attributes, keys, and types. +Beginning with CLDR version 1.7.2, machine-readable files are available listing the valid attributes, keys, and types for each successive version of [UTS #35](https://unicode.org/reports/tr35/). The most recently released version is always available at http://unicode.org/Public/cldr/latest/ in a file of the form cldr-common\*.zip (in older versions the file was of the form cldr-core\*.zip). Inside that file, the directory "common/bcp47/" contains the data files defining the valid attributes, keys, and types. The BCP47 data is also currently maintained in a source code repository, with each release tagged, for viewing directly without unzipping. For example, see https://github.com/unicode-org/cldr/tree/release-38/common/bcp47. The current development snapshot is found at https://github.com/unicode-org/cldr/tree/master/common/bcp47. @@ -30,7 +30,7 @@ For example, the timezone.xml file looks like the following: \ -Using this data, an implementation would determine that "fr-u-tz-adalv" and fr-u-tz-aedxb" are both valid. Some data in the CLDR data files also requires reference to [LDML](https://unicode.org/reports/tr35/) for validation according to [Appendix Q](https://unicode.org/reports/tr35/#Locale_Extension_Key_and_Type_Data) of [LDML](https://unicode.org/reports/tr35/). For example, LDML defines the type 'codepoints' to define specific code point ranges in Unicode for specific purposes. +Using this data, an implementation would determine that "fr-u-tz-adalv" and fr-u-tz-aedxb" are both valid. Some data in the CLDR data files also requires reference to [UTS #35](https://unicode.org/reports/tr35/) for validation according to [Appendix Q](https://unicode.org/reports/tr35/#Locale_Extension_Key_and_Type_Data) of [UTS #35](https://unicode.org/reports/tr35/). For example, UTS #35 defines the type 'codepoints' to define specific code point ranges in Unicode for specific purposes. ## Version Information diff --git a/docs/site/index/charts.md b/docs/site/index/charts.md index 689a6a538ae..d6ef9f8d73c 100644 --- a/docs/site/index/charts.md +++ b/docs/site/index/charts.md @@ -4,13 +4,13 @@ title: CLDR Charts # CLDR Charts -The Unicode CLDR Charts provide different ways to view the Common Locale Data Repository data. +The Unicode CLDR Charts provide different ways to view the Common Locale Data & Rules data. - [Latest](https://www.unicode.org/cldr/charts/latest) - The charts for the latest release version - [Dev](https://www.unicode.org/cldr/charts/dev) - A snapshot of data under development - [Previous](/index/downloads) - Previous available charts are linked from the download page in the Charts column -The format of most of the fields in the charts will be clear from the Name and ID, such as the months of the year. The format for others, such as the date or time formats, is structured and requires more interpretation. For more information, see [UTS #35: Locale Data Markup Language (LDML)](http://www.unicode.org/reports/tr35/). +The format of most of the fields in the charts will be clear from the Name and ID, such as the months of the year. The format for others, such as the date or time formats, is structured and requires more interpretation. For more information, see [UTS #35: Unicode Common Locale Data & Rules (CLDR)](http://www.unicode.org/reports/tr35/). Most charts have "double links" somewhere in each row. These are links that put the address of that row into the address bar of the browser for copying. diff --git a/docs/site/index/cldr-spec.md b/docs/site/index/cldr-spec.md index ed81a87ffa1..9b94615a155 100644 --- a/docs/site/index/cldr-spec.md +++ b/docs/site/index/cldr-spec.md @@ -4,7 +4,7 @@ title: CLDR Specifications # CLDR Specifications -- [UTS #35: Unicode Locale Data Markup Language (LDML)](https://www.unicode.org/reports/tr35/) is the latest released specification of the CLDR XML format and algorithms, including the interpretation of the CLDR data. +- [UTS #35: Unicode Common Locale Data & Rules (CLDR)](https://www.unicode.org/reports/tr35/) is the latest released specification of the CLDR XML format and algorithms, including the interpretation of the CLDR data. - [**Development version**](https://www.unicode.org/reports/tr35/proposed.html) - [Updating the Spec](/index/cldr-spec/updating-spec) - [Definitions](/index/cldr-spec/definitions) - common definitions used in CLDR diff --git a/docs/site/index/cldr-spec/cldr-json-bindings.md b/docs/site/index/cldr-spec/cldr-json-bindings.md index ddf32de6e53..192b2738db3 100644 --- a/docs/site/index/cldr-spec/cldr-json-bindings.md +++ b/docs/site/index/cldr-spec/cldr-json-bindings.md @@ -26,7 +26,7 @@ The sample is based on the preliminary specification and CLDR version 22.1. ## Introduction -Unicode Technical Standard \#35 describes an XML format for the exchange of structured locale data named Unicode Locale Data Markup Language (LDML). Data gathered and vetted through the Common Locale Data Repository (CLDR) project is stored in LDML format. +Unicode Technical Standard \#35 specifies the data format and usage of Common Locale Data & Rules (CLDR) data. This data is used for many purposes. However, distribution of it tends to be unwieldy for various reasons. For many potential users, the only alternative is to use an internationalization library, such as ICU. Such library might not exist in users’ platform, or too much a burden due to its size and performance requirement, or just too much overhead for a seemingly simple task. The rapidly growing area of web applications needs a way to access i18n data in a simple way. diff --git a/docs/site/index/cldr-spec/coverage-levels.md b/docs/site/index/cldr-spec/coverage-levels.md index 6c5fa1bcd03..587691d074e 100644 --- a/docs/site/index/cldr-spec/coverage-levels.md +++ b/docs/site/index/cldr-spec/coverage-levels.md @@ -4,7 +4,7 @@ title: Coverage Levels # Coverage Levels -There are four main coverage levels as defined in the [UTS \#35: Unicode Locale Data Markup Language (LDML) Part 6: Supplemental: 8 Coverage Levels](https://www.unicode.org/reports/tr35/tr35-info.html#Coverage_Levels). They are described more fully below. +There are four main coverage levels as defined in the [UTS \#35: Unicode Common Locale Data & Rules (CLDR) Part 6: Supplemental: 8 Coverage Levels](https://www.unicode.org/reports/tr35/tr35-info.html#Coverage_Levels). They are described more fully below. ## Usage diff --git a/docs/site/index/cldr-spec/picking-the-right-language-code.md b/docs/site/index/cldr-spec/picking-the-right-language-code.md index b46848c2c1a..969ce028b80 100644 --- a/docs/site/index/cldr-spec/picking-the-right-language-code.md +++ b/docs/site/index/cldr-spec/picking-the-right-language-code.md @@ -1,102 +1,104 @@ ---- -title: Picking the Right Language Identifier ---- - -# Picking the Right Language Identifier - -Within programs and structured data, languages are indicated with stable identifiers of the form '[en](http://unicode.org/cldr/utility/languageid.jsp?a=en)', '[fr\-CA](http://unicode.org/cldr/utility/languageid.jsp?a=fr-CA)', or '[zh\-Hant](http://unicode.org/cldr/utility/languageid.jsp?a=zh-Hant&l=en)'. The standard Unicode language identifiers follow IETF BCP 47, with some small differences defined in [UTS \#35: Locale Data Markup Language (LDML)](http://www.unicode.org/reports/tr35/). Locale identifiers use the same format, with certain possible extensions. - -Often it is not clear which language identifier to use. For example, what most people call Punjabi in Pakistan actually has the code '[lah](http://unicode.org/cldr/utility/languageid.jsp?a=lah)', and formal name "Lahnda". There are many other cases where the same name is used for different languages, or where the name that people search for is not listed in the IANA registry. Moreover, a language identifier uses not only the 'base' language code, like '[en](http://unicode.org/cldr/utility/languageid.jsp?a=en)' for English or '[ku](http://unicode.org/cldr/utility/languageid.jsp?a=ku)' for Kurdish, but also certain modifiers such as '[en\-CA](http://unicode.org/cldr/utility/languageid.jsp?a=en-CA)' for *Canadian English*, or '[ku\-Latn](http://ku-Latn)' for *Kurdish written in Latin script*. Each of these modifiers are called *subtags* (or sometimes *codes*), and are separated by "\-" or "\_". The language identifier itself is also called a *language tag*, and sometimes a *language code*. - -Here is an example of the steps to take to find the right language identifier to use. Let's say you want to find the identifier for a language called "Ganda" which you know is spoken in Uganda. You'll first pick the base language subtag as described below, then add any necessary script/territory subtags, and then verify. If you can't find the name after following these steps or have other questions, ask on the [Unicode CLDR Mailing List](http://www.unicode.org/consortium/distlist.html#cldr_list). - -If you are looking at a prospective language code, like 'swh', the process is similar; follow the steps below, starting with the verification. - -## Choosing the Base Language Code - -1. Go to [iso639\-3](https://iso639-3.sil.org/code_tables/639/data) to find the language. In the "Language Name(s) containing" field enter the language name (**Ganda** in this example) and select "Apply". -2. There may be multiple entries for the item you want, so you'll need to examine them carefully. -3. You'll find an entry like: - - lug  lug  **lg**  Ganda  Individual  Living - -While you may think that you are done, you have to verify that the three\-letter code is correct. - -1. Click on the item in the first column ('[lug](https://iso639-3.sil.org/code/lug)' in this case) and you'll find links to Ethnologue, Glottolog and Wikipedia. -2. Click on the [Ethnologue](http://www.ethnologue.com/language/lug) link and you get to the corresponding Ethnologue entry. -3. Verify that is indeed the language: - 1. Look at the information on the Ethnologue page - 2. Check Glottolog, Wikipedia and other web sources -4. ***AND IMPORTANTLY: Review "[Caution](#caution)" section below*** - -Once you have the right three\-letter code, you are still not done. Unicode and BCP 47 use the two\-letter ISO code if it exists. Unicode also uses the "macro language" where suitable. *So:* - - -1. Use the two\-letter code if there is one. In the example above, use the highlighted 'lg' from the first table. -2. Verify that the code is in the [IANA subtag registry](http://www.iana.org/assignments/language-subtag-registry) -3. If the code occurs in the [supplementalMetadata.xml](http://github.com/unicode-org/cldr/blob/main/common/supplemental/supplementalMetadata.xml) file in the type attribute of a languageAlias element, then use the replacement instead. - - For example, because 'swh' occurs in ``, 'sw' must be used instead of 'swh'. - - -## Choosing Script/Territory Subtags - -If you need a particular variant of a language, then you'll add additional subtags, typically script or territory. Consult [Sample Subtags](http://unicode.org/cldr/utility/sample_subtags.html) for the most common choices. ***Again, review "[Caution](#caution)" section below.*** - -## Verifying Your Choice - -1. Verify your choice by using the [online language identifier](http://unicode.org/cldr/utility/languageid.jsp) demo. -2. You need to fix the identifier and try again in *any* if the demo shows any of the following: - 1. the language identifer is illegal, or - 2. one of the subtags is invalid, or - 3. there are any replacement values. - -## Documenting Your Choice - -If you are requesting a new locale / language in CLDR, please include the links to the particular pages above so that we can process your request more quickly, as we have to double-check the information before making the addition. The links will be of the form: - - -- https://iso639-3.sil.org/code/xxx -- https://www.ethnologue.com/language/xxx -- https://en.wikipedia.org/wiki/xxx -- and so on - -## Caution - -### Canonical Form - -Unicode language and locale IDs are based on BCP 47, but differ in a few ways. The canonical form is produced by using the canonicalization based on BCP 47 (thus changing 'iw' → 'he', and 'zh\-yue' → 'yue'), plus a few other steps: - -1. Replacing the most prominent encompassed subtag by the macrolanguage ('cmn' → 'zh') -2. Canonicalizing overlong 3 letter codes ('eng\-840' → 'en\-US') -3. Minimizing according to the likely subtag data ('ru\-Cyrl' → 'ru', 'en\-US' → 'en'). -4. BCP 47 also provides for "variant subtags", such as '[zh\-Latn\-pinyin](http://unicode.org/cldr/utility/languageid.jsp?a=zh-Latn-pinyin)'. When there are multiple variant subtags, the canonical format for Unicode language identifiers puts them in alphabetical order. - -Note that the CLDR likely subtag data is used to minimize scripts and regions, *not* the IANA Suppress\-Script. The latter had a much more constrained design goal and is more limited. - - -In some cases, systems (or companies) may have different conventions than the Preferred\-Values in BCP 47 \-\- such as those in the Replacement column in the the [online language identifier](http://unicode.org/cldr/utility/languageid.jsp) demo. For example, for backwards compatibility, 'iw' is used with Java instead of 'he' (Hebrew). Be aware of these compatibility issues when picking the subtags. *If a target system uses a different canonical form for locale IDs than CLDR, the CLDR data needs to be processed by remapping its IDs to those of the target system.* - -For compatibility, it is strongly recommended that all implementations accept both the preferred values and their alternates: for example, both 'iw' and 'he'. Although BCP 47 itself only allows "\-" as a separator, Unicode language identifiers allow both "\-" and "\_" for compatibility reasons. Implementations should also accept both. - -### Macrolanguages - -ISO (and hence BCP 47\) has the notion of an individual language (like 'en' \= English) versus a Collection or Macrolanguage. For compatibility, Unicode language and locale identifiers always use the Macrolanguage to identify the predominant form. Thus the Macrolanguage subtag 'zh' (Chinese) is used instead of 'cmn' (Mandarin). Similarly, suppose that you are looking for Kurdish written in Latin letters, as in Turkey. It is a mistake to think that because that is in the north, that you should use the subtag 'kmr' for Northern Kurdish. You should instead use 'ku\-Latn\-TR'. See also: [ISO 639 Deprecation Requests](/development/development-process/design-proposals/iso-639-deprecation-requests-draft). - -Unicode language identifiers do not allow the "extlang" form defined in BCP 47\. For example, use 'yue' instead of 'zh\-yue' for Cantonese. - -### Ethnologue - -The Ethnologue is a great source of information, but it must be approached with a certain degree of caution. Many of the population figures are out of date or not well substantiated. The Ethnologue also focus on native, spoken languages, whereas CLDR and many other systems are focused on written language, for computer UI and document translation, and on fluent speakers (not necessarily native speakers). So, for example, it would be a mistake to look at the [Ethnologue entry for Egypt](https://www.ethnologue.com/country/EG/) and conclude that the right language subtag for the Arabic used in Egypt is 'arz', which has the largest population. Instead, the right code is 'ar', Standard Arabic, which would be the one used for document and UI translation. - - -### Wikipedia - -Wikipedia is also a great source of information, but it must be approached with a certain degree of caution as well. Be sure to follow up on the linked references, not just look at articles. - - - - - - - - +--- +title: Picking the Right Language Identifier +--- + +# Picking the Right Language Identifier + +Within programs and structured data, languages are indicated with stable identifiers of the form '[en](http://unicode.org/cldr/utility/languageid.jsp?a=en)', '[fr\-CA](http://unicode.org/cldr/utility/languageid.jsp?a=fr-CA)', or '[zh\-Hant](http://unicode.org/cldr/utility/languageid.jsp?a=zh-Hant&l=en)'. +The standard Unicode language identifiers follow IETF BCP 47, with some small differences defined in [UTS \#35: Unicode Common Locale Data & Rules (CLDR)](http://www.unicode.org/reports/tr35/). +Locale identifiers use the same format, with certain possible extensions. + +Often it is not clear which language identifier to use. For example, what most people call Punjabi in Pakistan actually has the code '[lah](http://unicode.org/cldr/utility/languageid.jsp?a=lah)', and formal name "Lahnda". There are many other cases where the same name is used for different languages, or where the name that people search for is not listed in the IANA registry. Moreover, a language identifier uses not only the 'base' language code, like '[en](http://unicode.org/cldr/utility/languageid.jsp?a=en)' for English or '[ku](http://unicode.org/cldr/utility/languageid.jsp?a=ku)' for Kurdish, but also certain modifiers such as '[en\-CA](http://unicode.org/cldr/utility/languageid.jsp?a=en-CA)' for *Canadian English*, or '[ku\-Latn](http://ku-Latn)' for *Kurdish written in Latin script*. Each of these modifiers are called *subtags* (or sometimes *codes*), and are separated by "\-" or "\_". The language identifier itself is also called a *language tag*, and sometimes a *language code*. + +Here is an example of the steps to take to find the right language identifier to use. Let's say you want to find the identifier for a language called "Ganda" which you know is spoken in Uganda. You'll first pick the base language subtag as described below, then add any necessary script/territory subtags, and then verify. If you can't find the name after following these steps or have other questions, ask on the [Unicode CLDR Mailing List](http://www.unicode.org/consortium/distlist.html#cldr_list). + +If you are looking at a prospective language code, like 'swh', the process is similar; follow the steps below, starting with the verification. + +## Choosing the Base Language Code + +1. Go to [iso639\-3](https://iso639-3.sil.org/code_tables/639/data) to find the language. In the "Language Name(s) containing" field enter the language name (**Ganda** in this example) and select "Apply". +2. There may be multiple entries for the item you want, so you'll need to examine them carefully. +3. You'll find an entry like: + + lug  lug  **lg**  Ganda  Individual  Living + +While you may think that you are done, you have to verify that the three\-letter code is correct. + +1. Click on the item in the first column ('[lug](https://iso639-3.sil.org/code/lug)' in this case) and you'll find links to Ethnologue, Glottolog and Wikipedia. +2. Click on the [Ethnologue](http://www.ethnologue.com/language/lug) link and you get to the corresponding Ethnologue entry. +3. Verify that is indeed the language: + 1. Look at the information on the Ethnologue page + 2. Check Glottolog, Wikipedia and other web sources +4. ***AND IMPORTANTLY: Review "[Caution](#caution)" section below*** + +Once you have the right three\-letter code, you are still not done. Unicode and BCP 47 use the two\-letter ISO code if it exists. Unicode also uses the "macro language" where suitable. *So:* + + +1. Use the two\-letter code if there is one. In the example above, use the highlighted 'lg' from the first table. +2. Verify that the code is in the [IANA subtag registry](http://www.iana.org/assignments/language-subtag-registry) +3. If the code occurs in the [supplementalMetadata.xml](http://github.com/unicode-org/cldr/blob/main/common/supplemental/supplementalMetadata.xml) file in the type attribute of a languageAlias element, then use the replacement instead. + - For example, because 'swh' occurs in ``, 'sw' must be used instead of 'swh'. + + +## Choosing Script/Territory Subtags + +If you need a particular variant of a language, then you'll add additional subtags, typically script or territory. Consult [Sample Subtags](http://unicode.org/cldr/utility/sample_subtags.html) for the most common choices. ***Again, review "[Caution](#caution)" section below.*** + +## Verifying Your Choice + +1. Verify your choice by using the [online language identifier](http://unicode.org/cldr/utility/languageid.jsp) demo. +2. You need to fix the identifier and try again in *any* if the demo shows any of the following: + 1. the language identifer is illegal, or + 2. one of the subtags is invalid, or + 3. there are any replacement values. + +## Documenting Your Choice + +If you are requesting a new locale / language in CLDR, please include the links to the particular pages above so that we can process your request more quickly, as we have to double-check the information before making the addition. The links will be of the form: + + +- https://iso639-3.sil.org/code/xxx +- https://www.ethnologue.com/language/xxx +- https://en.wikipedia.org/wiki/xxx +- and so on + +## Caution + +### Canonical Form + +Unicode language and locale IDs are based on BCP 47, but differ in a few ways. The canonical form is produced by using the canonicalization based on BCP 47 (thus changing 'iw' → 'he', and 'zh\-yue' → 'yue'), plus a few other steps: + +1. Replacing the most prominent encompassed subtag by the macrolanguage ('cmn' → 'zh') +2. Canonicalizing overlong 3 letter codes ('eng\-840' → 'en\-US') +3. Minimizing according to the likely subtag data ('ru\-Cyrl' → 'ru', 'en\-US' → 'en'). +4. BCP 47 also provides for "variant subtags", such as '[zh\-Latn\-pinyin](http://unicode.org/cldr/utility/languageid.jsp?a=zh-Latn-pinyin)'. When there are multiple variant subtags, the canonical format for Unicode language identifiers puts them in alphabetical order. + +Note that the CLDR likely subtag data is used to minimize scripts and regions, *not* the IANA Suppress\-Script. The latter had a much more constrained design goal and is more limited. + + +In some cases, systems (or companies) may have different conventions than the Preferred\-Values in BCP 47 \-\- such as those in the Replacement column in the the [online language identifier](http://unicode.org/cldr/utility/languageid.jsp) demo. For example, for backwards compatibility, 'iw' is used with Java instead of 'he' (Hebrew). Be aware of these compatibility issues when picking the subtags. *If a target system uses a different canonical form for locale IDs than CLDR, the CLDR data needs to be processed by remapping its IDs to those of the target system.* + +For compatibility, it is strongly recommended that all implementations accept both the preferred values and their alternates: for example, both 'iw' and 'he'. Although BCP 47 itself only allows "\-" as a separator, Unicode language identifiers allow both "\-" and "\_" for compatibility reasons. Implementations should also accept both. + +### Macrolanguages + +ISO (and hence BCP 47\) has the notion of an individual language (like 'en' \= English) versus a Collection or Macrolanguage. For compatibility, Unicode language and locale identifiers always use the Macrolanguage to identify the predominant form. Thus the Macrolanguage subtag 'zh' (Chinese) is used instead of 'cmn' (Mandarin). Similarly, suppose that you are looking for Kurdish written in Latin letters, as in Turkey. It is a mistake to think that because that is in the north, that you should use the subtag 'kmr' for Northern Kurdish. You should instead use 'ku\-Latn\-TR'. See also: [ISO 639 Deprecation Requests](/development/development-process/design-proposals/iso-639-deprecation-requests-draft). + +Unicode language identifiers do not allow the "extlang" form defined in BCP 47\. For example, use 'yue' instead of 'zh\-yue' for Cantonese. + +### Ethnologue + +The Ethnologue is a great source of information, but it must be approached with a certain degree of caution. Many of the population figures are out of date or not well substantiated. The Ethnologue also focus on native, spoken languages, whereas CLDR and many other systems are focused on written language, for computer UI and document translation, and on fluent speakers (not necessarily native speakers). So, for example, it would be a mistake to look at the [Ethnologue entry for Egypt](https://www.ethnologue.com/country/EG/) and conclude that the right language subtag for the Arabic used in Egypt is 'arz', which has the largest population. Instead, the right code is 'ar', Standard Arabic, which would be the one used for document and UI translation. + + +### Wikipedia + +Wikipedia is also a great source of information, but it must be approached with a certain degree of caution as well. Be sure to follow up on the linked references, not just look at articles. + + + + + + + + diff --git a/docs/site/index/cldr-spec/plural-rules.md b/docs/site/index/cldr-spec/plural-rules.md index 47f1b3baa13..5534dc4611b 100644 --- a/docs/site/index/cldr-spec/plural-rules.md +++ b/docs/site/index/cldr-spec/plural-rules.md @@ -241,7 +241,7 @@ If you were to substitute a different number for "1" in a sentence or phrase, wo ## Plural Rule Syntax -See [LDML Language Plural Rules](https://unicode.org/reports/tr35/tr35-numbers.html#Language_Plural_Rules). +See [CLDR Language Plural Rules](https://unicode.org/reports/tr35/tr35-numbers.html#Language_Plural_Rules). ## Plural Message Migration diff --git a/docs/site/index/cldr-spec/transliteration-guidelines.md b/docs/site/index/cldr-spec/transliteration-guidelines.md index 98aa78f4933..19f91ec2bc6 100644 --- a/docs/site/index/cldr-spec/transliteration-guidelines.md +++ b/docs/site/index/cldr-spec/transliteration-guidelines.md @@ -351,5 +351,5 @@ For more information, see: - [ISO\-15915 (Gujarati)](http://transliteration.eki.ee/pdf/Gujarati.pdf) - [ISO\-15915 (Kannada)](http://transliteration.eki.ee/pdf/Kannada.pdf) - [ISCII\-91](http://www.cdacindia.com/html/gist/down/iscii_d.asp) -- [UTS \#35: Locale Data Markup Language (LDML)](https://www.unicode.org/reports/tr35/) +- [UTS \#35: Unicode Common Locale Data & Rules (CLDR)](https://www.unicode.org/reports/tr35/) diff --git a/docs/site/index/keyboard-workgroup.md b/docs/site/index/keyboard-workgroup.md index ee3aba1990f..69e4a6f4349 100644 --- a/docs/site/index/keyboard-workgroup.md +++ b/docs/site/index/keyboard-workgroup.md @@ -8,7 +8,7 @@ The CLDR Keyboard Working Group is responsible for the development and maintenan ## Background -**CLDR (Common Locale Data Repository)** +**CLDR (Common Locale Data & Rules)** Computing devices have become increasingly personal and increasingly affordable to the point that they are now within reach of most people on the planet. The diverse linguistic requirements of the world's 7+ billion people do not scale to traditional models of software development. In response to this, Unicode [CLDR](/) has emerged as a standards-based solution that empowers specialist and community input, as a means of balancing the needs of language communities with the technologies of major platform and service providers. diff --git a/docs/site/index/process.md b/docs/site/index/process.md index 31c35338ae9..4e6877a6c32 100644 --- a/docs/site/index/process.md +++ b/docs/site/index/process.md @@ -20,7 +20,7 @@ For more information on the formal procedures for the Unicode CLDR Technical Com ## Specification Changes -The [UTS #35: Locale Data Markup Language (LDML)] specification are kept up to date with each release with change/added structure for new data types or other features. +The [UTS #35: Unicode Common Locale Data & Rules (CLDR)] specification is kept up to date with each release with change/added structure for new data types or other features. The CLDR TC maintains redirects that redirect users to from each major CLDR release to the latest version of the spec. For example, reports/tr35/46/tr35.html will redirect to the 46.1 version of the specification since the 46.1 was the latest revision of the LDML for 46. @@ -255,4 +255,4 @@ The current Technical Committee Officers are: [Requesting Changes]: /requesting_changes [Unicode Calendar]: https://www.unicode.org/timesens/calendar.html [UCA]: https://www.unicode.org/reports/tr10/#Default_Unicode_Collation_Element_Table -[UTS #35: Locale Data Markup Language (LDML)]: https://www.unicode.org/reports/tr35/ +[UTS #35: Unicode Common Locale Data & Rules (CLDR)]: https://www.unicode.org/reports/tr35/ diff --git a/docs/site/index/process/keyboard-repository-process.md b/docs/site/index/process/keyboard-repository-process.md index 9e37b191411..01a956ca2ea 100644 --- a/docs/site/index/process/keyboard-repository-process.md +++ b/docs/site/index/process/keyboard-repository-process.md @@ -4,7 +4,7 @@ title: 'Keyboard Intake Procedures in CLDR' > Editorial note: Please post feedback on this document to [CLDR-17254] -The goal of this document is to clarify the CLDR TC procedures for onboarding keyboard layouts into the [Unicode Common Locale Data Repository](https://cldr.unicode.org/) (CLDR). The guide for contributing keyboards is currently in development. +The goal of this document is to clarify the CLDR TC procedures for onboarding keyboard layouts into the [Unicode Common Locale Data & Rules](https://cldr.unicode.org/) (CLDR). The guide for contributing keyboards is currently in development. The keyboard section of the repository provides data for keyboard layouts in a stable and machine-readable format. The most significant benefit of hosting the keyboard data in CLDR for vendors is that they already use CLDR as a trusted source of locale data and will now be able to also have a trusted source for keyboard layouts. Through CLDR’s policies that establish the credibility of its data content, and its standardization of formats to represent the data, including the new LDML format for keyboard layouts, CLDR lowers the barrier for vendors to include keyboards in their products. diff --git a/docs/site/index/survey-tool.md b/docs/site/index/survey-tool.md index 1b827237440..4b2d26a0dd0 100644 --- a/docs/site/index/survey-tool.md +++ b/docs/site/index/survey-tool.md @@ -10,7 +10,7 @@ title: CLDR Survey Tool CLDR provides key building blocks for software to support the world's languages, with the largest and most extensive standard repository of locale data available. -Translations in the Unicode Common Locale Data Repository are gathered and processed via what is called the Survey Tool, an online tool that can be used to view data for different languages and propose additions or changes. This tool provides a way to propose new localized data, see what others have proposed, and communicate with them to resolve differences. During each submission period, contributors from Unicode Consortium members, other organizations and the public at large are invited to review the data for their languages and countries, and propose new translations of terms or modifications, including language translations entirely new to the repository. +Translations in the Unicode Common Locale Data & Rules are gathered and processed via what is called the Survey Tool, an online tool that can be used to view data for different languages and propose additions or changes. This tool provides a way to propose new localized data, see what others have proposed, and communicate with them to resolve differences. During each submission period, contributors from Unicode Consortium members, other organizations and the public at large are invited to review the data for their languages and countries, and propose new translations of terms or modifications, including language translations entirely new to the repository. Below are the main pages to look at. diff --git a/docs/site/requesting_changes.md b/docs/site/requesting_changes.md index 745a016ee2c..84068fb040e 100644 --- a/docs/site/requesting_changes.md +++ b/docs/site/requesting_changes.md @@ -12,10 +12,10 @@ There are two ways to request changes (for bug fixes or new features). Filing a * Defects in the survey tool * Adding or changing non-language data (such as currency usage) * Additions or changes to data that is not yet handled by the survey tool (collation, segmentation, and transliteration) -* Feature requests in CLDR or [UTS #35: Locale Data Markup Language (LDML)](https://www.unicode.org/reports/tr35/). +* Feature requests in CLDR or [UTS #35: Unicode Common Locale Data & Rules (CLDR)](https://www.unicode.org/reports/tr35/). In CLDR Bug Reports, please try to give as much information as possible to help address the issue, and please group related bugs -(such as a list of problems with the LDML specification) into a single bug report. +(such as a list of problems with the CLDR specification) into a single bug report. A few areas are particularly tricky. diff --git a/docs/site/translation/core-data/exemplars.md b/docs/site/translation/core-data/exemplars.md index 18ee59d9812..d254612f2b2 100644 --- a/docs/site/translation/core-data/exemplars.md +++ b/docs/site/translation/core-data/exemplars.md @@ -76,7 +76,7 @@ The very last line shows an internal UnicodeSet format. You can normally ignore The exemplar character sets contain the commonly used letters for a given modern form of a language. These are used for testing and for determining the appropriate repertoire of letters for various tasks, like choosing charset converters that can handle a given language. The term “letter” is interpreted broadly, and includes characters used to form words, such as 是 or 가. It should not include presentation forms, like [U+FE90](https://util.unicode.org/UnicodeJsps/character.jsp?a=FE90) ( ‎ﺐ‎ ) ARABIC LETTER BEH FINAL FORM, or isolated Jamo characters (for Hangul). - For charts of the standard (non-CJK) exemplar characters, see a chart of the [standard exemplar characters](https://www.unicode.org/cldr/charts/latest/by_type/core_data.alphabetic_information.main.html). -- For more information, please see [Section 5.6 Character Elements](http://unicode.org/reports/tr35/tr35-6.html#Character_Elements) in UTS#35: Locale Data Markup Language (LDML). +- For more information, please see [Section 5.6 Character Elements](http://unicode.org/reports/tr35/tr35-6.html#Character_Elements) in UTS#35: Unicode Common Locale Data & Rules (CLDR). There are different categories: diff --git a/docs/site/translation/date-time/date-times-terminology.md b/docs/site/translation/date-time/date-times-terminology.md index d494dbc3994..019948e24e5 100644 --- a/docs/site/translation/date-time/date-times-terminology.md +++ b/docs/site/translation/date-time/date-times-terminology.md @@ -6,7 +6,7 @@ title: Date & Time terminology This topic is **in-progress** and and **not finalized** yet for use. -Following are terminology and definitions that are used for Date and Time structure and data in CLDR. The terminology used in CLDR have dependency on LDML Spec #35 and names of methods and objects in ICU. +Following are terminology and definitions that are used for Date and Time structure and data in CLDR. The terminology used in CLDR have dependency on the CLDR Spec #35 and names of methods and objects in ICU. | Terminology | Definition | Examples | |---|---|---|