README.chromium

Name: icu
URL: http://site.icu-project.org/
Version: 52.1
License: MIT
Security Critical: yes

Description:
This directory contains the source code of ICU 52.1 for C/C++

1. It was obtained with the following:

    $ svn export --native-eol LF http://source.icu-project.org/repos/icu/icu/tags/release-52-1 icu52

  The following directories we don't use are removed:

   - as_is
   - packaging
   - source/layout
   - source/layoutex
   - source/data/xml

  patches/configure.patch is applied to get runConfigureICU work in the 
  icudata generation step without layout and layoutex directory by removing the
  corresponding Makefile's from ac_config variable.

2. Apply the following patch for platform related headers (putilimpl.h and 
   others).

  - patches/putil.patch for Android and QNX
    Upstream bug for Android : http://bugs.icu-project.org/trac/ticket/10478
    Upstream bug for QNX : http://bugs.icu-project.org/trac/ticket/10811


3. Breakiterator patches

   - Apply patches/brkitr.patch 
     * word.txt 
       a. Move full stops (U+002E, U+FF0E) from MidNumLet to MidNum so that
          FQDN labels can be split at '.'
       b. Move fullwidth digits (U+FF10 - U+FF19) from Ideographic to Numeric.
          See http://unicode.org/cldr/trac/ticket/6555 
     * line.txt
       a. Use Japanese rules for all locales because Japanese tailoring only
          affects Japanese specific characters.
          See http://unicode.org/cldr/trac/ticket/3974
       b. Minor changes in CL, OP and IS definitions to handle 'comma-variants'
          more consistenly.
          See http://unicode.org/cldr/trac/ticket/6557
       c. Fix line breaking for Chinese characters and quotation marks
          See http://unicode.org/cldr/trac/ticket/4200 and 
              http://crbug.com/39779
             

   - Add a new file brklocal.mk (copied from brkfiles.mk) with line_ja.txt
     and word_POSIX.txt dropped from the build list.

   - Apply patches/khmer-dictbe.patch and put in a smaller Khmer dictionary
     (source/data/brkitr/khmerdict.txt) obtained from 
     http://bugs.icu-project.org/trac/ticket/9451

   - android/brkitr.patch (to be applied for Android build only) :
       Reverts some changes about Chinese/Japanese segmentation rules in
       patches/brkitr.patch to reduce binary size for Android.

4. Converter changes : 

  - converters.patch : 
     a. revises existing mapping tables
     b. Remove a lot of unused aliases in the converter alias table
        (source/data/mappings/convrtrs.txt ) leading to 40kB size reduction.

  - Add source/data/mappings/ucmlocal.txt : to list only converters we need.
  - Add two new tables per WHATWG encoding standards for EUC-JP and CP866.
    They're generated with scripts/{eucjp, ibm866}_gen.sh. 
  - Add three 'fake' tables for ISO-2022-CN(-Ext) : noop-*.ucm.

  - uconv.patch 
    a. ucnv2022 uses 3 fake tables for ISO-2022-CN(-Ext) instead of two
       huge tables.
    b. ISO-2022-JP-[1-4] is dropped.
    c. SCSU, BOCU, ISCII, UTF-7 conversion is diabled leading to 
       the 47kB reduction in the code size.

5. Locale changes
  - patches/locale1.patch :
      a. Exemplar character set changes for zh*, ja + 9 Indian locales
      b. Minor fixes for Korean, a few Indic (AmPmMarkers) and 
      others (datetime format)

  - Locale build configuration files: To include the full locale data
    for Chrome's UI languages and the minimum locale data for other locales,
    add reslocal.mk or {trns,sprep,rbnf,coll}local.mk files to
    source/data/{coll,curr,lang.locale,curr,region,translit,zone,rbnf,sprep}.

  - Run scripts/trimdata.sh
      a. Trim the locale data for Chrome's UI langauges :
         locales, lang, region, currency
      b. Trim the locale data for non-UI languages to the bare minimum :
        ExemplarCharacters, LocaleScript, layout, and the name of the
        language for a locale in its native language.
      c. Remove the legacy Chinese character set-based collation 
         (big5han/gb2312han) that don't make any sense and nobdoy uses.

  - android/patch_locale.sh (to be run for Android build only):
      Makes changes to source/data/{curr,region,lang} to exclude these data
      except the language and script names of zh_Hans and zh_Hant.
 
  - Add tg.txt to source/data/locale source/data/lang to add the minimal locale
    data necessary for the spellchecker. In both directories, add tg.txt to
    reslocal.mk

6. Timezone data update
  - Grab the latest version of the following timezone data files and
    put them in source/data/misc.

     metaZones.txt
     timezoneTypes.txt
     windowsZones.txt
     zoneinfo64.txt

   As of April 2014, the latest version is 2014b and the above files
   are available at
   http://source.icu-project.org/repos/icu/data/trunk/tzdata/icunew/2014b/44/

7. Transliterator customization

   - Also add css3transform.txt to source/data/trnslit.
   - Put the following line in trnslocal.mk

     TRANSLIT_SOURCE=css3transform.txt

8. Build-related changes

  - patches/wpo.patch
    Upstream bugs : http://bugs.icu-project.org/trac/ticket/8043
                    http://bugs.icu-project.org/trac/ticket/5701
  - patches/vscomp.patch for building with Visual Studio on Windows.
     a. do not use WINDOWS_LOCALE_API in locmap.c
     b. do not redefine stringpiece::npos
  - patches/data.build.patch :
      Remove unnecessary resources : invuca, unames, collator source, stringprep
  - patches/data.build.win.patch :
      Windows-only data build patch. Add a new target DATALIB to makedata.mak
  - add an empty file (stubdatabuilt.txt) to source/stubdata

9. Pre-built data files are checked in with the following steps on Linux:

    a. Make a icu data build directory outside the Chromium source tree.
    b. Run 'runConfigureICU Linux' outside the source tree.
    c. Run 'make'
    d. 'make' will fail in the 1st pass. Copy source/data/in/coll/invuca.icu
      to {BUILD_DIR_ROOT}/data/out/build/icudt52l/coll and re-run 'make'
      in {BUILD_DIR_ROOT}/data.

    e. 'make' will fail again when pkgdata looks for css3transform.res. Edit
    data/out/tmp/icudata.lst to replace 'css3transform.res' with 'root.res'.
    (see http://bugs.icu-project.org/trac/ticket/10570 ) and run 'make' again.


    - source/data/in/icudtl.dat : Built on Linux with all the patches
      above applied. icudt52l.dat is generated in
      {BUILD_DIR_ROOT}/data/out/tmp and copied to the above location with a
      version number (52) dropped.

    - windows/icudt.dll : With icudt52l.dat in place, all the patches applied
      and header files moved (#11 below), generated by building icudt_build
      project of build/icudt_build.sln on Windows. icudt52.dll is
      generated in bin/{Release,Debug} and copied to windows/icudt.dll
      and checked in. Note that we drop the version number ('52') from the
      dll name to avoind having to update our build scripts/configuration
      files everytime ICU is upgraded to a new version.

    - {mac,linux}/icudtl_dat.S : Built on Linux with all the
      patches above (except android/brkitr.patch) applied and checked in.
      This file will be generated in {BUILD_DIR_ROOT}/data/out/tmp as
      icudt52l_dat.S, but '52' is dropped while copying.

      mac/icudtl_dat.S is identical to linux/icudtl_dat.S. It's made
      by changing the header portion of the Linux version to read as following
      (no leading whitespace) :

          .globl _icudt52_dat
          #ifdef U_HIDE_DATA_SYMBOL
                 .private_extern _icudt52_dat
          #endif
                 .data
                 .const
                 .align 4
          _icudt52_dat:


    - android/icudtl_dat.S : Built on Linux with all the patches above and
      android/brkitr.patch applied and android/patch_locale.sh executed.
      '52' is dropped from the name generated in the build tree.

    - android/icudtl.dat : Generated as icudt52l.dat in
      {BUILD_DIR_ROOT}/data/out/tmp along with icudt52l_dat.S and
      copied to the above location with '52' dropped in its name.


10. Change export of U_ICUDATA_ENTRY_POINT from U_IMPORT to U_EXPORT.
    - patches/declspec.patch