1. Update ucmlocal.mk and convertrs.txt to refer to euc-kr-html.ucm
instead of windows-949.ucm

2. Tighten up the valid code range for the following converters:

   EUC-KR, Shift_JIS, Big5

This is to add back an ASCII range byte to the stream per
the encoding spec when they're either illegal as a 'trail byte' or
there's no assigned code point for a "lead + trail" sequence.
For instance, with this change, '0xF3 0x41' in EUC-KR is converted to
'U+FFFD U+0041' instead of 'U+FFFD'.

This change requires adding 2 ~ 8 new states to the conversion
table of each converter mentioned above leading to 6.5kB net increase
in the final data size.

3. Tighten the trail byte range for 2-byte sequences starting with 0x8E
from [A1,E2] to [A1,DF] in EUC-JP and update the corresponding generating
script.

4. Change the substitution characters for EUC-JP and Shift_JIS to
match other converters. i.e. make them produce U+FFFD when encountering
an invalid input. Before this chaange, they emitted U+001A.

5. Enable 'U_CHARSET_IS_UTF8' configuration flag.
Chromium/Blink does not rely on ICU for the code conversion between
the 'system native encoding' (if it's one of legacy encodings)
and Unicode. With this configuration, we can cut down the code size
a bit.

6. Update the icudtl.dat (all platforms) and assembly files (mac,linux)
   and the icudata dll (windows)

See https://codereview.chromium.org/1026453002 for a new blink test
added ( fast/encoding/char-decoding-invalid-trail.html )

BUG=450312,430823
TEST=Blink: fast/encoding/char-decoding-{truncated,invalid-trail}.html
TEST=base_unittests --gtest_filter=*Conv*, browser_tests --gtest_filter=*ncoding*
R=jsbell@chromium.org, mark@chromium.org

Review URL: https://codereview.chromium.org/984233002

When icu_use_data_file_flag==1, nothing is done other than copying
icudtl.dat. [1]  Therefore, the type is changed to 'none' from
'static_library'.

Although I couldn't locally reproduce the problme, one of iOS try bots
tried to link with libicudata.a [2] and failed when the type is 'static_library'.
This change should fix the problem.

[1] http://codereview.chromium.org/926113004 moved stubdata.c to icuuc.
[2] https://codereview.chromium.org/878723002/ PS 14 has the above change in
icu.gyp. ios_dbg_simulator_ng failed to link looking for libicudata.a in the
1st try. When I ran the trybot again, the compile went through.

BUG=428145
TEST=All bots (including iOS bots) can build successfully.
R=mark@chromium.org

Review URL: https://codereview.chromium.org/937513002

This is to fix a linker error when linking icuuc.dll;  ICU_DATA_ENTRY (icudt54_dat) symbol is not found on Windows clean build from the scratch (component=shared_library).

Move stubdata.c to icuuc target from icudata target. Also, make
U_DATA_API (used for U_ICU_DATA_ENTRYPOINT in common/udata.cpp) to be
U_EXPORT instead of U_IMPORT when icu_use_data_file_flag = 1 or
on Windows. On Windows, using the icudt.dll  (i.e. icu_use_data_file_flag=0) also requires this change.

BUG=428145
TEST=All trybots can build a target that requires ICU.
R=mark@chromium.org, scottmg@chromium.org

Review URL: https://codereview.chromium.org/926113004

1. Replace 'src/' with a fresh copy of ICU 54.1 from the upstream.
2. Remove unncessary files/directories
3. Update the source/header file lists in icu.gypi and icu.gyp
4. Update README.chromium to warn that the ICU head is not usable for
   building Chromium/Blink/v8,yet.

BUG=428145
TEST=None

Review URL: https://codereview.chromium.org/845603002

We don't support using the system ICU on android any more (the library
is not included in the NDK and so cannot be relied on). Remove the
support for it in ICU gyp files, to eliminate remaining references to
the deprecated "android_src" gyp variable.

BUG=
R=mkosiba@chromium.org

Review URL: https://codereview.chromium.org/690603002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@292738 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

UCONFIG_NO_NON_HTML5_CONVERTER was added earlier to our copy of ICU, but
it was never set to 1.  It's my oversight.

1. Turns UCON..CONVERTER on in icu.gyp to drop all the encodings not
   required by the Encoding spec. Dropped encodings include
   UTF-7, BOCU, SCSU, CESU, ISCII, ISO-2022-{KR, CN*}, HZ-GB, ISO-2022-JP's
   other than the original.

2. A lot more sections of the ICU converter code are excluded when
   it's set to 1 including the code for LMB (Lotus Multibyte) encodings and
   X11 compound text encoding (icu common).

3. The character encoding detections for encodings excluded are also disabled.
   (icu i18n)

4. ISO-2022-{KR, CN*} and HZ-GB can be dropped now because Blink treats them
   as replacement encoding. The corresponding alias entries from convertrs.txt
   are also removed.

5. ibm-874 was removed. We used to need it before Blink started, but not any
   more. We only need windows-874.

6. A mistaken in convertrs.txt was corrected : Big5-HKSCS was pointing to
   an old mapping table.

7. Per ICU upstream's suggestion, use '-html' suffix instead of '-html5'
for the encoding tables derived from the WHATWG's encoding spec (ibm866,
shift_jis and euc-jp).

The static 64-bit release build of Chrome on Linux went down from
141,596,616 to 141,491,968 bytes (~ 100 kB reduction). Besides, the icu data
size got smaller by ~ 19 kB ( 10,490,576 to 10,471,008 bytes).

See http://bugs.icu-project.org/trac/ticket/11296 for an upstream bug
I've filed on the issue.


BUG=76328
TEST=browser_tests --gtest_filter="*ncoding*"
TEST=net_unittest --gtest_filter="*ilenameUtil*"
TEST=base_unittests --gtest_filter="*Conv*"
TEST=Blink: fast/encoding/*
TEST=With shared library build, the following has no match.
  nm libicuuc.so | egrep  -i '(bocu|scsu|utf7|2022kr|2022cn|iscii)'
  nm libicui18n.so | egrep  -i '(2022kr|2022cn|ibm42)'
TEST=With static library build, the following has no match.
  nm chrome | egrep -i '(bocu|scsu|utf7|2022kr|2022cn|iscii|ibm42)'

R=jsbell@chromium.org, mark@chromium.org

Review URL: https://codereview.chromium.org/587833004

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@292131 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

ICU fails to gyp when run with GYP_DEFINES="android_webview_build=1
use_system_icu=0 use_system_stlport=1" which is a combination of
settings we're trying to bring up temporarily as we migrate away from
system libraries. It fails because it's not permitted to specify
link_settings in a target_condition as the processing is too late.

Remove the invalid link_settings, since we can't do this outside
target_conditions as it's not valid to use -lgabi++ on the host build of
ICU. The link dependency on gabi++ will have to be satisfied manually in
the main libwebviewchromium target instead for this configuration.

BUG=409851
R=mkosiba@chromium.org
TBR=jshin@chromium.org

Review URL: https://codereview.chromium.org/527193003

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@291781 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

TBR=rnk@chromium.org

Review URL: https://codereview.chromium.org/515263002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@291706 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

while building icu.

The warning isn't very serious, and these functions are supposed to trap. Since
ICU's operator new/delete overrides are technically invalid, I filed
http://bugs.icu-project.org/trac/ticket/11122 with ICU for this.

BUG=82385

Review URL: https://codereview.chromium.org/508343002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@291699 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

It defaults to 1 everywhere except android webview, but it looks like
the windows condition wasn't updated after the switch.

TBR=jshin@chromium.org
R=jam@chromium.org
BUG=404809

Review URL: https://codereview.chromium.org/486103002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@291179 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

Review URL: https://codereview.chromium.org/433183002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@287122 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

The Android WebView's use of the system ICU needs to be updated. The
current copy of ICU in external/icu4c is being replaced with a copy in
external/icu/icu4c/source so we need to update the include paths
referenced by third_party/icu/icu.gyp.

BUG=395244
R=andrewhayden@chromium.org

Committed: 285202

Review URL: https://codereview.chromium.org//412473002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@285220 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

Taken from https://codereview.chromium.org/297293005 (the CL for icu 4.6.1)
by jam

BUG=132145
TEST=none
R=jam@chromium.org

Review URL: https://codereview.chromium.org/341653003

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@277999 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

Applying cjhopman's CL [1] to icu 52

BUG=351052,132145
TEST=For Android build, ninja does not complain about multiple rules
generating icudtl.dat
TBR=cjhopman@chromium.org

[1] https://codereview.chromium.org/317373005/

Review URL: https://codereview.chromium.org/322903003

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@275893 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

dl_open and friends are only used by ICU plugins that we don't use.
Nacl-related targets (with newlib) fail because dlfcn.h is missing.

Setting U_ENABLE_DYLOAD to 0, we don't have to worry about dlfcn.h any more (it's not included)

BUG=132145
TEST=After running build/gyp_chromium, build 'base_i18n_nacl' target. It should succeed without complaining about dlfcn.h.
R=sergeyu@chromium.org

Review URL: https://codereview.chromium.org/263763002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@267357 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

Port the icu46 gyp file change to icu52.
See https://codereview.chromium.org/212593002

BUG=346626,132145
R=torne@chromium.org
TBR=torne@chromium.org

Review URL: https://codereview.chromium.org/257303002

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@267148 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

Add a shell script to trim the ICU data further : trim_data.sh along with
locale list files.  The script does the following:

1. Remove the display names of languages NOT listed in Chrome's Accept-Language
   list. (800kB)
2. Minimize the locale data for locales listed in the A-L list that are
   not a UI locale in Chrome. For those locales, exemplar characters,
   the display name in the native language and layout direction are included.
   (640kB)
3. Filter the region data to drop numeric region display names other than 419
   (Latin-America). (50kB)
4. Filter the currency data (display name and plurals) for historic currencies.
   (200kB)

This CL also checks in icudtl.dat (source/data/in) and
icudt_dat.S (mac and linux). Note that I dropped '52' (the version number)
in the assembly source file name and icu.gyp was adjusted accordingly.

With all these changes, icudtl.dat is ~ 800kB larger than that in ICU 4.6.
The 7z compression (as used by the installer) makes the size difference
go down to ~ 130kB.

BUG=132145
TEST=The icudtl.dat (uncompressed) is about 10.7MB instead of 12.4MB without this CL.
R=mark@chromium.org

Review URL: https://codereview.chromium.org/239543018

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@264811 4ff67af0-8c30-449e-8e8b-ad334ec8d88c

Update icu.gyp and related files to list C/C++ source files and header files.

Besides, HAVE_DLOPEN is set to 0 explicitly in icu.gyp. It's 0 by default
in ICU 4.6.1 but is 1 by default in ICU 52.1, but we don't need it. (it's only
necessary for ICU plugins that Chromium does not use).

I also applied https://codereview.chromium.org/218153002/ (Nico's CL to
simplify icu.gyp).

BUG=132145
TEST=None
R=mark@chromium.org, thakis@chromium.org

Review URL: https://codereview.chromium.org/212623010

git-svn-id: http://src.chromium.org/svn/trunk/deps/third_party/icu52@260941 4ff67af0-8c30-449e-8e8b-ad334ec8d88c