The following locale variants are added to the ICU. Only source/data/locales
are added and other locale categories are left alone.

- es-* variants
- en-SG and en-HK
- Arabic variants whose NumberElements are different from ar.

Data file size changes:

- desktop:
   10192200 - 10178576 = 13624  (adding es variants)
   10197040 - 10192200 =  4840  (adding ar variants)
- Android: 6635632 - 6616864 = 18768
- iOS: 6623888 - 6605152 = 18736


Bug:chromium:732762
Test:See the bug report and comment 13
TBR=littledan@chromium.org
Change-Id: I3e87be3413e22aa49f24f81b0e25bd6e4187c0d4
Reviewed-on: https://chromium-review.googlesource.com/576113


Reviewed-by: Jungshik Shin <jshin@chromium.org>

They're supported by Google Translate and their names need to be shown
in the Translate UI.

Hmong (hmn)
Javanese (jv)
Luxembourgish (lb)
Samoan (sm)

The size impact for desktop Chrome is about 3kB.
( 10,175,056 => 10,178,576)

BUG=733398
TEST=See the bug
TBR=yyushkina@chromium.org

Change-Id: I9ca232343847a9a95c1dabacb6336bd43ba39849
Reviewed-on: https://chromium-review.googlesource.com/574771


Reviewed-by: Jungshik Shin <jshin@chromium.org>

http://bugs.icu-project.org/trac/ticket/13277

Also fix README.chromium file to refer to 59.1 instead of 58.1.

BUG=chromium:734093
TEST=None (upstream has a test but not included in the Chrome's copy of ICU)
TBR=avi@chromium.org

Change-Id: I0177d80d7c94199fdea107f891a922b67f0d8481
Reviewed-on: https://chromium-review.googlesource.com/574935


Reviewed-by: Jungshik Shin <jshin@chromium.org>

BUG=729032
R=jshin@chromium.org

Review-Url: https://codereview.chromium.org/2921843003 .

Bug:chromium:726654
Test: gn gen works without an error
Review-Url: https://codereview.chromium.org/2913423002 .

This is necessary for Chrome OS to calculate sunset time.

Bug:chromium:726654
Test: With https://codereview.chromium.org/2887913004 patched in, 'gn
Change-Id: Icf6a8cd833f8afcd1a2d0ce874cf1c8c92fcba44
gen <out_dir> --check' does not complain.
TBR=afakhry@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/520323


Reviewed-by: Jungshik Shin <jshin@chromium.org>

Without a Linux-ish define for Fuchsia, neither U_WCHAR_IS_UTF16 nor
U_WCHAR_IS_UTF32 are defined, resulting in warnings in ICU code when
compiled on Fuchsia. Filed upstream at
http://www.icu-project.org/trac/ticket/13202.

BUG=706592
R=jshin@chromium.org

Review-Url: https://codereview.chromium.org/2885143008 .

   http://www.icu-project.org/trac/ticket/12333
   http://www.icu-project.org/trac/ticket/13189
   http://www.icu-project.org/trac/ticket/12635

   - patches/ucase_utf8.patch
   - patches/ucurr_locale.patch
   - patches/collator_range.patch

Bug=chromium:722124
TEST=See the test
TBR=inferno@chromium.org

Change-Id: I3e754734a1c1121c8059a386ca1b5d0f004adb79
Reviewed-on: https://chromium-review.googlesource.com/506592


Reviewed-by: Abhishek Arya <inferno@chromium.org>
Reviewed-by: Jungshik Shin <jshin@chromium.org>

Also add support for host_byteorder logic which is introduced in -
https://codereview.chromium.org/2815453004/

R=adamk@chromium.org, dpranke@chromium.org, machenbach@chromium.org
BUG=706728

Review-Url: https://codereview.chromium.org/2812173002 .

* Highlights:
  - Emoji 5.0 data (partial; Emoji_Component property not included)
  - CLDR 31.0.1 (http://blog.unicode.org/2017/03/cldr-version-31-released.html)
    UTC and GMT are treated as distinct)
  - New case mapping API for styled text
  - C++ 11 is required
  - char16_t for UChar (UTF-16)
  - Source code is in UTF-8

* Size changes

  common: 10,130,560 => 10,175,056
  android: 6,573,872 => 6,616,864
  iOS: 6,562,352 => 6,605,152

On top of ICU 59.1 from the upstream, the following changes were applied.
See https://chromium.googlesource.com/chromium/deps/icu/+log/chromium/59staging

  - Fix C++ 11 string literal assignment issue (upstream bug: 13192)
  - Fix C4229 warning by MSVC
  - Apply utf32.patch and include unistr.h in fuzzer_util
  - Update ICU data files
  - Fix wpo.patch
  - Apply Google locale patch and locale1.patch
  - update readme
  - Apply breakiterator related patches
  - Apply and update wpo.patch
  - Drop unused patch, apply data.build.win.patch, update README.chromium
  - Add /utf-8 flag for Windows/Visual Studio
  - Update BUILD.gn for UChar, stubdata and apply data_sym.patch
  - use stubdata.cpp instead of stubdata.c in icu.gyp
  - Update icu.gyp* files for v8
  - Update BUILD.gn, apply data.build.patch and vscomp.patch
  - Add new files in ICU 59.1
  - Get a fresh copy of ICU 59.1 from the upstream
  - Update update.sh script

TBR=drott@chromium.org, yangguo@chromium.org
Bug:699469
TEST: layout tests, all unittests, browser tests
Change-Id: Ie1e77323aa0c7f872153680c4deca6471a771a5c
Reviewed-on: https://chromium-review.googlesource.com/505173


Reviewed-by: Jungshik Shin <jshin@chromium.org>

Add ios/icudtl.dat and ios/patch_locale.sh.

Update README.chromium and BUILD.gn accordingly.
Update scripts/copy_data.sh to take (ios|common|android).

At the moment, iOS data is almost identical to that of Android, but in the future
more cuts may be made (e.g. dictionary data for breakiterator).

Bug: 718955
TEST: iOS Chrome works as before.
Review-Url: https://codereview.chromium.org/2743123002 .

BUG=539572
R=jshin@chromium.org

Review-Url: https://codereview.chromium.org/2835993002 .

See

  http://mm.icann.org/pipermail/tz-announce/2017-March/000046.html

for details.

  - Haiti resumed observing DST.

Data file size changes:

   android/icudtl.dat 6573792 =>  6573872
   common/icudt*.dat 10130480 => 10130560

BUG=473288
TEST=Try the following in d8 or Chrome's JS console:
 > (new Date("3/12/2016 12:00Z")).toLocaleString("en", {timeZone:
 "America/Port-au-Prince"})
 "3/12/2016, 7:00:00 AM"
 > (new Date("3/12/2017 12:00Z")).toLocaleString("en", {timeZone:
 "America/Port-au-Prince"})
 "3/12/2017, 8:00:00 AM"

Review-Url: https://codereview.chromium.org/2807343004 .

Delete empty units,units{Narrow,Short} blocks after trimming units data.
Empty units* blocks in en_GB and a few other locales after trimming
causes ICU to fail to fall back to get the duration data for those
locales.

In addition, fix source/data/translit/root_subset.txt. Rule*Ids block has
to be present even though it's empty. When dropping Hans-Hant transform
rules, root_subset.txt was changed to be completely empty, which broke
"components_unittests --g_test_filter=AutofillProfileComparato*" .

With these changes, regenerate ICU data files. The size is slightly smaller.

android/icudtl.dat  6573872 => 6573792
common/icudt*dat    10130560 => 10130480

BUG=707515,677043,684609
TEST=components_unittests --gtest_filter=AutofillProfileComparato*
TEST=ui_base_unittests --gtest_filter=L10nUtilTest.TimeDurationForm*
R=derat@chromium.org

Review-Url: https://codereview.chromium.org/2812943003 .

See

  http://mm.icann.org/pipermail/tz-announce/2017-February/000045.html

for details.

  - Mongolia stops observing DST.
  - America/Punta_Arenas: a new timezone for Soutthern Chile

   android/icudtl.dat 6573024 =>  6573872
   common/icudtb.dat 10129712 => 10130560
   common/icudtl.dat 10129712 => 10130560

BUG=473288
TEST=Try the following in d8 or Chrome's JS console:
 > (new Date("4/4/2017 12:00Z")).toLocaleString("en", {timeZone:
 "Asia/Ulaanbaatar"})
 "4/4/2016, 8:00:00 PM"
 > (new Date("4/4/2016 12:00Z")).toLocaleString("en", {timeZone:
 "Asia/Ulaanbaatar"})
 "4/4/2016, 9:00:00 PM"
TBR=mark

Review-Url: https://codereview.chromium.org/2755833002 .

The file was mistakenly dropped from
https://codereview.chromium.org/2747173004/

BUG=684609
TBR=riesa@chromium.org

Review-Url: https://codereview.chromium.org/2750943005 .

Summary of data size decrease:
 android/icudtl.dat   6610128 ->  6573024 bytes
 common/icudtb.dat   10166816 -> 10129712 bytes
 common/icudtl.dat   10166816 -> 10129712 bytes

Patch by riesa@chromium.org

BUG=684609
R=jshin@chromium.org

Review-Url: https://codereview.chromium.org/2747173004 .

BUG=None

Review-Url: https://codereview.chromium.org/2736013003 .

Fix the path for Khmer dictionary and word_ja in data_files_to_preserve.txt.

Add icu4c to the upstream repository path.

BUG=None

Review-Url: https://codereview.chromium.org/2732393002 .

Size increase for affected data files is as follows:
 android/icudtl.dat   6573776 ->  6610128 bytes
 common/icudtb.dat   10130464 -> 10166816 bytes
 common/icudtl.dat   10130464 -> 10166816 bytes

This CL supercedes CL 2328013002.

CL by riesa@chromium.org.

BUG=684609
R=jshin@chromium.org

Review-Url: https://codereview.chromium.org/2652023002 .

Ported from OSS-Fuzz: https://github.com/google/oss-fuzz/pull/304/files.

TBR=aizatsky@chromium.org, jshin@chromium.org
BUG=

Review-Url: https://codereview.chromium.org/2650653003 .

https://codereview.chromium.org/2504463002 added suppresion of
deprecation warnings in an "if clang" block, but gcc also warns about
these, as can be seen on Chrome-Android waterfall for example. Fix
by moving the supression to an if linux||android block.

BUG=21515
R=thakis@chromium.org

Review URL: https://codereview.chromium.org/2571433002 .

Patch from Tomasz Śniatowski <tsniatowski@opera.com>.

Recent changes in detecting big endian targets don't work when cross
compiling for a target with different endianness. This brakes MIPS.
Added additional checks for big endian MIPS and MIPS64 targets to fix
this.

Patch by miran.karic@imgtec.com

BUG=None
R=bjaideep@ca.ibm.com, jshin@chromium.org

Review URL: https://codereview.chromium.org/2518533002 .

Europe/Saratov is a new timezone spun off from Europe/Volgograd.
It switched from +03 to +04 on 2016-12-04 at 02:00.

For more details, see

http://mm.icann.org/pipermail/tz-announce/2016-November/000044.html

BUG=473288
TEST=Try the following in d8 or Chrome's JS console:
> (new Date("12/04/2016 12:00Z")).toLocaleString("en", {timeZone:
"Europe/Saratov"})
"12/4/2016, 4:00:00 PM"
> (new Date("12/03/2016 12:00Z")).toLocaleString("en", {timeZone:
"Europe/Saratov"})
"12/3/2016, 3:00:00 PM"
TBR=mark

Review URL: https://codereview.chromium.org/2558183003 .

BUG=473288
TEST=None

Review URL: https://codereview.chromium.org/2511863003 .

Update IANA timezone db to 2016i from 2016h

See http://mm.icann.org/pipermail/tz-announce/2016-November/000043.html
for what's changed.

BUG=473288
TBR=mark@chromium.org
TEST=Try the following in d8 or Chrome's JS console:
> (new Date("11/07/2016 12:00Z")).toLocaleString("en", {timeZone:
"Pacific/Tongatapu"})
"11/8/2016, 2:00:00 AM"
> (new Date("11/05/2016 12:00Z")).toLocaleString("en", {timeZone:
"Pacific/Tongatapu"})
"11/6/2016, 1:00:00 AM"

Review URL: https://codereview.chromium.org/2510163002 .

In crrev.com/2492553006 I accidentally deleted a comment line from
BUILD.gn. This change restores that comment, per reviewer request.

TBR=jshin@chromium.org

BUG=21515

Review URL: https://codereview.chromium.org/2503833003 .

The previous change re-enabled warnings too aggressively. While the icu
target built correctly the icu nacl targets (such as
irt_x64/obj/third_party/icu/icuuc/uidna.o on Windows and
newlib_pnacl/obj/third_party/icu/icuuc/uidna.o on other platforms)
failed to compile due to deprecation warnings. This change adds back
suppression of that one warning.

BUG=21515
R=thakis@chromium.org

Review URL: https://codereview.chromium.org/2504463002 .

ICU has historically had many clang warnings disabled, but some of the
underlying causes have been fixed. One of the VC++ specific warning
disables was recently removed (crrev.com/2494793002) and this change
removes a bunch more, for clang and VC++. For some of them there are
specific icu bugs that have been fixed that allow this, and for others
I just have the evidence that on Linux and Windows icu compiles with
clang and VC++ with these changes.

BUG=21515
R=thakis@chromium.org

Review URL: https://codereview.chromium.org/2492553006 .

Update config to use icudtb.dat if the byteorder is big
endian, this will work for other big endian platforms
such as PPC and s390.
Original CL for BE support: https://codereview.chromium.org/2162393003

R=jshin@chromium.org
BUG=v8:5567

Review URL: https://codereview.chromium.org/2434153002 .

They don't affect Chrome/Blink at the moment, but had better
be fixed now.

  http://www.icu-project.org/trac/ticket/12815
  http://www.icu-project.org/trac/ticket/12832

BUG=None

Review URL: https://codereview.chromium.org/2499493002 .

Follw-up to https://chromium.googlesource.com/chromium/deps/icu/+/5feb9ad5
(due to a rietveld issue, part 1 was manually pushed).

Update ICU to 58.1 release from ICU 56.1 part2.

Listed below a tiny subset of what's new in 58.1:

  1. Unicode 9.0 from Unicode 8.0
    - Updated character properties including Emoji data up to 4.0beta.
    - Updated grapheme/word/line breaking rules for Emoji sequences and others.

  2. CLDR 30.0.2 from CLDR 28
    - Numerous locale data updates/improvements

  3. Spoofing API changes
  4. Greek uppercasing support as a part of regular case-mapping API.
  5. Line breaking rule file format optimization. This change enables me
     to add CJ loose line breaking rules back (previously, it's dropped
     to save space) so that Blink can use it for CJ.

See http://site.icu-project.org/download/58 for more details on ICU 58.1
and http://site.icu-project.org/download/57 for more details on ICU 57.1

For CLDR 30, see http://cldr.unicode.org/index/downloads/cldr-30 .

The size impact:
   Non-Android: 10,127,200 => 10,128,624 (delta = 1,424 / 0.014%)
   Android: 6,563,152 => 6,571,936 (delta = 8,784 / 0.13%)

Below are the list of changes made on top of the upstream ICU 58.1
in reverse order. Most of these changes were made in 58staging branch
to run trybots and cherry-picked back for this CL. See
   https://chromium.googlesource.com/chromium/deps/icu/+/log/chromium/58staging
   https://codereview.chromium.org/2447513002/ : cr+blink update cl with
       58staging branch head.

* Fix a build on Win without std::string (v8)
* Add ms932 alias to Shift_JIS

* Apply Google-specific locale data patches

* Fix a bug in scriptset

* Update windows-1255 mapping

* Disable C4333 warning by MSVC (harmless)

* Apply and update utf32.patch and README.chromium

* Update and apply vscomp.patch
  stringpiece patch removed. VS2015 seems to be fine with a redefinition.

* Update pre-built ICU data files
   Update *local.mk with a new copyright line

* Apply more patches
  The following patches were applied and updated: data_symb, vscomp, wpo

  The unnecessary part was dropped from vscomp

* Update BUILD.gn and icu.gyp* files

* Update android/brkitr.patch

* Update and apply more patches

* Update and apply cjdict.patch
   Apply data.build.patch

* Delete obsolete patches: cmemory,regex

* Update README.chromium and apply brkitr patches

  - Update README.chromium
  - Remove obsolete patches
  - Update linebrk.patch and apply it: add back line_loose_cj

* Update wordbrk.patch and apply it

* Update and apply khmer-dictbe.patch

* Update data trimming

  - android/patch_locale.sh
  - scripts/trim_data.sh
     ExemplarCh* removed
     charac*Label removed
     relative/relativeTime removed for daysOfWeek and quarter

* Update the following patches

  android/brkitr.patch
  patches/linebrk.patch
  patches/data.build.patch

* Update cjdict.patch and linebrk.patch

BUG=637001
TEST=Layout tests, all unittests, browser tests, ui tests.
R=jsbell@chromium.org, mark@chromium.org

Review URL: https://codereview.chromium.org/2442923002 .

* Note that this CL will be followed by CLs with local changes.
  Until then, ICU should not be rolled in DEPS. See READ_THIS_FIRST
  for details.

* Adjust scripts/update.sh and scripts/data_files_to_preserve.txt
  - CLDR/ICU added ckb/ast locale data. Drop them from the list to preserve.
  - source/layout does not exist in 58.1 any more.

* Update the tree to ICU 58.1 from the upstream by running
  scripts/update.sh

* Update README.chromium and add READ_THIS_FIRST to warn about the
  status of the tree.

BUG=637001
TEST=None

There's no need for VS build files.
Besides, update scripts/update.sh to post-edit source/configure
for missing test/ directory.

This clean up is necessary to get 'git cl upload/rietveld' work
smoothly in an upcoming ICU update.

BUG=637001
TEST=source/runConfigureICU Linux --disable-tests --disable-layout

Review URL: https://codereview.chromium.org/2443653002 .

We don't use source/test. It's kept to give API usage examples, but
it got in the way of a version update (git cl upload keeps timing out).

Also, update update.sh to delete source/test after downloading a new
version from the upstream.

BUG=637001
TEST=None

Review URL: https://codereview.chromium.org/2435373002 .

The file was removed in chromium:
https://codereview.chromium.org/2331893004

This moves it here to keep the logic working for
projects depending on ICU that still have a few gyp bots.

BUG=chromium:474921
R=jochen@chromium.org

Review URL: https://codereview.chromium.org/2335993003 .

Make icu depend on //buildtools/third_party/libc++abi
on mac for sanitizer builds.

BUG=636127
TEST=Libfuzzer Upload Mac ASan builds
R=mmoroz@chromium.org

Review URL: https://codereview.chromium.org/2247953005 .

This is only used in a non-default build configuration.
It's for IANA timezone DB 2016f. The ICU data files for
the default build configuration were updated earlier.

BUG=473288

Review URL: https://codereview.chromium.org/2222353003 .

BUG=626078
TEST=trybot succeeds on non-Windows.
R=rsesek@chromium.org

Review URL: https://codereview.chromium.org/2218673002 .

Private headers are listed in source[] while public headers are listed
in public[] for icuuc and icu18n targets.

This change is to let trybot analyze step recompile icuuc or icu18n even
when the only change in CL is in a header file (public or private).

BUG=632438
TEST=`gn check <out_dir>` does not complain.
TEST=`gn ref <out_dir> third_party/icu/source/common/umutex.h` matches icuuc.
TEST=`gn ref <out_dir> third_party/icu/source/common/unicode/unistr.h`
     matches icuuc.

R=brettw@chromium.org

Review URL: https://codereview.chromium.org/2186343002 .