Compare commits

...
Author SHA1 Message Date
Jeff Genovy 4f715ae124 ICU-20536 ICU 63.2 release tasks + tzdata2019a 2019-04-11 12:04:36 -07:00
Jeff Genovy c948a1ae31 ICU-20536 Japanese era Reiwa support in ICU4J 63 2019-04-11 12:04:36 -07:00
Jeff Genovy 838a14ee42 ICU-20536 Japanese era Reiwa support in ICU4C 63 2019-04-11 12:04:36 -07:00
Markus Scherer 0af507c4da ICU-20255 revert to reflection for methods not yet in Android API level 21..23
(cherry picked from commit cee3c150ab)
2019-02-11 17:17:20 -08:00
Markus Scherer f3fa0d604e ICU-20250 make UnicodeSet(intprop=value) faster
- fastpath for UnicodeSet.add(new last range)
- fewer UnicodeSet memory allocations:
  initial internal list array, exponential array growth,
  allocate strings list/set only when first one is added
- faster CodePointTrie.getRange(): fewer calls to filter function
- revert UnicodeSet(intprop=value) from trie ranges to range starts + lookup
- cache per-int-prop range starts: fewer lookups

(cherry picked from commit 98f9170004)
2018-11-15 13:59:02 -08:00
Markus Scherer 38b882fe30 ICU-20250 faster MutableCodePointTrie.build(): use a hashtable to find equal blocks in earlier parts of the compacted data & index arrays
(cherry picked from commit 59006770ed)
2018-11-15 13:59:02 -08:00
Shane Carr 6cbd62e59e ICU-20246 Fixing another integer overflow in number parsing.
(cherry picked from commit 53d8c8f3d1)
2018-10-31 18:11:18 -07:00
Jeff Genovy 2094a48c20 ICU-20240 Fix Windows build failure in japancal.cpp
(cherry picked from commit 3d86d24a68)
2018-10-31 18:09:36 -07:00
Jeff Genovy fced46c6a9 ICU-20209 Fix build failures on Windows with std::atomic not in enclosing namespace std
Also: Change to use the ICU namespace macro instead, and skip doxygen as well.
(cherry picked from commit 5a34bfb151)
2018-10-31 18:08:52 -07:00
Ken Brown 78d61f7cb9 ICU-20214 Fix namespace error on Cygwin
(cherry picked from commit 31b89b5484)
2018-10-22 17:13:36 -07:00
Jeff Genovy 8baff8f03e ICU-20208 uspoof.cpp function checkImpl should be static, regenerate urename.h
(cherry picked from commit 9ec2c332c1)
2018-10-22 13:46:18 -07:00
Andy Heninger 46895456ad ICU-20205 RelativeDateTimeFormatter pt data fix, improved error handling and test. (#210)
* ICU-20205 Add locale test for RelativeDateTimeFormatter.

* ICU-20205 Fix error in pt relative date data. Improve error handling in code.

* ICU-20205 Add instantiation test & regen data from ICU4C

* ICU-20205 Added DateFormatSymbols error check per jefgen's comments.

(cherry picked from commit e509105c9b)
2018-10-13 09:30:01 -07:00
leadpony 7451a2c0c7 ICU-20192 Add Automatic-Module-Name to META-INF/MANIFEST.MF (#193)
* Add Automatic-Module-Name to META-INF/MANIFEST.MF

* Added com.ibm.icu.number in Export-Package list

* Add Automatic-Module-Name to charset and localespi

(cherry picked from commit 2e580c0504)
2018-10-10 21:50:27 -07:00
Andy Heninger 437bec9a8b ICU-20120 Travis TSAN testing, limit time.
(cherry picked from commit a8eeeaf1d47e255c16885cb3c17ee3d39bff4aef)
2018-10-10 20:43:52 -07:00
Andy Heninger a84940ee61 ICU-20120 Travis TSAN testing, limit time.
(cherry picked from commit 71f31c2a167226eb62360e08895a75b7e2cd6b84)
2018-10-10 20:43:52 -07:00
pedberg-icu c0e0586223 ICU-20120 63 BRS, integrate cldr release-34-beta with various fixes (#186)
(cherry picked from commit b098078ab8)
2018-10-10 19:57:09 -05:00
Yoshito Umaoka c264f0193b ICU-20120 BRS63GA Updating ICU4J 63 readme.html (#203)
Just updating API change report link from svn to local.

(cherry picked from commit 35e3ba8530)
2018-10-10 19:56:17 -05:00
Yoshito Umaoka 70f341bea7 ICU-20182 Updated ICU4J pom to reflect the repository migration from svn to git. (#196)
(cherry picked from commit 31d40cc5ee)
2018-10-10 19:56:17 -05:00
Jeff Genovy d6fe1ed312 ICU-20120 BRS63GA: Update version numbers/readme for ICU 63 GA.
(cherry picked from commit 1af92cd3e7)
2018-10-09 15:44:24 -07:00
Jeff Genovy aff4b20fb6 ICU-20196 Fix MSVC/clang compiler warning C4251, export explicit template, required for building DLLs on Windows. Also fix a few other warnings as well.
(cherry picked from commit ba001f4616)
2018-10-05 15:48:09 -07:00
gvictor 90bb850311 ICU-20193 urename.h shouldn't be ignored by other compilers except VS IntelliSense (#189)
The macro _MSC_VER is not defined in other compilers.
Change the logic and so urename.h is ignored on VS IntelliSense

(cherry picked from commit ff72d74d1e)
2018-10-05 11:05:00 -07:00
kbrow1i 2e6bfebcf7 ICU-20191 Don't use Win32 APIs on Cygwin (#185)
This was fixed in earlier commits (see, for example, ICU-12786), but
the problem was reintroduced in commits f8ba68e and fcb82cb.

(cherry picked from commit 7d99ba4e57)
2018-10-05 11:04:11 -07:00
259 changed files with 3125 additions and 2277 deletions
+10 -2
View File
@@ -55,7 +55,11 @@ matrix:
#
- name: "c: linux tsan"
language: cpp
env: BUILD=ICU4C_CLANG_THREAD_SAN
env:
- BUILD=ICU4C_CLANG_THREAD_SAN
- INTLTEST_OPTS="utility/MultithreadTest rbbi/RBBIMonkeyTest format/CalendarLimitTest"
- CPPFLAGS="-fsanitize=thread"
- LDFLAGS=-fsanitize=thread
os: linux
dist: trusty
sudo: true
@@ -68,7 +72,11 @@ matrix:
packages:
- clang-5.0
script:
- cd icu4c/source && CPPFLAGS="-fsanitize=thread" LDFLAGS="-fsanitize=thread" ./runConfigureICU --enable-debug --disable-release Linux --disable-renaming && make -j2 check
- cd icu4c/source &&
./runConfigureICU --enable-debug --disable-release Linux --disable-renaming &&
make -j2 &&
make -j2 -C test &&
make -j2 -C test/intltest check
# copyright scan / future linter
- name: "lint"
+5 -5
View File
@@ -3,7 +3,7 @@
<html lang="en-US" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US">
<head>
<title>ReadMe for ICU 63.1</title>
<title>ReadMe for ICU 63.2</title>
<meta name="COPYRIGHT" content=
"Copyright (C) 2016 and later: Unicode, Inc. and others. License &amp; terms of use: http://www.unicode.org/copyright.html"/>
<!-- meta name="COPYRIGHT" content=
@@ -23,8 +23,8 @@
milestone - if the release note is a milestone release
-->
<!-- <body> -->
<body class="rc">
<body>
<!-- <body class="rc"> -->
<p class="only-draft"><b>Note:</b> This is a draft readme.</p>
<h1>
@@ -32,7 +32,7 @@
International Components for Unicode<br/>
<span class="only-rc">Release Candidate</span>
<span class="only-milestone">(Milestone Release)</span>
<abbr title="International Components for Unicode">ICU</abbr> 63.1 ReadMe
<abbr title="International Components for Unicode">ICU</abbr> 63.2 ReadMe
</h1>
<!-- Shouldn't need to comment/uncomment this paragraph, just change the body class -->
@@ -44,7 +44,7 @@
<p class="note only-rc">This is a release candidate version of ICU4C.
It is not recommended for production use.</p>
<p>Last updated: 2018-Sept-18<br/>
<p>Last updated: 2018-Oct-05<br/>
Copyright &copy; 2016 and later: Unicode, Inc. and others. License &amp; terms of use:
<a href="http://www.unicode.org/copyright.html">http://www.unicode.org/copyright.html</a><br/>
Copyright &copy; 1997-2016 International Business Machines Corporation and others.
+72 -27
View File
@@ -23,6 +23,9 @@
#include "umutex.h"
#include "uprops.h"
using icu::LocalPointer;
using icu::Normalizer2Factory;
using icu::Normalizer2Impl;
using icu::UInitOnce;
using icu::UnicodeSet;
@@ -30,11 +33,13 @@ namespace {
UBool U_CALLCONV characterproperties_cleanup();
constexpr int32_t NUM_INCLUSIONS = UPROPS_SRC_COUNT + UCHAR_INT_LIMIT - UCHAR_INT_START;
struct Inclusion {
UnicodeSet *fSet;
UInitOnce fInitOnce;
};
Inclusion gInclusions[UPROPS_SRC_COUNT]; // cached getInclusions()
Inclusion gInclusions[NUM_INCLUSIONS]; // cached getInclusions()
UnicodeSet *sets[UCHAR_BINARY_LIMIT] = {};
@@ -80,35 +85,22 @@ UBool U_CALLCONV characterproperties_cleanup() {
return TRUE;
}
} // namespace
U_NAMESPACE_BEGIN
/*
Reduce excessive reallocation, and make it easier to detect initialization problems.
Usually you don't see smaller sets than this for Unicode 5.0.
*/
constexpr int32_t DEFAULT_INCLUSION_CAPACITY = 3072;
void U_CALLCONV CharacterProperties::initInclusion(UPropertySource src, UErrorCode &errorCode) {
void U_CALLCONV initInclusion(UPropertySource src, UErrorCode &errorCode) {
// This function is invoked only via umtx_initOnce().
// This function is a friend of class UnicodeSet.
U_ASSERT(0 <= src && src < UPROPS_SRC_COUNT);
if (src == UPROPS_SRC_NONE) {
errorCode = U_INTERNAL_PROGRAM_ERROR;
return;
}
UnicodeSet * &incl = gInclusions[src].fSet;
U_ASSERT(incl == nullptr);
U_ASSERT(gInclusions[src].fSet == nullptr);
incl = new UnicodeSet();
if (incl == nullptr) {
LocalPointer<UnicodeSet> incl(new UnicodeSet());
if (incl.isNull()) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return;
}
USetAdder sa = {
(USet *)incl,
(USet *)incl.getAlias(),
_set_add,
_set_addRange,
_set_addString,
@@ -116,7 +108,6 @@ void U_CALLCONV CharacterProperties::initInclusion(UPropertySource src, UErrorCo
nullptr // don't need removeRange()
};
incl->ensureCapacity(DEFAULT_INCLUSION_CAPACITY, errorCode);
switch(src) {
case UPROPS_SRC_CHAR:
uchar_addPropertyStarts(&sa, &errorCode);
@@ -183,12 +174,15 @@ void U_CALLCONV CharacterProperties::initInclusion(UPropertySource src, UErrorCo
}
if (U_FAILURE(errorCode)) {
delete incl;
incl = nullptr;
return;
}
// Compact for caching
if (incl->isBogus()) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return;
}
// Compact for caching.
incl->compact();
gInclusions[src].fSet = incl.orphan();
ucln_common_registerCleanup(UCLN_COMMON_CHARACTERPROPERTIES, characterproperties_cleanup);
}
@@ -199,15 +193,66 @@ const UnicodeSet *getInclusionsForSource(UPropertySource src, UErrorCode &errorC
return nullptr;
}
Inclusion &i = gInclusions[src];
umtx_initOnce(i.fInitOnce, &CharacterProperties::initInclusion, src, errorCode);
umtx_initOnce(i.fInitOnce, &initInclusion, src, errorCode);
return i.fSet;
}
void U_CALLCONV initIntPropInclusion(UProperty prop, UErrorCode &errorCode) {
// This function is invoked only via umtx_initOnce().
U_ASSERT(UCHAR_INT_START <= prop && prop < UCHAR_INT_LIMIT);
int32_t inclIndex = UPROPS_SRC_COUNT + prop - UCHAR_INT_START;
U_ASSERT(gInclusions[inclIndex].fSet == nullptr);
UPropertySource src = uprops_getSource(prop);
const UnicodeSet *incl = getInclusionsForSource(src, errorCode);
if (U_FAILURE(errorCode)) {
return;
}
LocalPointer<UnicodeSet> intPropIncl(new UnicodeSet(0, 0));
if (intPropIncl.isNull()) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return;
}
int32_t numRanges = incl->getRangeCount();
int32_t prevValue = 0;
for (int32_t i = 0; i < numRanges; ++i) {
UChar32 rangeEnd = incl->getRangeEnd(i);
for (UChar32 c = incl->getRangeStart(i); c <= rangeEnd; ++c) {
// TODO: Get a UCharacterProperty.IntProperty to avoid the property dispatch.
int32_t value = u_getIntPropertyValue(c, prop);
if (value != prevValue) {
intPropIncl->add(c);
prevValue = value;
}
}
}
if (intPropIncl->isBogus()) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return;
}
// Compact for caching.
intPropIncl->compact();
gInclusions[inclIndex].fSet = intPropIncl.orphan();
ucln_common_registerCleanup(UCLN_COMMON_CHARACTERPROPERTIES, characterproperties_cleanup);
}
} // namespace
U_NAMESPACE_BEGIN
const UnicodeSet *CharacterProperties::getInclusionsForProperty(
UProperty prop, UErrorCode &errorCode) {
if (U_FAILURE(errorCode)) { return nullptr; }
UPropertySource src = uprops_getSource(prop);
return getInclusionsForSource(src, errorCode);
if (UCHAR_INT_START <= prop && prop < UCHAR_INT_LIMIT) {
int32_t inclIndex = UPROPS_SRC_COUNT + prop - UCHAR_INT_START;
Inclusion &i = gInclusions[inclIndex];
umtx_initOnce(i.fInitOnce, &initIntPropInclusion, prop, errorCode);
return i.fSet;
} else {
UPropertySource src = uprops_getSource(prop);
return getInclusionsForSource(src, errorCode);
}
}
U_NAMESPACE_END
@@ -216,7 +261,7 @@ namespace {
UnicodeSet *makeSet(UProperty property, UErrorCode &errorCode) {
if (U_FAILURE(errorCode)) { return nullptr; }
icu::LocalPointer<UnicodeSet> set(new UnicodeSet());
LocalPointer<UnicodeSet> set(new UnicodeSet());
if (set.isNull()) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return nullptr;
+1 -1
View File
@@ -1737,7 +1737,7 @@ The leftmost codepage (.xxx) wins.
return posixID;
#elif U_PLATFORM_HAS_WIN32_API
#elif U_PLATFORM_USES_ONLY_WIN32_API
#define POSIX_LOCALE_CAPACITY 64
UErrorCode status = U_ZERO_ERROR;
char *correctedPOSIXLocale = nullptr;
+22 -11
View File
@@ -280,7 +280,7 @@ UChar32 getRange(const void *t, UChar32 start,
int32_t prevI3Block = -1;
int32_t prevBlock = -1;
UChar32 c = start;
uint32_t value;
uint32_t trieValue, value;
bool haveValue = false;
do {
int32_t i3Block;
@@ -319,6 +319,7 @@ UChar32 getRange(const void *t, UChar32 start,
return c - 1;
}
} else {
trieValue = trie->nullValue;
value = nullValue;
if (pValue != nullptr) { *pValue = nullValue; }
haveValue = true;
@@ -357,6 +358,7 @@ UChar32 getRange(const void *t, UChar32 start,
return c - 1;
}
} else {
trieValue = trie->nullValue;
value = nullValue;
if (pValue != nullptr) { *pValue = nullValue; }
haveValue = true;
@@ -364,23 +366,32 @@ UChar32 getRange(const void *t, UChar32 start,
c = (c + dataBlockLength) & ~dataMask;
} else {
int32_t di = block + (c & dataMask);
uint32_t value2 = getValue(trie->data, valueWidth, di);
value2 = maybeFilterValue(value2, trie->nullValue, nullValue,
filter, context);
uint32_t trieValue2 = getValue(trie->data, valueWidth, di);
if (haveValue) {
if (value2 != value) {
return c - 1;
if (trieValue2 != trieValue) {
if (filter == nullptr ||
maybeFilterValue(trieValue2, trie->nullValue, nullValue,
filter, context) != value) {
return c - 1;
}
trieValue = trieValue2; // may or may not help
}
} else {
value = value2;
trieValue = trieValue2;
value = maybeFilterValue(trieValue2, trie->nullValue, nullValue,
filter, context);
if (pValue != nullptr) { *pValue = value; }
haveValue = true;
}
while ((++c & dataMask) != 0) {
if (maybeFilterValue(getValue(trie->data, valueWidth, ++di),
trie->nullValue, nullValue,
filter, context) != value) {
return c - 1;
trieValue2 = getValue(trie->data, valueWidth, ++di);
if (trieValue2 != trieValue) {
if (filter == nullptr ||
maybeFilterValue(trieValue2, trie->nullValue, nullValue,
filter, context) != value) {
return c - 1;
}
trieValue = trieValue2; // may or may not help
}
}
}
+2 -2
View File
@@ -2132,7 +2132,7 @@ ultag_parse(const char* tag, int32_t tagLen, int32_t* parsedLen, UErrorCode* sta
// The preferred tag for a redundant tag is always shorter than redundant
// tag. A redundant tag may or may not be followed by other subtags.
// (i.e. "zh-yue" or "zh-yue-u-co-pinyin").
if (uprv_strnicmp(redundantTag, tagBuf, redundantTagLen) == 0) {
if (uprv_strnicmp(redundantTag, tagBuf, static_cast<uint32_t>(redundantTagLen)) == 0) {
const char* redundantTagEnd = tagBuf + redundantTagLen;
if (*redundantTagEnd == '\0' || *redundantTagEnd == SEP) {
const char* preferredTag = REDUNDANT[i + 1];
@@ -2644,7 +2644,7 @@ uloc_toLanguageTag(const char* localeID,
const char* pKeywordStart;
/* Note: uloc_canonicalize returns "en_US_POSIX" for input locale ID "". See #6835 */
int32_t resultCapacity = uprv_strlen(localeID);
int32_t resultCapacity = static_cast<int32_t>(uprv_strlen(localeID));
if (resultCapacity > 0) {
char* buffer;
+284 -115
View File
@@ -60,6 +60,7 @@ constexpr uint8_t I3_18 = 3;
constexpr int32_t INDEX_3_18BIT_BLOCK_LENGTH = UCPTRIE_INDEX_3_BLOCK_LENGTH + UCPTRIE_INDEX_3_BLOCK_LENGTH / 8;
class AllSameBlocks;
class MixedBlocks;
class MutableCodePointTrie : public UMemory {
public:
@@ -92,8 +93,10 @@ private:
void maskValues(uint32_t mask);
UChar32 findHighStart() const;
int32_t compactWholeDataBlocks(int32_t fastILimit, AllSameBlocks &allSameBlocks);
int32_t compactData(int32_t fastILimit, uint32_t *newData, int32_t dataNullIndex);
int32_t compactIndex(int32_t fastILimit, UErrorCode &errorCode);
int32_t compactData(
int32_t fastILimit, uint32_t *newData, int32_t newDataCapacity,
int32_t dataNullIndex, MixedBlocks &mixedBlocks, UErrorCode &errorCode);
int32_t compactIndex(int32_t fastILimit, MixedBlocks &mixedBlocks, UErrorCode &errorCode);
int32_t compactTrie(int32_t fastILimit, UErrorCode &errorCode);
uint32_t *index = nullptr;
@@ -301,41 +304,56 @@ UChar32 MutableCodePointTrie::getRange(
uint32_t nullValue = initialValue;
if (filter != nullptr) { nullValue = filter(context, nullValue); }
UChar32 c = start;
uint32_t value;
uint32_t trieValue, value;
bool haveValue = false;
int32_t i = c >> UCPTRIE_SHIFT_3;
do {
if (flags[i] == ALL_SAME) {
uint32_t value2 = maybeFilterValue(index[i], initialValue, nullValue,
filter, context);
uint32_t trieValue2 = index[i];
if (haveValue) {
if (value2 != value) {
return c - 1;
if (trieValue2 != trieValue) {
if (filter == nullptr ||
maybeFilterValue(trieValue2, initialValue, nullValue,
filter, context) != value) {
return c - 1;
}
trieValue = trieValue2; // may or may not help
}
} else {
value = value2;
trieValue = trieValue2;
value = maybeFilterValue(trieValue2, initialValue, nullValue, filter, context);
if (pValue != nullptr) { *pValue = value; }
haveValue = true;
}
c = (c + UCPTRIE_SMALL_DATA_BLOCK_LENGTH) & ~UCPTRIE_SMALL_DATA_MASK;
} else /* MIXED */ {
int32_t di = index[i] + (c & UCPTRIE_SMALL_DATA_MASK);
uint32_t value2 = maybeFilterValue(data[di], initialValue, nullValue,
filter, context);
uint32_t trieValue2 = data[di];
if (haveValue) {
if (value2 != value) {
return c - 1;
if (trieValue2 != trieValue) {
if (filter == nullptr ||
maybeFilterValue(trieValue2, initialValue, nullValue,
filter, context) != value) {
return c - 1;
}
trieValue = trieValue2; // may or may not help
}
} else {
value = value2;
trieValue = trieValue2;
value = maybeFilterValue(trieValue2, initialValue, nullValue, filter, context);
if (pValue != nullptr) { *pValue = value; }
haveValue = true;
}
while ((++c & UCPTRIE_SMALL_DATA_MASK) != 0) {
if (maybeFilterValue(data[++di], initialValue, nullValue,
filter, context) != value) {
return c - 1;
trieValue2 = data[++di];
if (trieValue2 != trieValue) {
if (filter == nullptr ||
maybeFilterValue(trieValue2, initialValue, nullValue,
filter, context) != value) {
return c - 1;
}
}
trieValue = trieValue2; // may or may not help
}
}
++i;
@@ -548,28 +566,8 @@ void MutableCodePointTrie::maskValues(uint32_t mask) {
}
}
inline bool
equalBlocks(const uint32_t *s, const uint32_t *t, int32_t length) {
while (length > 0 && *s == *t) {
++s;
++t;
--length;
}
return length == 0;
}
inline bool
equalBlocks(const uint16_t *s, const uint32_t *t, int32_t length) {
while (length > 0 && *s == *t) {
++s;
++t;
--length;
}
return length == 0;
}
inline bool
equalBlocks(const uint16_t *s, const uint16_t *t, int32_t length) {
template<typename UIntA, typename UIntB>
bool equalBlocks(const UIntA *s, const UIntB *t, int32_t length) {
while (length > 0 && *s == *t) {
++s;
++t;
@@ -585,36 +583,6 @@ bool allValuesSameAs(const uint32_t *p, int32_t length, uint32_t value) {
}
/** Search for an identical block. */
int32_t findSameBlock(const uint32_t *p, int32_t pStart, int32_t length,
const uint32_t *q, int32_t qStart, int32_t blockLength) {
// Ensure that we do not even partially get past length.
length -= blockLength;
q += qStart;
while (pStart <= length) {
if (equalBlocks(p + pStart, q, blockLength)) {
return pStart;
}
++pStart;
}
return -1;
}
int32_t findSameBlock(const uint16_t *p, int32_t pStart, int32_t length,
const uint32_t *q, int32_t qStart, int32_t blockLength) {
// Ensure that we do not even partially get past length.
length -= blockLength;
q += qStart;
while (pStart <= length) {
if (equalBlocks(p + pStart, q, blockLength)) {
return pStart;
}
++pStart;
}
return -1;
}
int32_t findSameBlock(const uint16_t *p, int32_t pStart, int32_t length,
const uint16_t *q, int32_t qStart, int32_t blockLength) {
// Ensure that we do not even partially get past length.
@@ -655,30 +623,9 @@ int32_t findAllSameBlock(const uint32_t *p, int32_t start, int32_t limit,
* Look for maximum overlap of the beginning of the other block
* with the previous, adjacent block.
*/
int32_t getOverlap(const uint32_t *p, int32_t length,
const uint32_t *q, int32_t qStart, int32_t blockLength) {
int32_t overlap = blockLength - 1;
U_ASSERT(overlap <= length);
q += qStart;
while (overlap > 0 && !equalBlocks(p + (length - overlap), q, overlap)) {
--overlap;
}
return overlap;
}
int32_t getOverlap(const uint16_t *p, int32_t length,
const uint32_t *q, int32_t qStart, int32_t blockLength) {
int32_t overlap = blockLength - 1;
U_ASSERT(overlap <= length);
q += qStart;
while (overlap > 0 && !equalBlocks(p + (length - overlap), q, overlap)) {
--overlap;
}
return overlap;
}
int32_t getOverlap(const uint16_t *p, int32_t length,
const uint16_t *q, int32_t qStart, int32_t blockLength) {
template<typename UIntA, typename UIntB>
int32_t getOverlap(const UIntA *p, int32_t length,
const UIntB *q, int32_t qStart, int32_t blockLength) {
int32_t overlap = blockLength - 1;
U_ASSERT(overlap <= length);
q += qStart;
@@ -807,6 +754,171 @@ private:
int32_t refCounts[CAPACITY];
};
// Custom hash table for mixed-value blocks to be found anywhere in the
// compacted data or index so far.
class MixedBlocks {
public:
MixedBlocks() {}
~MixedBlocks() {
uprv_free(table);
}
bool init(int32_t maxLength, int32_t newBlockLength) {
// We store actual data indexes + 1 to reserve 0 for empty entries.
int32_t maxDataIndex = maxLength - newBlockLength + 1;
int32_t newLength;
if (maxDataIndex <= 0xfff) { // 4k
newLength = 6007;
shift = 12;
mask = 0xfff;
} else if (maxDataIndex <= 0x7fff) { // 32k
newLength = 50021;
shift = 15;
mask = 0x7fff;
} else if (maxDataIndex <= 0x1ffff) { // 128k
newLength = 200003;
shift = 17;
mask = 0x1ffff;
} else {
// maxDataIndex up to around MAX_DATA_LENGTH, ca. 1.1M
newLength = 1500007;
shift = 21;
mask = 0x1fffff;
}
if (newLength > capacity) {
uprv_free(table);
table = (uint32_t *)uprv_malloc(newLength * 4);
if (table == nullptr) {
return false;
}
capacity = newLength;
}
length = newLength;
uprv_memset(table, 0, length * 4);
blockLength = newBlockLength;
return true;
}
template<typename UInt>
void extend(const UInt *data, int32_t minStart, int32_t prevDataLength, int32_t newDataLength) {
int32_t start = prevDataLength - blockLength;
if (start >= minStart) {
++start; // Skip the last block that we added last time.
} else {
start = minStart; // Begin with the first full block.
}
for (int32_t end = newDataLength - blockLength; start <= end; ++start) {
uint32_t hashCode = makeHashCode(data, start);
addEntry(data, start, hashCode, start);
}
}
template<typename UIntA, typename UIntB>
int32_t findBlock(const UIntA *data, const UIntB *blockData, int32_t blockStart) const {
uint32_t hashCode = makeHashCode(blockData, blockStart);
int32_t entryIndex = findEntry(data, blockData, blockStart, hashCode);
if (entryIndex >= 0) {
return (table[entryIndex] & mask) - 1;
} else {
return -1;
}
}
int32_t findAllSameBlock(const uint32_t *data, uint32_t blockValue) const {
uint32_t hashCode = makeHashCode(blockValue);
int32_t entryIndex = findEntry(data, blockValue, hashCode);
if (entryIndex >= 0) {
return (table[entryIndex] & mask) - 1;
} else {
return -1;
}
}
private:
template<typename UInt>
uint32_t makeHashCode(const UInt *blockData, int32_t blockStart) const {
int32_t blockLimit = blockStart + blockLength;
uint32_t hashCode = blockData[blockStart++];
do {
hashCode = 37 * hashCode + blockData[blockStart++];
} while (blockStart < blockLimit);
return hashCode;
}
uint32_t makeHashCode(uint32_t blockValue) const {
uint32_t hashCode = blockValue;
for (int32_t i = 1; i < blockLength; ++i) {
hashCode = 37 * hashCode + blockValue;
}
return hashCode;
}
template<typename UInt>
void addEntry(const UInt *data, int32_t blockStart, uint32_t hashCode, int32_t dataIndex) {
U_ASSERT(0 <= dataIndex && dataIndex < (int32_t)mask);
int32_t entryIndex = findEntry(data, data, blockStart, hashCode);
if (entryIndex < 0) {
table[~entryIndex] = (hashCode << shift) | (dataIndex + 1);
}
}
template<typename UIntA, typename UIntB>
int32_t findEntry(const UIntA *data, const UIntB *blockData, int32_t blockStart,
uint32_t hashCode) const {
uint32_t shiftedHashCode = hashCode << shift;
int32_t initialEntryIndex = (hashCode % (length - 1)) + 1; // 1..length-1
for (int32_t entryIndex = initialEntryIndex;;) {
uint32_t entry = table[entryIndex];
if (entry == 0) {
return ~entryIndex;
}
if ((entry & ~mask) == shiftedHashCode) {
int32_t dataIndex = (entry & mask) - 1;
if (equalBlocks(data + dataIndex, blockData + blockStart, blockLength)) {
return entryIndex;
}
}
entryIndex = nextIndex(initialEntryIndex, entryIndex);
}
}
int32_t findEntry(const uint32_t *data, uint32_t blockValue, uint32_t hashCode) const {
uint32_t shiftedHashCode = hashCode << shift;
int32_t initialEntryIndex = (hashCode % (length - 1)) + 1; // 1..length-1
for (int32_t entryIndex = initialEntryIndex;;) {
uint32_t entry = table[entryIndex];
if (entry == 0) {
return ~entryIndex;
}
if ((entry & ~mask) == shiftedHashCode) {
int32_t dataIndex = (entry & mask) - 1;
if (allValuesSameAs(data + dataIndex, blockLength, blockValue)) {
return entryIndex;
}
}
entryIndex = nextIndex(initialEntryIndex, entryIndex);
}
}
inline int32_t nextIndex(int32_t initialEntryIndex, int32_t entryIndex) const {
// U_ASSERT(0 < initialEntryIndex && initialEntryIndex < length);
return (entryIndex + initialEntryIndex) % length;
}
// Hash table.
// The length is a prime number, larger than the maximum data length.
// The "shift" lower bits store a data index + 1.
// The remaining upper bits store a partial hashCode of the block data values.
uint32_t *table = nullptr;
int32_t capacity = 0;
int32_t length = 0;
int32_t shift = 0;
uint32_t mask = 0;
int32_t blockLength = 0;
};
int32_t MutableCodePointTrie::compactWholeDataBlocks(int32_t fastILimit, AllSameBlocks &allSameBlocks) {
#ifdef UCPTRIE_DEBUG
bool overflow = false;
@@ -962,8 +1074,9 @@ void printBlock(const uint32_t *block, int32_t blockLength, uint32_t value,
*
* It does not try to find an optimal order of writing, deduplicating, and overlapping blocks.
*/
int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
uint32_t *newData, int32_t dataNullIndex) {
int32_t MutableCodePointTrie::compactData(
int32_t fastILimit, uint32_t *newData, int32_t newDataCapacity,
int32_t dataNullIndex, MixedBlocks &mixedBlocks, UErrorCode &errorCode) {
#ifdef UCPTRIE_DEBUG
int32_t countSame=0, sumOverlaps=0;
bool printData = dataLength == 29088 /* line.brk */ ||
@@ -983,8 +1096,14 @@ int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
#endif
}
int32_t iLimit = highStart >> UCPTRIE_SHIFT_3;
int32_t blockLength = UCPTRIE_FAST_DATA_BLOCK_LENGTH;
if (!mixedBlocks.init(newDataCapacity, blockLength)) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return 0;
}
mixedBlocks.extend(newData, 0, 0, newDataLength);
int32_t iLimit = highStart >> UCPTRIE_SHIFT_3;
int32_t inc = SMALL_DATA_BLOCKS_PER_BMP_BLOCK;
int32_t fastLength = 0;
for (int32_t i = ASCII_I_LIMIT; i < iLimit; i += inc) {
@@ -992,12 +1111,17 @@ int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
blockLength = UCPTRIE_SMALL_DATA_BLOCK_LENGTH;
inc = 1;
fastLength = newDataLength;
if (!mixedBlocks.init(newDataCapacity, blockLength)) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return 0;
}
mixedBlocks.extend(newData, 0, 0, newDataLength);
}
if (flags[i] == ALL_SAME) {
uint32_t value = index[i];
int32_t n;
// Find an earlier part of the data array of length blockLength
// that is filled with this value.
int32_t n = mixedBlocks.findAllSameBlock(newData, value);
// If we find a match, and the current block is the data null block,
// and it is not a fast block but matches the start of a fast block,
// then we need to continue looking.
@@ -1005,12 +1129,10 @@ int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
// and not all of the rest of the fast block is filled with this value.
// Otherwise trie.getRange() would detect that the fast block starts at
// dataNullOffset and assume incorrectly that it is filled with the null value.
for (int32_t start = 0;
(n = findAllSameBlock(newData, start, newDataLength,
value, blockLength)) >= 0 &&
i == dataNullIndex && i >= fastILimit && n < fastLength &&
isStartOfSomeFastBlock(n, index, fastILimit);
start = n + 1) {}
while (n >= 0 && i == dataNullIndex && i >= fastILimit && n < fastLength &&
isStartOfSomeFastBlock(n, index, fastILimit)) {
n = findAllSameBlock(newData, n + 1, newDataLength, value, blockLength);
}
if (n >= 0) {
DEBUG_DO(++countSame);
index[i] = n;
@@ -1023,14 +1145,16 @@ int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
}
#endif
index[i] = newDataLength - n;
int32_t prevDataLength = newDataLength;
while (n < blockLength) {
newData[newDataLength++] = value;
++n;
}
mixedBlocks.extend(newData, 0, prevDataLength, newDataLength);
}
} else if (flags[i] == MIXED) {
const uint32_t *block = data + index[i];
int32_t n = findSameBlock(newData, 0, newDataLength, block, 0, blockLength);
int32_t n = mixedBlocks.findBlock(newData, block, 0);
if (n >= 0) {
DEBUG_DO(++countSame);
index[i] = n;
@@ -1043,9 +1167,11 @@ int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
}
#endif
index[i] = newDataLength - n;
int32_t prevDataLength = newDataLength;
while (n < blockLength) {
newData[newDataLength++] = block[n++];
}
mixedBlocks.extend(newData, 0, prevDataLength, newDataLength);
}
} else /* SAME_AS */ {
uint32_t j = index[i];
@@ -1061,7 +1187,8 @@ int32_t MutableCodePointTrie::compactData(int32_t fastILimit,
return newDataLength;
}
int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &errorCode) {
int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, MixedBlocks &mixedBlocks,
UErrorCode &errorCode) {
int32_t fastIndexLength = fastILimit >> (UCPTRIE_FAST_SHIFT - UCPTRIE_SHIFT_3);
if ((highStart >> UCPTRIE_FAST_SHIFT) <= fastIndexLength) {
// Only the linear fast index, no multi-stage index tables.
@@ -1095,6 +1222,12 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
}
}
if (!mixedBlocks.init(fastIndexLength, UCPTRIE_INDEX_3_BLOCK_LENGTH)) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return 0;
}
mixedBlocks.extend(fastIndex, 0, 0, fastIndexLength);
// Examine index-3 blocks. For each determine one of:
// - same as the index-3 null block
// - same as a fast-index block
@@ -1105,6 +1238,7 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
// Also determine an upper limit for the index-3 table length.
int32_t index3Capacity = 0;
i3FirstNull = index3NullOffset;
bool hasLongI3Blocks = false;
// If the fast index covers the whole BMP, then
// the multi-stage index is only for supplementary code points.
// Otherwise, the multi-stage index covers all of Unicode.
@@ -1129,13 +1263,13 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
index3Capacity += UCPTRIE_INDEX_3_BLOCK_LENGTH;
} else {
index3Capacity += INDEX_3_18BIT_BLOCK_LENGTH;
hasLongI3Blocks = true;
}
i3FirstNull = 0;
}
} else {
if (oredI3 <= 0xffff) {
int32_t n = findSameBlock(fastIndex, 0, fastIndexLength,
index, i, UCPTRIE_INDEX_3_BLOCK_LENGTH);
int32_t n = mixedBlocks.findBlock(fastIndex, index, i);
if (n >= 0) {
flags[i] = I3_BMP;
index[i] = n;
@@ -1146,6 +1280,7 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
} else {
flags[i] = I3_18;
index3Capacity += INDEX_3_18BIT_BLOCK_LENGTH;
hasLongI3Blocks = true;
}
}
i = j;
@@ -1166,6 +1301,18 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
}
uprv_memcpy(index16, fastIndex, fastIndexLength * 2);
if (!mixedBlocks.init(index16Capacity, UCPTRIE_INDEX_3_BLOCK_LENGTH)) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return 0;
}
MixedBlocks longI3Blocks;
if (hasLongI3Blocks) {
if (!longI3Blocks.init(index16Capacity, INDEX_3_18BIT_BLOCK_LENGTH)) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
return 0;
}
}
// Compact the index-3 table and write an uncompacted version of the index-2 table.
uint16_t index2[UNICODE_LIMIT >> UCPTRIE_SHIFT_2]; // index2Capacity
int32_t i2Length = 0;
@@ -1185,8 +1332,7 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
} else if (f == I3_BMP) {
i3 = index[i];
} else if (f == I3_16) {
int32_t n = findSameBlock(index16, index3Start, indexLength,
index, i, UCPTRIE_INDEX_3_BLOCK_LENGTH);
int32_t n = mixedBlocks.findBlock(index16, index, i);
if (n >= 0) {
i3 = n;
} else {
@@ -1198,12 +1344,18 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
index, i, UCPTRIE_INDEX_3_BLOCK_LENGTH);
}
i3 = indexLength - n;
int32_t prevIndexLength = indexLength;
while (n < UCPTRIE_INDEX_3_BLOCK_LENGTH) {
index16[indexLength++] = index[i + n++];
}
mixedBlocks.extend(index16, index3Start, prevIndexLength, indexLength);
if (hasLongI3Blocks) {
longI3Blocks.extend(index16, index3Start, prevIndexLength, indexLength);
}
}
} else {
U_ASSERT(f == I3_18);
U_ASSERT(hasLongI3Blocks);
// Encode an index-3 block that contains one or more data indexes exceeding 16 bits.
int32_t j = i;
int32_t jLimit = i + UCPTRIE_INDEX_3_BLOCK_LENGTH;
@@ -1236,8 +1388,7 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
index16[k++] = v;
index16[k - 9] = upperBits;
} while (j < jLimit);
int32_t n = findSameBlock(index16, index3Start, indexLength,
index16, indexLength, INDEX_3_18BIT_BLOCK_LENGTH);
int32_t n = longI3Blocks.findBlock(index16, index16, indexLength);
if (n >= 0) {
i3 = n | 0x8000;
} else {
@@ -1249,6 +1400,7 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
index16, indexLength, INDEX_3_18BIT_BLOCK_LENGTH);
}
i3 = (indexLength - n) | 0x8000;
int32_t prevIndexLength = indexLength;
if (n > 0) {
int32_t start = indexLength;
while (n < INDEX_3_18BIT_BLOCK_LENGTH) {
@@ -1257,6 +1409,10 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
} else {
indexLength += INDEX_3_18BIT_BLOCK_LENGTH;
}
mixedBlocks.extend(index16, index3Start, prevIndexLength, indexLength);
if (hasLongI3Blocks) {
longI3Blocks.extend(index16, index3Start, prevIndexLength, indexLength);
}
}
}
if (index3NullOffset < 0 && i3FirstNull >= 0) {
@@ -1279,16 +1435,23 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
}
// Compact the index-2 table and write the index-1 table.
static_assert(UCPTRIE_INDEX_2_BLOCK_LENGTH == UCPTRIE_INDEX_3_BLOCK_LENGTH,
"must re-init mixedBlocks");
int32_t blockLength = UCPTRIE_INDEX_2_BLOCK_LENGTH;
int32_t i1 = fastIndexLength;
for (int32_t i = 0; i < i2Length; i += blockLength) {
if ((i2Length - i) < blockLength) {
int32_t n;
if ((i2Length - i) >= blockLength) {
// normal block
U_ASSERT(blockLength == UCPTRIE_INDEX_2_BLOCK_LENGTH);
n = mixedBlocks.findBlock(index16, index2, i);
} else {
// highStart is inside the last index-2 block. Shorten it.
blockLength = i2Length - i;
n = findSameBlock(index16, index3Start, indexLength,
index2, i, blockLength);
}
int32_t i2;
int32_t n = findSameBlock(index16, index3Start, indexLength,
index2, i, blockLength);
if (n >= 0) {
i2 = n;
} else {
@@ -1299,9 +1462,11 @@ int32_t MutableCodePointTrie::compactIndex(int32_t fastILimit, UErrorCode &error
n = getOverlap(index16, indexLength, index2, i, blockLength);
}
i2 = indexLength - n;
int32_t prevIndexLength = indexLength;
while (n < blockLength) {
index16[indexLength++] = index2[i + n++];
}
mixedBlocks.extend(index16, index3Start, prevIndexLength, indexLength);
}
// Set the index-1 table entry.
index16[i1++] = i2;
@@ -1369,7 +1534,11 @@ int32_t MutableCodePointTrie::compactTrie(int32_t fastILimit, UErrorCode &errorC
uprv_memcpy(newData, asciiData, sizeof(asciiData));
int32_t dataNullIndex = allSameBlocks.findMostUsed();
int32_t newDataLength = compactData(fastILimit, newData, dataNullIndex);
MixedBlocks mixedBlocks;
int32_t newDataLength = compactData(fastILimit, newData, newDataCapacity,
dataNullIndex, mixedBlocks, errorCode);
if (U_FAILURE(errorCode)) { return 0; }
U_ASSERT(newDataLength <= newDataCapacity);
uprv_free(data);
data = newData;
@@ -1394,7 +1563,7 @@ int32_t MutableCodePointTrie::compactTrie(int32_t fastILimit, UErrorCode &errorC
dataNullOffset = UCPTRIE_NO_DATA_NULL_OFFSET;
}
int32_t indexLength = compactIndex(fastILimit, errorCode);
int32_t indexLength = compactIndex(fastILimit, mixedBlocks, errorCode);
highStart = realHighStart;
return indexLength;
}
+11 -3
View File
@@ -54,15 +54,23 @@ U_NAMESPACE_END
#include <atomic>
U_NAMESPACE_BEGIN
// Export an explicit template instantiation of std::atomic<int32_t>.
// When building DLLs for Windows this is required as it is used as a data member of the exported SharedObject class.
// See digitlst.h, pluralaffix.h, datefmt.h, and others for similar examples.
#if U_PF_WINDOWS <= U_PLATFORM && U_PLATFORM <= U_PF_CYGWIN
#if U_PF_WINDOWS <= U_PLATFORM && U_PLATFORM <= U_PF_CYGWIN && !defined(U_IN_DOXYGEN)
#if defined(__clang__)
// Suppress the warning that the explicit instantiation after explicit specialization has no effect.
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Winstantiation-after-specialization"
#endif
template struct U_COMMON_API std::atomic<int32_t>;
#if defined(__clang__)
#pragma clang diagnostic pop
#endif
#endif
U_NAMESPACE_BEGIN
typedef std::atomic<int32_t> u_atomic_int32_t;
#define ATOMIC_INT32_T_INITIALIZER(val) ATOMIC_VAR_INIT(val)
+39 -22
View File
@@ -27,7 +27,6 @@ U_NAMESPACE_BEGIN
// Forward Declarations.
class BMPSet;
class CharacterProperties;
class ParsePosition;
class RBBIRuleScanner;
class SymbolTable;
@@ -276,14 +275,23 @@ class RuleCharacterIterator;
* @stable ICU 2.0
*/
class U_COMMON_API UnicodeSet U_FINAL : public UnicodeFilter {
private:
/**
* Enough for sets with few ranges.
* For example, White_Space has 10 ranges, list length 21.
*/
static constexpr int32_t INITIAL_CAPACITY = 25;
// fFlags constant
static constexpr uint8_t kIsBogus = 1; // This set is bogus (i.e. not valid)
int32_t len; // length of list used; 0 <= len <= capacity
int32_t capacity; // capacity of list
UChar32* list; // MUST be terminated with HIGH
BMPSet *bmpSet; // The set is frozen iff either bmpSet or stringSpan is not NULL.
UChar32* buffer; // internal buffer, may be NULL
int32_t bufferCapacity; // capacity of buffer
int32_t patLen;
UChar32* list = stackList; // MUST be terminated with HIGH
int32_t capacity = INITIAL_CAPACITY; // capacity of list
int32_t len = 1; // length of list used; 1 <= len <= capacity
uint8_t fFlags = 0; // Bit flag (see constants above)
BMPSet *bmpSet = nullptr; // The set is frozen iff either bmpSet or stringSpan is not NULL.
UChar32* buffer = nullptr; // internal buffer, may be NULL
int32_t bufferCapacity = 0; // capacity of buffer
/**
* The pattern representation of this set. This may not be the
@@ -294,15 +302,19 @@ class U_COMMON_API UnicodeSet U_FINAL : public UnicodeFilter {
* indicating that toPattern() must generate a pattern
* representation from the inversion list.
*/
char16_t *pat;
UVector* strings; // maintained in sorted order
UnicodeSetStringSpan *stringSpan;
char16_t *pat = nullptr;
int32_t patLen = 0;
UVector* strings = nullptr; // maintained in sorted order
UnicodeSetStringSpan *stringSpan = nullptr;
/**
* Initial list array.
* Avoids some heap allocations, and list is never nullptr.
* Increases the object size a bit.
*/
UChar32 stackList[INITIAL_CAPACITY];
private:
enum { // constants
kIsBogus = 1 // This set is bogus (i.e. not valid)
};
uint8_t fFlags; // Bit flag (see constants above)
public:
/**
* Determine if this object contains a valid set.
@@ -1480,8 +1492,6 @@ private:
friend class USetAccess;
int32_t getStringCount() const;
const UnicodeString* getString(int32_t index) const;
//----------------------------------------------------------------
@@ -1528,13 +1538,18 @@ private:
// Implementation: Utility methods
//----------------------------------------------------------------
void ensureCapacity(int32_t newLen, UErrorCode& ec);
static int32_t nextCapacity(int32_t minCapacity);
void ensureBufferCapacity(int32_t newLen, UErrorCode& ec);
bool ensureCapacity(int32_t newLen);
bool ensureBufferCapacity(int32_t newLen);
void swapBuffers(void);
UBool allocateStrings(UErrorCode &status);
UBool hasStrings() const;
int32_t stringsSize() const;
UBool stringsContains(const UnicodeString &s) const;
UnicodeString& _toPattern(UnicodeString& result,
UBool escapeUnprintable) const;
@@ -1614,7 +1629,6 @@ private:
UnicodeString& rebuiltPat,
UErrorCode& ec);
friend class CharacterProperties;
static const UnicodeSet* getInclusions(int32_t src, UErrorCode &status);
/**
@@ -1646,7 +1660,10 @@ private:
/**
* Set the new pattern to cache.
*/
void setPattern(const UnicodeString& newPat);
void setPattern(const UnicodeString& newPat) {
setPattern(newPat.getBuffer(), newPat.length());
}
void setPattern(const char16_t *newPat, int32_t newPatLen);
/**
* Release existing cached pattern.
*/
+2 -3
View File
@@ -34,7 +34,7 @@
#if !U_DISABLE_RENAMING
// Disable Renaming for Visual Studio's IntelliSense feature, so that 'Go-to-Definition' (F12) will work.
#if defined(_MSC_VER) && !defined(__INTELLISENSE__)
#if !(defined(_MSC_VER) && defined(__INTELLISENSE__))
/* We need the U_ICU_ENTRY_POINT_RENAME definition. There's a default one in unicode/uvernum.h we can use, but we will give
the platform a chance to define it first.
@@ -110,7 +110,6 @@
#define _UTF7Data U_ICU_ENTRY_POINT_RENAME(_UTF7Data)
#define _UTF8Data U_ICU_ENTRY_POINT_RENAME(_UTF8Data)
#define allowedHourFormatsCleanup U_ICU_ENTRY_POINT_RENAME(allowedHourFormatsCleanup)
#define checkImpl U_ICU_ENTRY_POINT_RENAME(checkImpl)
#define cmemory_cleanup U_ICU_ENTRY_POINT_RENAME(cmemory_cleanup)
#define dayPeriodRulesCleanup U_ICU_ENTRY_POINT_RENAME(dayPeriodRulesCleanup)
#define deleteAllowedHourFormats U_ICU_ENTRY_POINT_RENAME(deleteAllowedHourFormats)
@@ -1858,7 +1857,7 @@
#define ztrans_setTime U_ICU_ENTRY_POINT_RENAME(ztrans_setTime)
#define ztrans_setTo U_ICU_ENTRY_POINT_RENAME(ztrans_setTo)
#endif /* defined(_MSC_VER) && !defined(__INTELLISENSE__) */
#endif /* !(defined(_MSC_VER) && defined(__INTELLISENSE__)) */
#endif /* U_DISABLE_RENAMING */
#endif /* URENAME_H */
+3 -3
View File
@@ -66,7 +66,7 @@
* This value will change in the subsequent releases of ICU
* @stable ICU 2.6
*/
#define U_ICU_VERSION_MINOR_NUM 1
#define U_ICU_VERSION_MINOR_NUM 2
/** The current ICU patchlevel version as an integer.
* This value will change in the subsequent releases of ICU
@@ -121,7 +121,7 @@
* This value will change in the subsequent releases of ICU
* @stable ICU 2.4
*/
#define U_ICU_VERSION "63.1"
#define U_ICU_VERSION "63.2"
/**
* The current ICU library major version number as a string, for library name suffixes.
@@ -140,7 +140,7 @@
/** Data version in ICU4C.
* @internal ICU 4.4 Internal Use Only
**/
#define U_ICU_DATA_VERSION "63.1"
#define U_ICU_DATA_VERSION "63.2"
#endif /* U_HIDE_INTERNAL_API */
/*===========================================================================
+237 -255
View File
@@ -14,6 +14,7 @@
#include "unicode/parsepos.h"
#include "unicode/symtable.h"
#include "unicode/uniset.h"
#include "unicode/ustring.h"
#include "unicode/utf8.h"
#include "unicode/utf16.h"
#include "ruleiter.h"
@@ -53,11 +54,8 @@
// LOW <= all valid values. ZERO for codepoints
#define UNICODESET_LOW 0x000000
// initial storage. Must be >= 0
#define START_EXTRA 16
// extra amount for growth. Must be >= 0
#define GROW_EXTRA START_EXTRA
/** Max list [0, 1, 2, ..., max code point, HIGH] */
constexpr int32_t MAX_LENGTH = UNICODESET_HIGH + 1;
U_NAMESPACE_BEGIN
@@ -137,6 +135,18 @@ static int8_t U_CALLCONV compareUnicodeString(UElement t1, UElement t2) {
return a.compare(b);
}
UBool UnicodeSet::hasStrings() const {
return strings != nullptr && !strings->isEmpty();
}
int32_t UnicodeSet::stringsSize() const {
return strings == nullptr ? 0 : strings->size();
}
UBool UnicodeSet::stringsContains(const UnicodeString &s) const {
return strings != nullptr && strings->contains((void*) &s);
}
//----------------------------------------------------------------
// Constructors &c
//----------------------------------------------------------------
@@ -144,24 +154,8 @@ static int8_t U_CALLCONV compareUnicodeString(UElement t1, UElement t2) {
/**
* Constructs an empty set.
*/
UnicodeSet::UnicodeSet() :
len(1), capacity(1 + START_EXTRA), list(0), bmpSet(0), buffer(0),
bufferCapacity(0), patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
UErrorCode status = U_ZERO_ERROR;
allocateStrings(status);
if (U_FAILURE(status)) {
setToBogus(); // If memory allocation failed, set to bogus state.
return;
}
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
if(list!=NULL){
list[0] = UNICODESET_HIGH;
} else { // If memory allocation failed, set to bogus state.
setToBogus();
return;
}
UnicodeSet::UnicodeSet() {
list[0] = UNICODESET_HIGH;
_dbgct(this);
}
@@ -172,89 +166,39 @@ UnicodeSet::UnicodeSet() :
* @param start first character, inclusive, of range
* @param end last character, inclusive, of range
*/
UnicodeSet::UnicodeSet(UChar32 start, UChar32 end) :
len(1), capacity(1 + START_EXTRA), list(0), bmpSet(0), buffer(0),
bufferCapacity(0), patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
UErrorCode status = U_ZERO_ERROR;
allocateStrings(status);
if (U_FAILURE(status)) {
setToBogus(); // If memory allocation failed, set to bogus state.
return;
}
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
if(list!=NULL){
list[0] = UNICODESET_HIGH;
complement(start, end);
} else { // If memory allocation failed, set to bogus state.
setToBogus();
return;
}
UnicodeSet::UnicodeSet(UChar32 start, UChar32 end) {
list[0] = UNICODESET_HIGH;
add(start, end);
_dbgct(this);
}
/**
* Constructs a set that is identical to the given UnicodeSet.
*/
UnicodeSet::UnicodeSet(const UnicodeSet& o) :
UnicodeFilter(o),
len(0), capacity(o.isFrozen() ? o.len : o.len + GROW_EXTRA), list(0),
bmpSet(0),
buffer(0), bufferCapacity(0),
patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
UErrorCode status = U_ZERO_ERROR;
allocateStrings(status);
if (U_FAILURE(status)) {
setToBogus(); // If memory allocation failed, set to bogus state.
return;
}
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
if(list!=NULL){
*this = o;
} else { // If memory allocation failed, set to bogus state.
setToBogus();
return;
}
UnicodeSet::UnicodeSet(const UnicodeSet& o) : UnicodeFilter(o) {
*this = o;
_dbgct(this);
}
// Copy-construct as thawed.
UnicodeSet::UnicodeSet(const UnicodeSet& o, UBool /* asThawed */) :
UnicodeFilter(o),
len(0), capacity(o.len + GROW_EXTRA), list(0),
bmpSet(0),
buffer(0), bufferCapacity(0),
patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
UErrorCode status = U_ZERO_ERROR;
allocateStrings(status);
if (U_FAILURE(status)) {
setToBogus(); // If memory allocation failed, set to bogus state.
return;
}
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
if(list!=NULL){
UnicodeSet::UnicodeSet(const UnicodeSet& o, UBool /* asThawed */) : UnicodeFilter(o) {
if (ensureCapacity(o.len)) {
// *this = o except for bmpSet and stringSpan
len = o.len;
uprv_memcpy(list, o.list, (size_t)len*sizeof(UChar32));
if (strings != NULL && o.strings != NULL) {
strings->assign(*o.strings, cloneUnicodeString, status);
} else { // Invalid strings.
setToBogus();
return;
if (o.hasStrings()) {
UErrorCode status = U_ZERO_ERROR;
if (!allocateStrings(status) ||
(strings->assign(*o.strings, cloneUnicodeString, status), U_FAILURE(status))) {
setToBogus();
return;
}
}
if (o.pat) {
setPattern(UnicodeString(o.pat, o.patLen));
setPattern(o.pat, o.patLen);
}
} else { // If memory allocation failed, set to bogus state.
setToBogus();
return;
_dbgct(this);
}
_dbgct(this);
}
/**
@@ -262,9 +206,11 @@ UnicodeSet::UnicodeSet(const UnicodeSet& o, UBool /* asThawed */) :
*/
UnicodeSet::~UnicodeSet() {
_dbgdt(this); // first!
uprv_free(list);
if (list != stackList) {
uprv_free(list);
}
delete bmpSet;
if (buffer) {
if (buffer != stackList) {
uprv_free(buffer);
}
delete strings;
@@ -290,32 +236,30 @@ UnicodeSet& UnicodeSet::copyFrom(const UnicodeSet& o, UBool asThawed) {
setToBogus();
return *this;
}
UErrorCode ec = U_ZERO_ERROR;
ensureCapacity(o.len, ec);
if (U_FAILURE(ec)) {
if (!ensureCapacity(o.len)) {
// ensureCapacity will mark the UnicodeSet as Bogus if OOM failure happens.
return *this;
}
len = o.len;
uprv_memcpy(list, o.list, (size_t)len*sizeof(UChar32));
if (o.bmpSet == NULL || asThawed) {
bmpSet = NULL;
} else {
if (o.bmpSet != nullptr && !asThawed) {
bmpSet = new BMPSet(*o.bmpSet, list, len);
if (bmpSet == NULL) { // Check for memory allocation error.
setToBogus();
return *this;
}
}
if (strings != NULL && o.strings != NULL) {
strings->assign(*o.strings, cloneUnicodeString, ec);
} else { // Invalid strings.
setToBogus();
return *this;
if (o.hasStrings()) {
UErrorCode status = U_ZERO_ERROR;
if ((strings == nullptr && !allocateStrings(status)) ||
(strings->assign(*o.strings, cloneUnicodeString, status), U_FAILURE(status))) {
setToBogus();
return *this;
}
} else if (hasStrings()) {
strings->removeAllElements();
}
if (o.stringSpan == NULL || asThawed) {
stringSpan = NULL;
} else {
if (o.stringSpan != nullptr && !asThawed) {
stringSpan = new UnicodeSetStringSpan(*o.stringSpan, *strings);
if (stringSpan == NULL) { // Check for memory allocation error.
setToBogus();
@@ -324,7 +268,7 @@ UnicodeSet& UnicodeSet::copyFrom(const UnicodeSet& o, UBool asThawed) {
}
releasePattern();
if (o.pat) {
setPattern(UnicodeString(o.pat, o.patLen));
setPattern(o.pat, o.patLen);
}
return *this;
}
@@ -357,7 +301,8 @@ UBool UnicodeSet::operator==(const UnicodeSet& o) const {
for (int32_t i = 0; i < len; ++i) {
if (list[i] != o.list[i]) return FALSE;
}
if (*strings != *o.strings) return FALSE;
if (hasStrings() != o.hasStrings()) { return FALSE; }
if (hasStrings() && *strings != *o.strings) return FALSE;
return TRUE;
}
@@ -393,7 +338,7 @@ int32_t UnicodeSet::size(void) const {
for (int32_t i = 0; i < count; ++i) {
n += getRangeEnd(i) - getRangeStart(i) + 1;
}
return n + strings->size();
return n + stringsSize();
}
/**
@@ -402,7 +347,7 @@ int32_t UnicodeSet::size(void) const {
* @return <tt>true</tt> if this set contains no elements.
*/
UBool UnicodeSet::isEmpty(void) const {
return len == 1 && strings->size() == 0;
return len == 1 && !hasStrings();
}
/**
@@ -502,7 +447,7 @@ UBool UnicodeSet::contains(const UnicodeString& s) const {
if (s.length() == 0) return FALSE;
int32_t cp = getSingleCP(s);
if (cp < 0) {
return strings->contains((void*) &s);
return stringsContains(s);
} else {
return contains((UChar32) cp);
}
@@ -524,8 +469,7 @@ UBool UnicodeSet::containsAll(const UnicodeSet& c) const {
return FALSE;
}
}
if (!strings->containsAll(*c.strings)) return FALSE;
return TRUE;
return !c.hasStrings() || (strings != nullptr && strings->containsAll(*c.strings));
}
/**
@@ -571,8 +515,7 @@ UBool UnicodeSet::containsNone(const UnicodeSet& c) const {
return FALSE;
}
}
if (!strings->containsNone(*c.strings)) return FALSE;
return TRUE;
return strings == nullptr || !c.hasStrings() || strings->containsNone(*c.strings);
}
/**
@@ -613,7 +556,7 @@ UBool UnicodeSet::matchesIndexValue(uint8_t v) const {
return TRUE;
}
}
if (strings->size() != 0) {
if (hasStrings()) {
for (i=0; i<strings->size(); ++i) {
const UnicodeString& s = *(const UnicodeString*)strings->elementAt(i);
//if (s.length() == 0) {
@@ -648,7 +591,7 @@ UMatchDegree UnicodeSet::matches(const Replaceable& text,
return U_MISMATCH;
}
} else {
if (strings->size() != 0) { // try strings first
if (hasStrings()) { // try strings first
// might separate forward and backward loops later
// for now they are combined
@@ -849,7 +792,39 @@ UnicodeSet& UnicodeSet::set(UChar32 start, UChar32 end) {
*/
UnicodeSet& UnicodeSet::add(UChar32 start, UChar32 end) {
if (pinCodePoint(start) < pinCodePoint(end)) {
UChar32 range[3] = { start, end+1, UNICODESET_HIGH };
UChar32 limit = end + 1;
// Fast path for adding a new range after the last one.
// Odd list length: [..., lastStart, lastLimit, HIGH]
if ((len & 1) != 0) {
// If the list is empty, set lastLimit low enough to not be adjacent to 0.
UChar32 lastLimit = len == 1 ? -2 : list[len - 2];
if (lastLimit <= start && !isFrozen() && !isBogus()) {
if (lastLimit == start) {
// Extend the last range.
list[len - 2] = limit;
if (limit == UNICODESET_HIGH) {
--len;
}
} else {
list[len - 1] = start;
if (limit < UNICODESET_HIGH) {
if (ensureCapacity(len + 2)) {
list[len++] = limit;
list[len++] = UNICODESET_HIGH;
}
} else { // limit == UNICODESET_HIGH
if (ensureCapacity(len + 1)) {
list[len++] = UNICODESET_HIGH;
}
}
}
releasePattern();
return *this;
}
}
// This is slow. Could be much faster using findCodePoint(start)
// and modifying the list, dealing with adjacent & overlapping ranges.
UChar32 range[3] = { start, limit, UNICODESET_HIGH };
add(range, 2, 0);
} else if (start == end) {
add(start);
@@ -918,9 +893,7 @@ UnicodeSet& UnicodeSet::add(UChar32 c) {
list[i] = c;
// if we touched the HIGH mark, then add a new one
if (c == (UNICODESET_HIGH - 1)) {
UErrorCode status = U_ZERO_ERROR;
ensureCapacity(len+1, status);
if (U_FAILURE(status)) {
if (!ensureCapacity(len+1)) {
// ensureCapacity will mark the object as Bogus if OOM failure happens.
return *this;
}
@@ -964,21 +937,13 @@ UnicodeSet& UnicodeSet::add(UChar32 c) {
// ^
// list[i]
UErrorCode status = U_ZERO_ERROR;
ensureCapacity(len+2, status);
if (U_FAILURE(status)) {
if (!ensureCapacity(len+2)) {
// ensureCapacity will mark the object as Bogus if OOM failure happens.
return *this;
}
//for (int32_t k=len-1; k>=i; --k) {
// list[k+2] = list[k];
//}
UChar32* src = list + len;
UChar32* dst = src + 2;
UChar32* srclimit = list + i;
while (src > srclimit) *(--dst) = *(--src);
UChar32 *p = list + i;
uprv_memmove(p + 2, p, (len - i) * sizeof(*p));
list[i] = c;
list[i+1] = c+1;
len += 2;
@@ -1014,7 +979,7 @@ UnicodeSet& UnicodeSet::add(const UnicodeString& s) {
if (s.length() == 0 || isFrozen() || isBogus()) return *this;
int32_t cp = getSingleCP(s);
if (cp < 0) {
if (!strings->contains((void*) &s)) {
if (!stringsContains(s)) {
_add(s);
releasePattern();
}
@@ -1033,12 +998,16 @@ void UnicodeSet::_add(const UnicodeString& s) {
if (isFrozen() || isBogus()) {
return;
}
UErrorCode ec = U_ZERO_ERROR;
if (strings == nullptr && !allocateStrings(ec)) {
setToBogus();
return;
}
UnicodeString* t = new UnicodeString(s);
if (t == NULL) { // Check for memory allocation error.
setToBogus();
return;
}
UErrorCode ec = U_ZERO_ERROR;
strings->sortedInsert(t, compareUnicodeString, ec);
if (U_FAILURE(ec)) {
setToBogus();
@@ -1121,7 +1090,10 @@ UnicodeSet& UnicodeSet::removeAll(const UnicodeString& s) {
}
UnicodeSet& UnicodeSet::removeAllStrings() {
strings->removeAllElements();
if (!isFrozen() && hasStrings()) {
strings->removeAllElements();
releasePattern();
}
return *this;
}
@@ -1217,8 +1189,9 @@ UnicodeSet& UnicodeSet::remove(const UnicodeString& s) {
if (s.length() == 0 || isFrozen() || isBogus()) return *this;
int32_t cp = getSingleCP(s);
if (cp < 0) {
strings->removeElement((void*) &s);
releasePattern();
if (strings != nullptr && strings->removeElement((void*) &s)) {
releasePattern();
}
} else {
remove((UChar32)cp, (UChar32)cp);
}
@@ -1260,24 +1233,17 @@ UnicodeSet& UnicodeSet::complement(void) {
if (isFrozen() || isBogus()) {
return *this;
}
UErrorCode status = U_ZERO_ERROR;
if (list[0] == UNICODESET_LOW) {
ensureBufferCapacity(len-1, status);
if (U_FAILURE(status)) {
return *this;
}
uprv_memcpy(buffer, list + 1, (size_t)(len-1)*sizeof(UChar32));
uprv_memmove(list, list + 1, (size_t)(len-1)*sizeof(UChar32));
--len;
} else {
ensureBufferCapacity(len+1, status);
if (U_FAILURE(status)) {
if (!ensureCapacity(len+1)) {
return *this;
}
uprv_memcpy(buffer + 1, list, (size_t)len*sizeof(UChar32));
buffer[0] = UNICODESET_LOW;
uprv_memmove(list + 1, list, (size_t)len*sizeof(UChar32));
list[0] = UNICODESET_LOW;
++len;
}
swapBuffers();
releasePattern();
return *this;
}
@@ -1294,7 +1260,7 @@ UnicodeSet& UnicodeSet::complement(const UnicodeString& s) {
if (s.length() == 0 || isFrozen() || isBogus()) return *this;
int32_t cp = getSingleCP(s);
if (cp < 0) {
if (strings->contains((void*) &s)) {
if (stringsContains(s)) {
strings->removeElement((void*) &s);
} else {
_add(s);
@@ -1325,7 +1291,7 @@ UnicodeSet& UnicodeSet::addAll(const UnicodeSet& c) {
if ( c.strings!=NULL ) {
for (int32_t i=0; i<c.strings->size(); ++i) {
const UnicodeString* s = (const UnicodeString*)c.strings->elementAt(i);
if (!strings->contains((void*) s)) {
if (!stringsContains(*s)) {
_add(*s);
}
}
@@ -1347,7 +1313,13 @@ UnicodeSet& UnicodeSet::retainAll(const UnicodeSet& c) {
return *this;
}
retain(c.list, c.len, 0);
strings->retainAll(*c.strings);
if (hasStrings()) {
if (!c.hasStrings()) {
strings->removeAllElements();
} else {
strings->retainAll(*c.strings);
}
}
return *this;
}
@@ -1365,7 +1337,9 @@ UnicodeSet& UnicodeSet::removeAll(const UnicodeSet& c) {
return *this;
}
retain(c.list, c.len, 2);
strings->removeAll(*c.strings);
if (hasStrings() && c.hasStrings()) {
strings->removeAll(*c.strings);
}
return *this;
}
@@ -1383,10 +1357,12 @@ UnicodeSet& UnicodeSet::complementAll(const UnicodeSet& c) {
}
exclusiveOr(c.list, c.len, 0);
for (int32_t i=0; i<c.strings->size(); ++i) {
void* e = c.strings->elementAt(i);
if (!strings->removeElement(e)) {
_add(*(const UnicodeString*)e);
if (c.strings != nullptr) {
for (int32_t i=0; i<c.strings->size(); ++i) {
void* e = c.strings->elementAt(i);
if (strings == nullptr || !strings->removeElement(e)) {
_add(*(const UnicodeString*)e);
}
}
}
return *this;
@@ -1400,18 +1376,14 @@ UnicodeSet& UnicodeSet::clear(void) {
if (isFrozen()) {
return *this;
}
if (list != NULL) {
list[0] = UNICODESET_HIGH;
}
list[0] = UNICODESET_HIGH;
len = 1;
releasePattern();
if (strings != NULL) {
strings->removeAllElements();
}
if (list != NULL && strings != NULL) {
// Remove bogus
fFlags = 0;
}
// Remove bogus
fFlags = 0;
return *this;
}
@@ -1445,10 +1417,6 @@ UChar32 UnicodeSet::getRangeEnd(int32_t index) const {
return list[index*2 + 1] - 1;
}
int32_t UnicodeSet::getStringCount() const {
return strings->size();
}
const UnicodeString* UnicodeSet::getString(int32_t index) const {
return (const UnicodeString*) strings->elementAt(index);
}
@@ -1462,22 +1430,32 @@ UnicodeSet& UnicodeSet::compact() {
return *this;
}
// Delete buffer first to defragment memory less.
if (buffer != NULL) {
if (buffer != stackList) {
uprv_free(buffer);
buffer = NULL;
bufferCapacity = 0;
}
if (len < capacity) {
// Make the capacity equal to len or 1.
// We don't want to realloc of 0 size.
int32_t newCapacity = len + (len == 0);
UChar32* temp = (UChar32*) uprv_realloc(list, sizeof(UChar32) * newCapacity);
if (list == stackList) {
// pass
} else if (len <= INITIAL_CAPACITY) {
uprv_memcpy(stackList, list, len * sizeof(UChar32));
uprv_free(list);
list = stackList;
capacity = INITIAL_CAPACITY;
} else if ((len + 7) < capacity) {
// If we have more than a little unused capacity, shrink it to len.
UChar32* temp = (UChar32*) uprv_realloc(list, sizeof(UChar32) * len);
if (temp) {
list = temp;
capacity = newCapacity;
capacity = len;
}
// else what the heck happened?! We allocated less memory!
// Oh well. We'll keep our original array.
}
if (strings != nullptr && strings->isEmpty()) {
delete strings;
strings = nullptr;
}
return *this;
}
@@ -1488,10 +1466,8 @@ UnicodeSet& UnicodeSet::compact() {
/**
* Deserialize constructor.
*/
UnicodeSet::UnicodeSet(const uint16_t data[], int32_t dataLen, ESerialization serialization, UErrorCode &ec)
: len(1), capacity(1+START_EXTRA), list(0), bmpSet(0), buffer(0),
bufferCapacity(0), patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0) {
UnicodeSet::UnicodeSet(const uint16_t data[], int32_t dataLen, ESerialization serialization,
UErrorCode &ec) {
if(U_FAILURE(ec)) {
setToBogus();
@@ -1506,24 +1482,15 @@ UnicodeSet::UnicodeSet(const uint16_t data[], int32_t dataLen, ESerialization se
return;
}
allocateStrings(ec);
if (U_FAILURE(ec)) {
setToBogus();
return;
}
// bmp?
int32_t headerSize = ((data[0]&0x8000)) ?2:1;
int32_t bmpLength = (headerSize==1)?data[0]:data[1];
len = (((data[0]&0x7FFF)-bmpLength)/2)+bmpLength;
int32_t newLength = (((data[0]&0x7FFF)-bmpLength)/2)+bmpLength;
#ifdef DEBUG_SERIALIZE
printf("dataLen %d headerSize %d bmpLen %d len %d. data[0]=%X/%X/%X/%X\n", dataLen,headerSize,bmpLength,len, data[0],data[1],data[2],data[3]);
printf("dataLen %d headerSize %d bmpLen %d len %d. data[0]=%X/%X/%X/%X\n", dataLen,headerSize,bmpLength,newLength, data[0],data[1],data[2],data[3]);
#endif
capacity = len+1;
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
if(!list || U_FAILURE(ec)) {
setToBogus();
if(!ensureCapacity(newLength + 1)) { // +1 for HIGH
return;
}
// copy bmp
@@ -1535,15 +1502,18 @@ UnicodeSet::UnicodeSet(const uint16_t data[], int32_t dataLen, ESerialization se
#endif
}
// copy smp
for(i=bmpLength;i<len;i++) {
for(i=bmpLength;i<newLength;i++) {
list[i] = ((UChar32)data[headerSize+bmpLength+(i-bmpLength)*2+0] << 16) +
((UChar32)data[headerSize+bmpLength+(i-bmpLength)*2+1]);
#ifdef DEBUG_SERIALIZE
printf("<<32@%d+[%d] %lX\n", headerSize+bmpLength+i, i, list[i]);
#endif
}
// terminator
list[len++]=UNICODESET_HIGH;
U_ASSERT(i == newLength);
if (i == 0 || list[i - 1] != UNICODESET_HIGH) {
list[i++] = UNICODESET_HIGH;
}
len = i;
}
@@ -1664,33 +1634,65 @@ UBool UnicodeSet::allocateStrings(UErrorCode &status) {
return TRUE;
}
void UnicodeSet::ensureCapacity(int32_t newLen, UErrorCode& ec) {
if (newLen <= capacity) {
return;
int32_t UnicodeSet::nextCapacity(int32_t minCapacity) {
// Grow exponentially to reduce the frequency of allocations.
if (minCapacity < INITIAL_CAPACITY) {
return minCapacity + INITIAL_CAPACITY;
} else if (minCapacity <= 2500) {
return 5 * minCapacity;
} else {
int32_t newCapacity = 2 * minCapacity;
if (newCapacity > MAX_LENGTH) {
newCapacity = MAX_LENGTH;
}
return newCapacity;
}
UChar32* temp = (UChar32*) uprv_realloc(list, sizeof(UChar32) * (newLen + GROW_EXTRA));
if (temp == NULL) {
ec = U_MEMORY_ALLOCATION_ERROR;
setToBogus(); // set the object to bogus state if an OOM failure occurred.
return;
}
list = temp;
capacity = newLen + GROW_EXTRA;
// else we keep the original contents on the memory failure.
}
void UnicodeSet::ensureBufferCapacity(int32_t newLen, UErrorCode& ec) {
if (buffer != NULL && newLen <= bufferCapacity)
return;
UChar32* temp = (UChar32*) uprv_realloc(buffer, sizeof(UChar32) * (newLen + GROW_EXTRA));
bool UnicodeSet::ensureCapacity(int32_t newLen) {
if (newLen > MAX_LENGTH) {
newLen = MAX_LENGTH;
}
if (newLen <= capacity) {
return true;
}
int32_t newCapacity = nextCapacity(newLen);
UChar32* temp = (UChar32*) uprv_malloc(newCapacity * sizeof(UChar32));
if (temp == NULL) {
setToBogus(); // set the object to bogus state if an OOM failure occurred.
return false;
}
// Copy only the actual contents.
uprv_memcpy(temp, list, len * sizeof(UChar32));
if (list != stackList) {
uprv_free(list);
}
list = temp;
capacity = newCapacity;
return true;
}
bool UnicodeSet::ensureBufferCapacity(int32_t newLen) {
if (newLen > MAX_LENGTH) {
newLen = MAX_LENGTH;
}
if (newLen <= bufferCapacity) {
return true;
}
int32_t newCapacity = nextCapacity(newLen);
UChar32* temp = (UChar32*) uprv_malloc(newCapacity * sizeof(UChar32));
if (temp == NULL) {
ec = U_MEMORY_ALLOCATION_ERROR;
setToBogus();
return;
return false;
}
// The buffer has no contents to be copied.
// It is always filled from scratch after this call.
if (buffer != stackList) {
uprv_free(buffer);
}
buffer = temp;
bufferCapacity = newLen + GROW_EXTRA;
// else we keep the original contents on the memory failure.
bufferCapacity = newCapacity;
return true;
}
/**
@@ -1727,9 +1729,7 @@ void UnicodeSet::exclusiveOr(const UChar32* other, int32_t otherLen, int8_t pola
if (isFrozen() || isBogus()) {
return;
}
UErrorCode status = U_ZERO_ERROR;
ensureBufferCapacity(len + otherLen, status);
if (U_FAILURE(status)) {
if (!ensureBufferCapacity(len + otherLen)) {
return;
}
@@ -1777,9 +1777,7 @@ void UnicodeSet::add(const UChar32* other, int32_t otherLen, int8_t polarity) {
if (isFrozen() || isBogus() || other==NULL) {
return;
}
UErrorCode status = U_ZERO_ERROR;
ensureBufferCapacity(len + otherLen, status);
if (U_FAILURE(status)) {
if (!ensureBufferCapacity(len + otherLen)) {
return;
}
@@ -1890,9 +1888,7 @@ void UnicodeSet::retain(const UChar32* other, int32_t otherLen, int8_t polarity)
if (isFrozen() || isBogus()) {
return;
}
UErrorCode status = U_ZERO_ERROR;
ensureBufferCapacity(len + otherLen, status);
if (U_FAILURE(status)) {
if (!ensureBufferCapacity(len + otherLen)) {
return;
}
@@ -2138,12 +2134,14 @@ UnicodeString& UnicodeSet::_generatePattern(UnicodeString& result,
}
}
for (int32_t i = 0; i<strings->size(); ++i) {
result.append(OPEN_BRACE);
_appendToPat(result,
*(const UnicodeString*) strings->elementAt(i),
escapeUnprintable);
result.append(CLOSE_BRACE);
if (strings != nullptr) {
for (int32_t i = 0; i<strings->size(); ++i) {
result.append(OPEN_BRACE);
_appendToPat(result,
*(const UnicodeString*) strings->elementAt(i),
escapeUnprintable);
result.append(CLOSE_BRACE);
}
}
return result.append(SET_CLOSE);
}
@@ -2162,13 +2160,12 @@ void UnicodeSet::releasePattern() {
/**
* Set the new pattern to cache.
*/
void UnicodeSet::setPattern(const UnicodeString& newPat) {
void UnicodeSet::setPattern(const char16_t *newPat, int32_t newPatLen) {
releasePattern();
int32_t newPatLen = newPat.length();
pat = (UChar *)uprv_malloc((newPatLen + 1) * sizeof(UChar));
if (pat) {
patLen = newPatLen;
newPat.extractBetween(0, patLen, pat);
u_memcpy(pat, newPat, patLen);
pat[patLen] = 0;
}
// else we don't care if malloc failed. This was just a nice cache.
@@ -2177,30 +2174,15 @@ void UnicodeSet::setPattern(const UnicodeString& newPat) {
UnicodeFunctor *UnicodeSet::freeze() {
if(!isFrozen() && !isBogus()) {
// Do most of what compact() does before freezing because
// compact() will not work when the set is frozen.
// Small modification: Don't shrink if the savings would be tiny (<=GROW_EXTRA).
// Delete buffer first to defragment memory less.
if (buffer != NULL) {
uprv_free(buffer);
buffer = NULL;
}
if (capacity > (len + GROW_EXTRA)) {
// Make the capacity equal to len or 1.
// We don't want to realloc of 0 size.
capacity = len + (len == 0);
list = (UChar32*) uprv_realloc(list, sizeof(UChar32) * capacity);
if (list == NULL) { // Check for memory allocation error.
setToBogus();
return this;
}
}
compact();
// Optimize contains() and span() and similar functions.
if (!strings->isEmpty()) {
if (hasStrings()) {
stringSpan = new UnicodeSetStringSpan(*this, *strings, UnicodeSetStringSpan::ALL);
if (stringSpan != NULL && !stringSpan->needsStringSpanUTF16()) {
if (stringSpan == nullptr) {
setToBogus();
return this;
} else if (!stringSpan->needsStringSpanUTF16()) {
// All strings are irrelevant for span() etc. because
// all of each string's code points are contained in this set.
// Do not check needsStringSpanUTF8() because UTF-8 has at most as
@@ -2233,7 +2215,7 @@ int32_t UnicodeSet::span(const UChar *s, int32_t length, USetSpanCondition spanC
}
if(stringSpan!=NULL) {
return stringSpan->span(s, length, spanCondition);
} else if(!strings->isEmpty()) {
} else if(hasStrings()) {
uint32_t which= spanCondition==USET_SPAN_NOT_CONTAINED ?
UnicodeSetStringSpan::FWD_UTF16_NOT_CONTAINED :
UnicodeSetStringSpan::FWD_UTF16_CONTAINED;
@@ -2270,7 +2252,7 @@ int32_t UnicodeSet::spanBack(const UChar *s, int32_t length, USetSpanCondition s
}
if(stringSpan!=NULL) {
return stringSpan->spanBack(s, length, spanCondition);
} else if(!strings->isEmpty()) {
} else if(hasStrings()) {
uint32_t which= spanCondition==USET_SPAN_NOT_CONTAINED ?
UnicodeSetStringSpan::BACK_UTF16_NOT_CONTAINED :
UnicodeSetStringSpan::BACK_UTF16_CONTAINED;
@@ -2308,7 +2290,7 @@ int32_t UnicodeSet::spanUTF8(const char *s, int32_t length, USetSpanCondition sp
}
if(stringSpan!=NULL) {
return stringSpan->spanUTF8((const uint8_t *)s, length, spanCondition);
} else if(!strings->isEmpty()) {
} else if(hasStrings()) {
uint32_t which= spanCondition==USET_SPAN_NOT_CONTAINED ?
UnicodeSetStringSpan::FWD_UTF8_NOT_CONTAINED :
UnicodeSetStringSpan::FWD_UTF8_CONTAINED;
@@ -2346,7 +2328,7 @@ int32_t UnicodeSet::spanBackUTF8(const char *s, int32_t length, USetSpanConditio
}
if(stringSpan!=NULL) {
return stringSpan->spanBackUTF8((const uint8_t *)s, length, spanCondition);
} else if(!strings->isEmpty()) {
} else if(hasStrings()) {
uint32_t which= spanCondition==USET_SPAN_NOT_CONTAINED ?
UnicodeSetStringSpan::BACK_UTF8_NOT_CONTAINED :
UnicodeSetStringSpan::BACK_UTF8_CONTAINED;
+6 -36
View File
@@ -31,10 +31,6 @@
#include "util.h"
#include "uvector.h"
// initial storage. Must be >= 0
// *** same as in uniset.cpp ! ***
#define START_EXTRA 16
U_NAMESPACE_BEGIN
// TODO memory debugging provided inside uniset.cpp
@@ -49,42 +45,16 @@ U_NAMESPACE_BEGIN
UnicodeSet::UnicodeSet(const UnicodeString& pattern,
uint32_t options,
const SymbolTable* symbols,
UErrorCode& status) :
len(0), capacity(START_EXTRA), list(0), bmpSet(0), buffer(0),
bufferCapacity(0), patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
if(U_SUCCESS(status)){
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
/* test for NULL */
if(list == NULL) {
status = U_MEMORY_ALLOCATION_ERROR;
}else{
allocateStrings(status);
applyPattern(pattern, options, symbols, status);
}
}
UErrorCode& status) {
applyPattern(pattern, options, symbols, status);
_dbgct(this);
}
UnicodeSet::UnicodeSet(const UnicodeString& pattern, ParsePosition& pos,
uint32_t options,
const SymbolTable* symbols,
UErrorCode& status) :
len(0), capacity(START_EXTRA), list(0), bmpSet(0), buffer(0),
bufferCapacity(0), patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
if(U_SUCCESS(status)){
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
/* test for NULL */
if(list == NULL) {
status = U_MEMORY_ALLOCATION_ERROR;
}else{
allocateStrings(status);
applyPattern(pattern, pos, options, symbols, status);
}
}
UErrorCode& status) {
applyPattern(pattern, pos, options, symbols, status);
_dbgct(this);
}
@@ -199,7 +169,7 @@ UnicodeSet& UnicodeSet::closeOver(int32_t attribute) {
// start with input set to guarantee inclusion
// USET_CASE: remove strings because the strings will actually be reduced (folded);
// therefore, start with no strings and add only those needed
if (attribute & USET_CASE_INSENSITIVE) {
if ((attribute & USET_CASE_INSENSITIVE) && foldSet.hasStrings()) {
foldSet.strings->removeAllElements();
}
@@ -234,7 +204,7 @@ UnicodeSet& UnicodeSet::closeOver(int32_t attribute) {
}
}
}
if (strings != NULL && strings->size() > 0) {
if (hasStrings()) {
if (attribute & USET_CASE_INSENSITIVE) {
for (int32_t j=0; j<strings->size(); ++j) {
str = *(const UnicodeString *) strings->elementAt(j);
+23 -66
View File
@@ -47,10 +47,6 @@
U_NAMESPACE_USE
// initial storage. Must be >= 0
// *** same as in uniset.cpp ! ***
#define START_EXTRA 16
// Define UChar constants using hex for EBCDIC compatibility
// Used #define to reduce private static exports and memory access time.
#define SET_OPEN ((UChar)0x005B) /*[*/
@@ -185,21 +181,8 @@ isPOSIXClose(const UnicodeString &pattern, int32_t pos) {
* @param pattern a string specifying what characters are in the set
*/
UnicodeSet::UnicodeSet(const UnicodeString& pattern,
UErrorCode& status) :
len(0), capacity(START_EXTRA), list(0), bmpSet(0), buffer(0),
bufferCapacity(0), patLen(0), pat(NULL), strings(NULL), stringSpan(NULL),
fFlags(0)
{
if(U_SUCCESS(status)){
list = (UChar32*) uprv_malloc(sizeof(UChar32) * capacity);
/* test for NULL */
if(list == NULL) {
status = U_MEMORY_ALLOCATION_ERROR;
}else{
allocateStrings(status);
applyPattern(pattern, status);
}
}
UErrorCode& status) {
applyPattern(pattern, status);
_dbgct(this);
}
@@ -713,6 +696,11 @@ static UBool numericValueFilter(UChar32 ch, void* context) {
return u_getNumericValue(ch) == *(double*)context;
}
static UBool generalCategoryMaskFilter(UChar32 ch, void* context) {
int32_t value = *(int32_t*)context;
return (U_GET_GC_MASK((UChar32) ch) & value) != 0;
}
static UBool versionFilter(UChar32 ch, void* context) {
static const UVersionInfo none = { 0, 0, 0, 0 };
UVersionInfo v;
@@ -721,6 +709,16 @@ static UBool versionFilter(UChar32 ch, void* context) {
return uprv_memcmp(&v, &none, sizeof(v)) > 0 && uprv_memcmp(&v, version, sizeof(v)) <= 0;
}
typedef struct {
UProperty prop;
int32_t value;
} IntPropertyContext;
static UBool intPropertyFilter(UChar32 ch, void* context) {
IntPropertyContext* c = (IntPropertyContext*)context;
return u_getIntPropertyValue((UChar32) ch, c->prop) == c->value;
}
static UBool scriptExtensionsFilter(UChar32 ch, void* context) {
return uscript_hasScript(ch, *(UScriptCode*)context);
}
@@ -781,43 +779,6 @@ void UnicodeSet::applyFilter(UnicodeSet::Filter filter,
namespace {
/** Maps map values to 1 if the mask contains their value'th bit, all others to 0. */
uint32_t U_CALLCONV generalCategoryMaskFilter(const void *context, uint32_t value) {
uint32_t mask = *(const uint32_t *)context;
value = U_MASK(value) & mask;
if (value != 0) { value = 1; }
return value;
}
/** Maps one map value to 1, all others to 0. */
uint32_t U_CALLCONV intValueFilter(const void *context, uint32_t value) {
uint32_t v = *(const uint32_t *)context;
return value == v ? 1 : 0;
}
} // namespace
void UnicodeSet::applyIntPropertyValue(const UCPMap *map,
UCPMapValueFilter *filter, const void *context,
UErrorCode &errorCode) {
if (U_FAILURE(errorCode)) { return; }
clear();
UChar32 start = 0, end;
uint32_t value;
while ((end = ucpmap_getRange(map, start, UCPMAP_RANGE_NORMAL, 0,
filter, context, &value)) >= 0) {
if (value != 0) {
add(start, end);
}
start = end + 1;
}
if (isBogus()) {
errorCode = U_MEMORY_ALLOCATION_ERROR;
}
}
namespace {
static UBool mungeCharName(char* dst, const char* src, int32_t dstCapacity) {
/* Note: we use ' ' in compiler code page */
int32_t j = 0;
@@ -845,11 +806,10 @@ static UBool mungeCharName(char* dst, const char* src, int32_t dstCapacity) {
UnicodeSet&
UnicodeSet::applyIntPropertyValue(UProperty prop, int32_t value, UErrorCode& ec) {
if (U_FAILURE(ec)) { return *this; }
// All of the following check isFrozen() before modifying this set.
if (U_FAILURE(ec) || isFrozen()) { return *this; }
if (prop == UCHAR_GENERAL_CATEGORY_MASK) {
const UCPMap *map = u_getIntPropertyMap(UCHAR_GENERAL_CATEGORY, &ec);
applyIntPropertyValue(map, generalCategoryMaskFilter, &value, ec);
const UnicodeSet* inclusions = CharacterProperties::getInclusionsForProperty(prop, ec);
applyFilter(generalCategoryMaskFilter, &value, inclusions, ec);
} else if (prop == UCHAR_SCRIPT_EXTENSIONS) {
const UnicodeSet* inclusions = CharacterProperties::getInclusionsForProperty(prop, ec);
UScriptCode script = (UScriptCode)value;
@@ -866,14 +826,11 @@ UnicodeSet::applyIntPropertyValue(UProperty prop, int32_t value, UErrorCode& ec)
clear();
}
} else if (UCHAR_INT_START <= prop && prop < UCHAR_INT_LIMIT) {
const UCPMap *map = u_getIntPropertyMap(prop, &ec);
applyIntPropertyValue(map, intValueFilter, &value, ec);
const UnicodeSet* inclusions = CharacterProperties::getInclusionsForProperty(prop, ec);
IntPropertyContext c = {prop, value};
applyFilter(intPropertyFilter, &c, inclusions, ec);
} else {
// This code used to always call getInclusions(property source)
// which sets an error for an unsupported property.
ec = U_ILLEGAL_ARGUMENT_ERROR;
// Otherwise we would just clear() this set because
// getIntPropertyValue(c, prop) returns 0 for all code points.
}
return *this;
}
-1
View File
@@ -462,7 +462,6 @@ class UnicodeSet;
class CharacterProperties {
public:
CharacterProperties() = delete;
static void U_CALLCONV initInclusion(UPropertySource src, UErrorCode &errorCode);
static const UnicodeSet *getInclusionsForProperty(UProperty prop, UErrorCode &errorCode);
};
+1 -1
View File
@@ -249,7 +249,7 @@ class USetAccess /* not : public UObject because all methods are static */ {
public:
/* Try to have the compiler inline these*/
inline static int32_t getStringCount(const UnicodeSet& set) {
return set.getStringCount();
return set.stringsSize();
}
inline static const UnicodeString* getString(const UnicodeSet& set,
int32_t i) {
+1 -1
View File
@@ -116,7 +116,7 @@ void UnicodeSetIterator::reset() {
stringCount = 0;
} else {
endRange = set->getRangeCount() - 1;
stringCount = set->strings->size();
stringCount = set->stringsSize();
}
range = 0;
endElement = -1;
+2 -2
View File
@@ -13,7 +13,7 @@
#include "unicode/utypes.h"
#if U_PLATFORM_HAS_WIN32_API
#if U_PLATFORM_USES_ONLY_WIN32_API
#include "wintz.h"
#include "cmemory.h"
@@ -123,4 +123,4 @@ uprv_detectWindowsTimeZone()
}
U_NAMESPACE_END
#endif /* U_PLATFORM_HAS_WIN32_API */
#endif /* U_PLATFORM_USES_ONLY_WIN32_API */
+2 -2
View File
@@ -16,7 +16,7 @@
#include "unicode/utypes.h"
#if U_PLATFORM_HAS_WIN32_API
#if U_PLATFORM_USES_ONLY_WIN32_API
/**
* \file
@@ -31,6 +31,6 @@ U_CDECL_END
U_CFUNC const char* U_EXPORT2
uprv_detectWindowsTimeZone();
#endif /* U_PLATFORM_HAS_WIN32_API */
#endif /* U_PLATFORM_USES_ONLY_WIN32_API */
#endif /* __WINTZ */
+9 -9
View File
@@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for ICU 63.1.
# Generated by GNU Autoconf 2.69 for ICU 63.2.
#
# Report bugs to <http://icu-project.org/bugs>.
#
@@ -582,8 +582,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='ICU'
PACKAGE_TARNAME='International Components for Unicode'
PACKAGE_VERSION='63.1'
PACKAGE_STRING='ICU 63.1'
PACKAGE_VERSION='63.2'
PACKAGE_STRING='ICU 63.2'
PACKAGE_BUGREPORT='http://icu-project.org/bugs'
PACKAGE_URL='http://icu-project.org'
@@ -1370,7 +1370,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures ICU 63.1 to adapt to many kinds of systems.
\`configure' configures ICU 63.2 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@@ -1437,7 +1437,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of ICU 63.1:";;
short | recursive ) echo "Configuration of ICU 63.2:";;
esac
cat <<\_ACEOF
@@ -1574,7 +1574,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
ICU configure 63.1
ICU configure 63.2
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
@@ -2266,7 +2266,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by ICU $as_me 63.1, which was
It was created by ICU $as_me 63.2, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
@@ -8434,7 +8434,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by ICU $as_me 63.1, which was
This file was extended by ICU $as_me 63.2, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@@ -8488,7 +8488,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
ICU config.status 63.1
ICU config.status 63.2
configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"
+1 -1
View File
@@ -2520,5 +2520,5 @@ ast{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -1853,5 +1853,5 @@ bn{
other{"জাম্বিয়ান কওয়াচা"}
}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -3138,5 +3138,5 @@ br{
other{"{0} {1}"}
two{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -3123,5 +3123,5 @@ cs{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -3273,5 +3273,5 @@ cy{
two{"{0} {1}"}
zero{"{0} {1}"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2527,5 +2527,5 @@ de{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.44.86"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2433,5 +2433,5 @@ en{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.44.79"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2225,5 +2225,5 @@ et{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2528,5 +2528,5 @@ fi{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -1437,5 +1437,5 @@ fil{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2353,5 +2353,5 @@ fr{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.44.2"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -3188,5 +3188,5 @@ ga{
other{"{0} {1}"}
two{"{0} {1}"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -3134,5 +3134,5 @@ gd{
other{"{0} {1}"}
two{"{0} {1}"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -1677,5 +1677,5 @@ is{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2223,5 +2223,5 @@ ja{
CurrencyUnitPatterns{
other{"{0}{1}"}
}
Version{"2.1.43.74"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -17,5 +17,5 @@ kl{
other{"euro"}
}
}
Version{"2.1.43.94"}
Version{"2.1.45.12"}
}
+1 -1
View File
@@ -1762,5 +1762,5 @@ ko{
CurrencyUnitPatterns{
other{"{0} {1}"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2249,5 +2249,5 @@ lb{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -3126,5 +3126,5 @@ lt{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -1821,5 +1821,5 @@ lv{
other{"{0} {1}"}
zero{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -1249,5 +1249,5 @@ ms{
CurrencyUnitPatterns{
other{"{0} {1}"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+2 -2
View File
@@ -272,7 +272,7 @@ nl{
}
CNY{
"CN¥",
"Chinese Yuan",
"Chinese yuan",
}
COP{
"COP",
@@ -2529,5 +2529,5 @@ nl{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.44.2"}
Version{"2.1.45.14"}
}
+1 -1
View File
@@ -196,7 +196,7 @@ root{
CurrencyUnitPatterns{
other{"{0} {1}"}
}
Version{"2.1.44.18"}
Version{"2.1.44.91"}
currencySpacing{
afterCurrency{
currencyMatch{"[:^S:]"}
+1 -1
View File
@@ -2236,5 +2236,5 @@ ru{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.45.15"}
}
+1 -1
View File
@@ -2197,5 +2197,5 @@ sl{
other{"{0} {1}"}
two{"{0} {1}"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2400,5 +2400,5 @@ tr{
one{"{0} {1}"}
other{"{0} {1}"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2221,5 +2221,5 @@ yue{
CurrencyUnitPatterns{
other{"{0} {1}"}
}
Version{"2.1.44.76"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2222,5 +2222,5 @@ yue_Hans{
CurrencyUnitPatterns{
other{"{0} {1}"}
}
Version{"2.1.44.50"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -2222,5 +2222,5 @@ zh_Hant{
CurrencyUnitPatterns{
other{"{0} {1}"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
}
+1 -1
View File
@@ -1029,7 +1029,7 @@ ast{
VALLADER{"VALLADER"}
WADEGILE{"romanización de Wade-Giles"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — too"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -872,7 +872,7 @@ bn{
wara{"ওয়ারেং সিটি সংখ্যা"}
}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — সমস্ত"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -844,7 +844,7 @@ br{
WADEGILE{"romanekadur Wade-Giles"}
XSISTEMO{"esperanteg sistem X"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
codePatterns{
language{"{0}"}
script{"{0}"}
+1 -1
View File
@@ -961,7 +961,7 @@ cs{
SCOTLAND{"angličtina (Skotsko)"}
WADEGILE{"Wade-Giles"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} – vše"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -732,7 +732,7 @@ cy{
VAIDIKA{"VAIDIKA"}
VALLADER{"VALLADER"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — y cwbl"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1008,7 +1008,7 @@ de{
VALENCIA{"Valencianisch"}
WADEGILE{"Wade-Giles"}
}
Version{"2.1.44.86"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — Alle"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1247,7 +1247,7 @@ en{
VALENCIA{"Valencian"}
WADEGILE{"Wade-Giles Romanization"}
}
Version{"2.1.44.79"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — all"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1036,7 +1036,7 @@ et{
VALENCIA{"valentsia"}
WADEGILE{"Wade’i-Gilesi latinisatsioon"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} – kõik"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1118,7 +1118,7 @@ fi{
Variants%secondary{
FONUPA{"suomalais-ugrilainen tarkekirjoitus"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} – kaikki"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -664,7 +664,7 @@ fil{
PINYIN{"Pinyin Romanization"}
WADEGILE{"Wade-Giles Romanization"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — lahat"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -970,7 +970,7 @@ fr{
VALENCIA{"valencien"}
WADEGILE{"Wade-Giles"}
}
Version{"2.1.44.2"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — tout"}
category-list{"{0} : {1}"}
+1 -1
View File
@@ -853,7 +853,7 @@ ga{
WADEGILE{"Rómhánú Wade-Giles"}
XSISTEMO{"XSISTEMO"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — Uile"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1032,7 +1032,7 @@ gd{
WADEGILE{"Ròmanachadh Wade-Giles"}
XSISTEMO{"XSISTEMO"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} – na h-uile"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -751,7 +751,7 @@ is{
vaii{"Vai-tölustafir"}
}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — allt"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1047,7 +1047,7 @@ ja{
VALLADER{"ヴァラダー"}
WADEGILE{"ウェード式ローマ字表記法"}
}
Version{"2.1.43.74"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — すべて"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -4,7 +4,7 @@ kl{
Languages{
kl{"kalaallisut"}
}
Version{"2.1.43.94"}
Version{"2.1.45.12"}
localeDisplayPattern{
keyTypePattern{"{0}: {1}"}
pattern{"{0} ({1})"}
+1 -1
View File
@@ -919,7 +919,7 @@ ko{
VAIDIKA{"바이디카"}
VALLADER{"발라더"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — 전체"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -875,7 +875,7 @@ lb{
VALENCIA{"Valencianesch"}
WADEGILE{"Wade-Giles"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
codePatterns{
language{"Sprooch: {0}"}
script{"Schrëft: {0}"}
+1 -1
View File
@@ -1023,7 +1023,7 @@ lt{
VALENCIA{"Valenciečiai"}
WADEGILE{"Wade-Giles Romanization"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} – visi"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -798,7 +798,7 @@ lv{
VALENCIA{"valensiešu"}
WADEGILE{"Veida-Džailza romanizācija"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — visas"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -717,7 +717,7 @@ ms{
Variants{
POSIX{"Komputer"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — Semua"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1096,7 +1096,7 @@ nl{
WADEGILE{"Wade-Giles-romanisering"}
XSISTEMO{"X-sistemo"}
}
Version{"2.1.44.2"}
Version{"2.1.45.14"}
characterLabelPattern{
all{"{0} — alle"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -4,7 +4,7 @@
* ICU <specials> source: <path>/common/main/root.xml
*/
root{
Version{"2.1.44.18"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — all"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -917,7 +917,7 @@ ru{
VALENCIA{"Валенсийский"}
WADEGILE{"Система Уэйда – Джайлза"}
}
Version{"2.1.43.94"}
Version{"2.1.45.15"}
characterLabelPattern{
all{"{0} — все"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -879,7 +879,7 @@ sl{
VALENCIA{"valencijski pravopis"}
WADEGILE{"romanizacija Wade-Giles"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} – vse"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1038,7 +1038,7 @@ tr{
VALENCIA{"Valensiyaca"}
WADEGILE{"Wade-Giles (Latin Alfabesinde Yazımı)"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — tümü"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1024,7 +1024,7 @@ yue{
VALLADER{"瑞士瓦勒德方言"}
WADEGILE{"威妥瑪式拼音"}
}
Version{"2.1.44.76"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — 全部"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1025,7 +1025,7 @@ yue_Hans{
VALLADER{"瑞士瓦勒德方言"}
WADEGILE{"威妥玛式拼音"}
}
Version{"2.1.44.50"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — 全部"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1074,7 +1074,7 @@ zh_Hant{
VALLADER{"瑞士瓦勒德方言"}
WADEGILE{"威妥瑪式拼音"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
characterLabelPattern{
all{"{0} — 全部"}
category-list{"{0}: {1}"}
+1 -1
View File
@@ -1637,7 +1637,7 @@ ar{
"تيشو",
"شووا",
"هيسي",
"Qqqq",
"ريوا",
}
}
}
+4 -4
View File
@@ -236,7 +236,7 @@ ast{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
@@ -2304,7 +2304,7 @@ ast{
"Taishō",
"e. Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
narrow{
"Taika",
@@ -2543,7 +2543,7 @@ ast{
"T",
"S",
"H",
"Q",
"R",
}
wide{
"Taika (645–650)",
@@ -2782,7 +2782,7 @@ ast{
"Taishō",
"era Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
intervalFormats{
+1 -1
View File
@@ -247,7 +247,7 @@ bn{
minimumGroupingDigits{"1"}
native{"beng"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
calendar{
buddhist{
eras{
+4 -4
View File
@@ -551,7 +551,7 @@ br{
}
}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
@@ -4106,7 +4106,7 @@ br{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
narrow{
"Taika (645–650)",
@@ -4345,7 +4345,7 @@ br{
"T",
"S",
"H",
"Q",
"R",
}
wide{
"Taika (645–650)",
@@ -4584,7 +4584,7 @@ br{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
intervalFormats{
+1 -1
View File
@@ -1035,7 +1035,7 @@ bs_Cyrl{
"Таишо",
"Шова",
"Хаисеи",
"Qqqq",
"Реива",
}
}
}
+4 -4
View File
@@ -521,7 +521,7 @@ cs{
}
}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
@@ -4363,7 +4363,7 @@ cs{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
narrow{
"Taika (645–650)",
@@ -4602,7 +4602,7 @@ cs{
"T",
"S",
"H",
"Q",
"R",
}
wide{
"Taika (645–650)",
@@ -4841,7 +4841,7 @@ cs{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
intervalFormats{
+1 -1
View File
@@ -395,7 +395,7 @@ cy{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
calendar{
generic{
DateTimePatterns{
+2 -2
View File
@@ -218,7 +218,7 @@ de{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.44.86"}
Version{"2.1.44.91"}
calendar{
buddhist{
eras{
@@ -1603,7 +1603,7 @@ de{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
}
+1 -1
View File
@@ -205,7 +205,7 @@ en{
}
}
}
Version{"2.1.44.79"}
Version{"2.1.44.91"}
calendar{
buddhist{
eras{
+1 -1
View File
@@ -212,7 +212,7 @@ et{
minimumGroupingDigits{"2"}
native{"latn"}
}
Version{"2.1.43.94"}
Version{"2.1.44.91"}
calendar{
buddhist{
eras{
+1 -1
View File
@@ -216,7 +216,7 @@ fi{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
+1 -1
View File
@@ -213,7 +213,7 @@ fil{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.43.61"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
+1 -1
View File
@@ -221,7 +221,7 @@ fr{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.44.2"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
+1 -1
View File
@@ -325,7 +325,7 @@ ga{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
+1 -1
View File
@@ -1002,7 +1002,7 @@ gd{
}
}
}
Version{"2.1.44.88"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
+1 -1
View File
@@ -1839,7 +1839,7 @@ he{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
}
+1 -1
View File
@@ -1241,7 +1241,7 @@ hi{
"ताईशो",
"शोवा",
"हेईसेई",
"Qqqq",
"रेइवा",
}
}
}
+1 -1
View File
@@ -1579,7 +1579,7 @@ hr{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
}
+1 -1
View File
@@ -2219,7 +2219,7 @@ id{
"Taishō",
"Shōwa",
"Heisei",
"Qqqq",
"Reiwa",
}
}
}
+1 -1
View File
@@ -221,7 +221,7 @@ is{
minimumGroupingDigits{"1"}
native{"latn"}
}
Version{"2.1.43.65"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
+3 -3
View File
@@ -235,7 +235,7 @@ ja{
native{"latn"}
traditional{"jpan"}
}
Version{"2.1.43.74"}
Version{"2.1.44.91"}
calendar{
buddhist{
DateTimePatterns{
@@ -2542,7 +2542,7 @@ ja{
"大正",
"昭和",
"平成",
"QQ",
"令和",
}
narrow{
"大化",
@@ -2781,7 +2781,7 @@ ja{
"T",
"S",
"H",
"Q",
"R",
}
}
}
+6 -6
View File
@@ -25,7 +25,7 @@ kl{
}
}
}
Version{"2.1.43.94"}
Version{"2.1.45.12"}
calendar{
gregorian{
DateTimePatterns{
@@ -46,7 +46,7 @@ kl{
dayNames{
format{
abbreviated{
"sab",
"sap",
"ata",
"mar",
"pin",
@@ -55,7 +55,7 @@ kl{
"arf",
}
short{
"sab",
"sap",
"ata",
"mar",
"pin",
@@ -64,7 +64,7 @@ kl{
"arf",
}
wide{
"sabaat",
"sapaat",
"ataasinngorneq",
"marlunngorneq",
"pingasunngorneq",
@@ -75,7 +75,7 @@ kl{
}
stand-alone{
short{
"sab",
"sap",
"ata",
"mar",
"pin",
@@ -84,7 +84,7 @@ kl{
"arf",
}
wide{
"sabaat",
"sapaat",
"ataasinngorneq",
"marlunngorneq",
"pingasunngorneq",

Some files were not shown because too many files have changed in this diff Show More