| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unicode Version 12.1.0 adds one single character, U+32FF SQUARE ERA NAME REIWA,
for the new Japanese era starting on May 1st. 12.1.0 will be finalized only on
May 7th, so we go with the beta version because further changes in the data we
need are highly unlikely, and we want to make sure Ruby is ready for the new era.
* common.mk: change UNICODE_VERSION to 12.1.0, UNICODE_BETA to YES
* enc/unicode/12.1.0, enc/unicode/12.1.0/casefold.h, enc/unicode/12.1.0/name2ctype.h:
add directory and generated data files for new version
* lib/unicode_normalize/tables.rb: update for new character
* test/ruby/test_regexp.rb: add test for character property age=12.1
* test/test_unicode_normalize.rb: add test for NFKC decomposition of new character
This (mostly) completes issue #15195.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67441 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* lib/unicode_normalize/normalize.rb: Fix the range check for trailing
Hangul jamo characters in Unicode normalization. Different from
leading or vowel jamos, where LBASE and VBASE are actual characters,
a value equal to TBASE expresses the absence of a trailing jamo.
This fix is technically correct, but there was no bug because
the regular expressions in lib/unicode_normalize/tables.rb
eliminate jamos equal to TBASE from normalization processing.
* test/test_unicode_normalize.rb: Add preventive test
test_no_trailing_jamo based on
https://github.com/python/cpython/commit/d134809cd3764c6a634eab7bb8995e3e2eff14d5
just for the case we ever get a regression.
This closes issue #14934, thanks to MaLin (Lin Ma) for reporting.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
* test/test_unicode_normalize.rb: extract tests that do not depend
on NormalizationTest.txt data file from conditionally constructed
part of TestUnicodeNormalize class, to always run them even if
the data file isn't found.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
* test/test_unicode_normalize.rb: search ucd directory
first if it exists. This change follows r61415.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
test/test_unicode_normalize.rb: Replace UNICODE_VERSION from
UnicodeNormalize with RbConfig::CONFIG['UNICODE_VERSION'] from
feature 12460.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55567 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
* test/ruby/enc/test_case_comprehensive.rb: noting to test if
Unicode data files are available.
[ruby-core:76160] [Bug #12433]
* test/test_unicode_normalize.rb: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55521 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
Unicode data file; refactoring; fix an error with tests for destructive
method (unicode_normalize!).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55196 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
When you change this to true, you may need to add more tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
* test/test_unicode_normalize.rb (setup): skip only once, and
silently ignore other tests.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48388 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
* test/test_unicode_normalize.rb (read_tests): do nothing if the
test data file is not found.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48387 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
* template/unicode_norm_gen.tmpl (UnicodeNormalize): embed the
version of Unicode data files used to generate.
* test/test_unicode_normalize.rb (TestUnicodeNormalize): use the
embedded version to load the test data.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48357 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
TestUnicodeNormalize.
Define constants under TestUnicodeNormalize.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48301 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
test data location.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48202 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48135 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
test_raise_exception_for_non_unicode_encoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48132 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
| |
* test/test_unicode_normalize.rb (generate_test_normalize): defer
building explicit error messages until assertion failed. this
is 3% slower than @@debug = false.
(generate_test_check_true): ditto.
(generate_test_check_false): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48113 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48092 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
at non-srcdir.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48087 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
../enc/unicode/data/NormalizationTest.txt).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
names, adjusted copyright.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48085 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
https://github.com/duerst/eprun/blob/master/test/test_normalize.rb.
(removing trailing whitespace, fixing EOLs and adding EOL property)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48084 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|