summaryrefslogtreecommitdiff
path: root/mysql-test/std_data
Commit message (Collapse)AuthorAgeFilesLines
* SSL test fixesSergei Golubchik2019-03-0132-961/+686
| | | | | | | | | | | * fix CRL tests to work * regenerate certificates to be at least 2048 bit (fixes buster and rhel8 in buildbot) * update generate-ssl-cert.sh to generate crl files * make all SSL tests to use certificates generated in generate-ssl-cert.sh, remove unused certificates Backport from 10.4 9c60535f8676
* Backporting MDEV-15597 Add class Load_data_outvar and avoid using ↵Alexander Barkov2019-02-231-0/+1
| | | | | | Item::STRING_ITEM for Item_user_var_as_out_param detection This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
* Backporting MDEV-15497 Wrong empty value in a GEOMETRY column on LOAD DATAAlexander Barkov2019-02-231-0/+1
| | | | This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
* Backporting MDEV-14628 Wrong autoinc value assigned by LOAD XML in the ↵Alexander Barkov2019-02-232-0/+7
| | | | | | NO_AUTO_VALUE_ON_ZERO mode This is a part of "MDEV-18045 Backporting the MDEV-15497 changes to 10.2 branch"
* MDEV-17909 Problem by MariaDB Update 10.1.32 -> 10.2.19 (Incorrect ↵Sergei Golubchik2018-12-206-0/+0
| | | | | | | | | | | information in file: .frm) use frm_version, not mysql_version when parsing frm In particular, virtual columns are stored according to frm_version. And CHECK TABLE will overwrite mysql_version to the current server version, so it cannot correctly describe frm format.
* Merge branch '10.1' into 10.2Sergei Golubchik2018-06-211-0/+0
|\
| * Merge remote-tracking branch 'origin/10.0' into 10.1Alexander Barkov2018-06-191-0/+0
| |\
| | * Merge remote-tracking branch 'origin/5.5' into 10.0Alexander Barkov2018-06-191-0/+0
| | |\
| | | * MDEV-15834 The code in TABLE_SHARE::init_from_binary_frm_image() is not safeAlexander Barkov2018-06-191-0/+0
| | | |
* | | | Merge branch '10.1' into 10.2Vicențiu Ciorbaru2018-04-101-0/+0
|\ \ \ \ | |/ / /
| * | | MDEV-13549 Fix and re-enable MTR test galera.galera_gra_logDaniele Sciascia2018-04-031-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Test galera checks that a `GRA_x_x.log` file is created whenever wsrep applier fails to apply some replication event. The file contains the corresponding binlog event that failed to apply. The test creates a new file by concatenating a pre-recorded file containing the binlog header (see `std-data/binlog-header.log`) and the `GRA_x_x.log` file. The test then checks that the resulting file, containing the binlog header and the event that failed to apply, is correctly read by `mysqlbinlog` program. The test fails in MariaDB because the GRA_x_x.log file created by MariaDB already contains the binlog header (see MDEV-7867). This patch fixes/simplifies test `galera.galera_gra_log` so that it doesn't concatenate `std-data/binlog-header.log` with the `GRA_x_x.log` file. File `std-data/binlog-header.log` is deleted altoghether, because not used by any other test.
* | | | Merge branch '10.1' into 10.2Sergei Golubchik2017-08-171-4/+4
|\ \ \ \ | |/ / /
| * | | Merge branch '10.0' into 10.1Sergei Golubchik2017-08-081-4/+4
| |\ \ \ | | |/ /
| | * | Merge remote-tracking branch 'origin/5.5' into 10.0Vicențiu Ciorbaru2017-07-251-4/+4
| | |\ \ | | | |/
| | | * test case for loadxml and spacesSergei Golubchik2017-07-181-4/+4
| | | |
| * | | MDEV-12696 Crash with LOAD XML and non-updatable VIEW columnSergei Golubchik2017-05-091-0/+9
| | | | | | | | | | | | | | | | extend the fix from 5.5 (in read_sep_field()) to apply to read_xml_field()
* | | | MDEV-12936 upgrade to 10.2.6 failed upon tables with virtual columnsSergei Golubchik2017-07-053-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when opening 10.1- table that has virtual columns: 1. don't error out if it has vcols over autoinc columns. just issue a warning. 2. set vcol type properly 3. in innodb: use table->s->stored_fields instead of table->s->fields, because that's what was stored in innodb data dictionary
* | | | Merge 10.1 into 10.2Marko Mäkelä2017-05-0610-394/+499
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | Also, include fixes by Vladislav Vaintroub to the aws_key_management plugin. The AWS C++ SDK specifically depends on OPENSSL_LIBRARIES, not generic SSL_LIBRARIES (such as YaSSL).
| * | | MDEV-10594 SSL hostname verification fails for SubjectAltNamesSergei Golubchik2017-04-272-0/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | use X509_check_host for OpenSSL 1.0.2+ This adds: * support for subjectAltNames * wildcards * sub-domain matching
| * | | regenerate SSL certificates againSergei Golubchik2017-04-278-394/+423
| | | | | | | | | | | | | | | | | | | | | | | | and make sure that private ca key is not deleted at the end of the procedure, so that we could generate additional certificates any time without regenerating everything
* | | | Merge 10.1 into 10.2Marko Mäkelä2017-05-053-3/+4
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This only merges MDEV-12253, adapting it to MDEV-12602 which is already present in 10.2 but not yet in the 10.1 revision that is being merged. TODO: Error handling in crash recovery needs to be improved. If a page cannot be decrypted (or read), we should cleanly abort the startup. If innodb_force_recovery is specified, we should ignore the problematic page and apply redo log to other pages. Currently, the test encryption.innodb-redo-badkey randomly fails like this (the last messages are from cmake -DWITH_ASAN): 2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption 2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown... i================================================================= ==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0 #0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750) #1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4 #2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17 #3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3 #4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2 #5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2 #6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3
| * | | MDEV-12253: Buffer pool blocks are accessed after they have been freedJan Lindström2017-04-263-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem was that bpage was referenced after it was already freed from LRU. Fixed by adding a new variable encrypted that is passed down to buf_page_check_corrupt() and used in buf_page_get_gen() to stop processing page read. This patch should also address following test failures and bugs: MDEV-12419: IMPORT should not look up tablespace in PageConverter::validate(). This is now removed. MDEV-10099: encryption.innodb_onlinealter_encryption fails sporadically in buildbot MDEV-11420: encryption.innodb_encryption-page-compression failed in buildbot MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8 Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing and replaced these with dict_table_t::file_unreadable. Table ibd file is missing if fil_get_space(space_id) returns NULL and encrypted if not. Removed dict_table_t::is_corrupted field. Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(), buf_page_decrypt_after_read(), buf_page_encrypt_before_write(), buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats(). Added test cases when enrypted page could be read while doing redo log crash recovery. Also added test case for row compressed blobs. btr_cur_open_at_index_side_func(), btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is NULL. buf_page_get_zip(): Issue error if page read fails. buf_page_get_gen(): Use dberr_t for error detection and do not reference bpage after we hare freed it. buf_mark_space_corrupt(): remove bpage from LRU also when it is encrypted. buf_page_check_corrupt(): @return DB_SUCCESS if page has been read and is not corrupted, DB_PAGE_CORRUPTED if page based on checksum check is corrupted, DB_DECRYPTION_FAILED if page post encryption checksum matches but after decryption normal page checksum does not match. In read case only DB_SUCCESS is possible. buf_page_io_complete(): use dberr_t for error handling. buf_flush_write_block_low(), buf_read_ahead_random(), buf_read_page_async(), buf_read_ahead_linear(), buf_read_ibuf_merge_pages(), buf_read_recv_pages(), fil_aio_wait(): Issue error if page read fails. btr_pcur_move_to_next_page(): Do not reference page if it is NULL. Introduced dict_table_t::is_readable() and dict_index_t::is_readable() that will return true if tablespace exists and pages read from tablespace are not corrupted or page decryption failed. Removed buf_page_t::key_version. After page decryption the key version is not removed from page frame. For unencrypted pages, old key_version is removed at buf_page_encrypt_before_write() dict_stats_update_transient_for_index(), dict_stats_update_transient() Do not continue if table decryption failed or table is corrupted. dict0stats.cc: Introduced a dict_stats_report_error function to avoid code duplication. fil_parse_write_crypt_data(): Check that key read from redo log entry is found from encryption plugin and if it is not, refuse to start. PageConverter::validate(): Removed access to fil_space_t as tablespace is not available during import. Fixed error code on innodb.innodb test. Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown to innodb-bad-key-change2. Removed innodb-bad-key-change5 test. Decreased unnecessary complexity on some long lasting tests. Removed fil_inc_pending_ops(), fil_decr_pending_ops(), fil_get_first_space(), fil_get_next_space(), fil_get_first_space_safe(), fil_get_next_space_safe() functions. fil_space_verify_crypt_checksum(): Fixed bug found using ASAN where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly accessed from row compressed tables. Fixed out of page frame bug for row compressed tables in fil_space_verify_crypt_checksum() found using ASAN. Incorrect function was called for compressed table. Added new tests for discard, rename table and drop (we should allow them even when page decryption fails). Alter table rename is not allowed. Added test for restart with innodb-force-recovery=1 when page read on redo-recovery cant be decrypted. Added test for corrupted table where both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted. Adjusted the test case innodb_bug14147491 so that it does not anymore expect crash. Instead table is just mostly not usable. fil0fil.h: fil_space_acquire_low is not visible function and fil_space_acquire and fil_space_acquire_silent are inline functions. FilSpace class uses fil_space_acquire_low directly. recv_apply_hashed_log_recs() does not return anything.
* | | | MDEV-12696 Crash with LOAD XML and non-updatable VIEW columnAlexander Barkov2017-05-051-0/+9
| | | |
* | | | Merge branch '10.1' into 10.2Sergei Golubchik2017-02-103-1/+2
|\ \ \ \ | |/ / /
| * | | Merge branch '10.0' into 10.1Vicențiu Ciorbaru2017-01-163-1/+2
| |\ \ \ | | |/ /
| | * | Merge remote-tracking branch 'origin/5.5' into 10.0Vicențiu Ciorbaru2017-01-102-0/+2
| | |\ \ | | | |/
| | | * Adding LOAD DATA tests for MDEV-11079 and MDEV-11631Alexander Barkov2017-01-092-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | c3cf7f47f0f4a1ec314001aaf0c3d9c1c1f62097 reverted the patch for BUG#24487120. After merging the reverting patch from MySQL to MariaDB the problems described in MDEV-11079 and MDEV-11631 disappeared. Adding test cases only.
| | * | Merge remote-tracking branch 'origin/5.5' into 10.0vicentiu2017-01-061-1/+0
| | |\ \ | | | |/
| | | * Merge branch 'mysql/5.5' into 5.5Sergei Golubchik2016-12-221-1/+0
| | | |\
| | | | * BUG#24487120 - SLAVE'S SLAVE_SQL_RUNNING IS STOPPED DURINGThayumanavar S2016-10-281-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | LOAD DATA AT MASTER. Revert "BUG#23080148 - BACKPORT BUG 14653594 AND BUG 20683959 TO" This reverts commit 1d31f5b3090d129382b50b95512f2f79305715a1. The commit causes replication incompatibility between minor revisions and based on discussion with Srinivasarao, the patch is reverted.
* | | | | Merge branch '10.1' into 10.2Sergei Golubchik2016-12-292-0/+13
|\ \ \ \ \ | |/ / / /
| * | | | Merge branch '10.0' into 10.1Sergei Golubchik2016-12-111-0/+12
| |\ \ \ \ | | |/ / /
| | * | | MDEV-11343 LOAD DATA INFILE fails to load data with an escape character ↵Alexander Barkov2016-11-291-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | followed by a multi-byte character Partially backporting MDEV-9874 from 10.2 to 10.0 READ_INFO::read_field() raised the ER_INVALID_CHARACTER_STRING error when reading an escape character followed by a multi-byte character. Raising wellformedness errors in READ_INFO::read_field() was wrong, because the main goal of READ_INFO::read_field() is to *unescape* the data which was presumably escaped using mysql_real_escape_string(), using the same character set with the one specified in "LOAD DATA INFILE ... CHARACTER SET ..." (or assumed by default). During LOAD DATA, multi-byte characters are not always scanned as a single entity! In case of escaped data, parts of a multi-byte character can be scanned on different loop iterations. So the old code erroneously tested welformedness in the middle of a multi-byte character. Moreover, the data after unescaping can go into a BLOB field, not a text field. Wellformedness tests are meaningless in this case. Ater this patch, wellformedness is only checked later, during Field::store(str,length,cs) time. The loop that scans bytes only makes sure to revert the changes made by mysql_real_escape_string(). Note, in some cases users can supply data which did not really go through mysql_real_escape_string() and was escaped by some other means, or was not escaped at all. The file reported in this MDEV contains the string "\ä", which is an example of such improperly escaped data, as - either there should be two backslashes: "\\ä" - or there should be no backslashes at all: "ä" mysql_real_escape_string() could not generate "\ä".
| * | | | MDEV-11004: Unable to start (Segfault or os error 2) when encryption key missingJan Lindström2016-10-291-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Two problems: (1) When pushing warning to sql-layer we need to check that thd != NULL to avoid NULL-pointer reference. (2) At tablespace key rotation if used key_id is not found from encryption plugin tablespace should not be rotated.
* | | | | MDEV-11255 LDML: allow defining 2-level UCA collationsAlexander Barkov2016-11-081-0/+39
| | | | |
* | | | | MDEV-10877 xxx_unicode_nopad_ci collationsAlexander Barkov2016-09-231-0/+2
| | | | |
* | | | | Merge branch '10.1' into 10.2Sergei Golubchik2016-09-091-0/+1
|\ \ \ \ \ | |/ / / /
| * | | | Merge branch '10.0' into 10.1Sergei Golubchik2016-08-251-0/+1
| |\ \ \ \ | | |/ / /
| | * | | Merge branch '5.5' into 10.0Sergei Golubchik2016-08-101-0/+1
| | |\ \ \ | | | |/ /
| | | * | Merge branch 'mysql/5.5' into 5.5Sergei Golubchik2016-08-031-0/+1
| | | |\ \ | | | | |/ | | | | | | | | | | without a fix for Bug#12818255 (MDEV-6581)
| | | | * BUG#23080148 - BACKPORT BUG 14653594 AND BUG 20683959 TOThayumanavar S2016-06-201-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MYSQL-5.5 The bug asks for a backport of bug#1463594 and bug#20682959. This is required because of the fact that if replication is enabled, master transaction can commit whereas slave can't commit due to not exact 'enviroment'. This manifestation is seen in bug#22024200.
* | | | | MDEV-9711 NO PAD collationsAlexander Barkov2016-09-061-0/+11
| | | | | | | | | | | | | | | | | | | | Based on the patch from Daniil Medvedev (a Google Summer of Code task)
* | | | | MDEV-10743 LDML: a new syntax to reuse sort order from another 8bit simple ↵Alexander Barkov2016-09-062-0/+27
| | | | | | | | | | | | | | | | | | | | collation
* | | | | LDML refactoring for "MDEV-9711 NO PAD collations"Alexander Barkov2016-09-033-0/+150
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Moving detection of the MY_CS_CSSORT, MY_CS_PUREASCII, MY_CS_NONASCII flags of loadable collations from add_collation() in mysys.c to my_cset_init_8bit() and my_coll_init_simple() in ctype-simple.c. - Adding tests that these flags are set properly for loadable collations - Moving LDML test related *.xml files from mysql-test/std_data/ to mysql-test/std_data/ldml/, as there will be more *.xml test files
* | | | | Merge branch '10.1' into 10.2Sergei Golubchik2016-06-301-0/+10
|\ \ \ \ \ | |/ / / /
| * | | | MDEV-8686 A user defined collation utf8_confusables doesn't workAlexander Barkov2016-06-231-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The collation customization code for the UCA (Unicode Collation Alrorithm) based collations now allows to reset to and shift of characters with implicit weights. Previously reset/shift worked only for the characters with explicit DUCET weights. An attempt to use reset/shift with character with implicit weights made the server crash.
* | | | | MDEV-7563 Support CHECK constraint as in (or close to) SQL StandardMichael Widenius2016-06-303-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MDEV-10134 Add full support for DEFAULT - Added support for using tables with MySQL 5.7 virtual fields, including MySQL 5.7 syntax - Better error messages also for old cases - CREATE ... SELECT now also updates timestamp columns - Blob can now have default values - Added new system variable "check_constraint_checks", to turn of CHECK constraint checking if needed. - Removed some engine independent tests in suite vcol to only test myisam - Moved some tests from 'include' to 't'. Should some day be done for all tests. - FRM version increased to 11 if one uses virtual fields or constraints - Changed to use a bitmap to check if a field has got a value, instead of setting HAS_EXPLICIT_VALUE bit in field flags - Expressions can now be up to 65K in total - Ensure we are not refering to uninitialized fields when handling virtual fields or defaults - Changed check_vcol_func_processor() to return a bitmap of used types - Had to change some functions that calculated cached value in fix_fields to do this in val() or getdate() instead. - store_now_in_TIME() now takes a THD argument - fill_record() now updates default values - Add a lookahead for NOT NULL, to be able to handle DEFAULT 1+1 NOT NULL - Automatically generate a name for constraints that doesn't have a name - Added support for ALTER TABLE DROP CONSTRAINT - Ensure that partition functions register virtual fields used. This fixes some bugs when using virtual fields in a partitioning function
* | | | | Merge commit 'd5822a3ad0657040114cdc185c6387b9eb3a12b2' into 10.2Monty2016-04-286-0/+213
|\ \ \ \ \ | |/ / / /
| * | | | MDEV-9383: Server fails to read master.info after upgrade 10.0 -> 10.1Kristian Nielsen2016-04-086-0/+213
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some cases, MariaDB 10.0 could write a master.info file that was read incorrectly by 10.1 and could cause server to fail to start after an upgrade. (If writing a new master.info file that is shorter than the old, extra junk may remain at the end of the file. This is handled properly in 10.1 with an END_MARKER line, but this line is not written by 10.0. The fix here is to make 10.1 robust at reading the master.info files written by 10.0). Fix several things around reading master.info and read_mi_key_from_file(): - read_mi_key_from_file() did not distinguish between a line with and without an eqals '=' sign. - If a line was empty, read_mi_key_from_file() would incorrectly return the key from the previous call. - An extra using_gtid=X line left-over by MariaDB 10.0 might incorrectly be read and overwrite the correct value. - Fix incorrect usage of strncmp() which should be strcmp(). - Add test cases.
* | | | | MDEV-9874 LOAD XML INFILE does not handle well broken multi-byte charactersAlexander Barkov2016-04-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Moving the new my_charlen()-based code handling multi-byte characters from READ_INFO::field_field() to a new method READ_INFO::read_mbtail() - Reusing read_mbtail() in READ_INFO::read_value(), instead of the old my_mbcharlen()-based code which did not catch broken byte sequences