summaryrefslogtreecommitdiff
path: root/storage/connect/mongo.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Use memory safe snprintf() in Connect EngineMikhail Chalov2022-07-261-4/+4
| | | | | | | | | | | | | | | | | | | | | This commit replaces sprintf(buf, ...) with snprintf(buf, sizeof(buf), ...), specifically in the "easy" cases where buf is allocated with a size known at compile time. The changes make sure we are not write outside array/string bounds which will lead to undefined behaviour. In case the code is trying to write outside bounds - safe version of functions simply cut the string messages so we process this gracefully. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc. bsonudf.cpp warnings cleanup by Daniel Black Reviewer: Daniel Black
* - Make function strz return null when LEX_STRING is nullOlivier Bertrand2021-06-241-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | modified: storage/connect/ha_connect.cc - Use NOTE instead of WARNING in connect_assisted_discovery This because MariaDB raise an error when doing so modified: storage/connect/ha_connect.cc modified: storage/connect/tabrest.cpp - Make MONGO tables recognize STRINGIFY and JsonAllPath modified: storage/connect/mongo.cpp modified: storage/connect/mongo.h modified: storage/connect/tabcmg.h modified: storage/connect/tabjmg.cpp modified: storage/connect/tabcmg.cpp modified: storage/connect/tabjmg.h - Fix OBJECT option for Pretty != 2 JSN and BSON tables Accept all syntaxes for the OBJECT option modified: storage/connect/tabbson.cpp modified: storage/connect/tabjson.cpp - Use my_snprintf in function xcurl (by vuvova) modified: storage/connect/tabrest.cpp - Format dates entered as integer when formatted modified: storage/connect/value.cpp modified: storage/connect/value.h
* - Inline MakePtr and MakeOff with OFFSET as size_tOlivier Bertrand2020-10-181-7/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also add a new member Saved_Size in the Global structure. modified: storage/connect/global.h modified: storage/connect/plugutil.cpp modified: storage/connect/user_connect.cc modified: storage/connect/jsonudf.cpp - Add session variables json_all_path and default_depth modified: storage/connect/ha_connect.cc modified: storage/connect/mongo.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/tabxml.cpp - ADD column options JPATH and XPATH Work as FIELD_FORMAT but are more readable modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h modified: storage/connect/mysql-test/connect/r/json_java_2.result modified: storage/connect/mysql-test/connect/r/json_java_3.result modified: storage/connect/mysql-test/connect/r/json_mongo_c.result - Handle negative numbes in the option list modified: storage/connect/ha_connect.cc - Fix Json parse that could crash the server. Was because it could use THROW out of the TRY block. Also handle all error by THROW. It is now done by a new class JSON. modified: storage/connect/json.cpp modified: storage/connect/json.h - Add a new UDF function jfile_translate. It translate a Json file to pretty = 0. Fast because it does not a real parse of the file. modified: storage/connect/jsonudf.cpp modified: storage/connect/jsonudf.h - Add a now options JSIZE and STRINGIFY to Json tables. STRINGIFY makes Objects or Arrays to be returned by their json representation instead of by their concatenated values. JSIZE allows to specify the LRECL (was 256) defaults to 1024. Also fix a bug about locating the sub-table by its path. modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h
* - Make possible to allocate work space larger than 4GBOlivier Bertrand2020-10-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | All variables handling sizes that were uint are now size_t. The variable connect_work_size is now ulong (was uint); Also make Json functiosn to allocate a larger memory (M=9 was 7) modified: storage/connect/global.h modified: storage/connect/ha_connect.cc modified: storage/connect/json.cpp modified: storage/connect/jsonudf.cpp modified: storage/connect/plgdbutl.cpp modified: storage/connect/plugutil.cpp modified: storage/connect/user_connect.cc - Fix uninitialised variable (pretty) in Json_File. Make Jbin_file accept the same arguments as Json_File ones. modified: storage/connect/jsonudf.cpp - Change the Level option to Depth (the word currently used) (Level being still accepted) modified: storage/connect/mongo.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/tabxml.cpp - Suppress 2nd argument default value for MYSQLtoPLG function modified: storage/connect/myutil.h - Allow REST tables to be create not specifying a file_name modified: storage/connect/tabrest.cpp
* - Fix MDEV-22561 Unable to access DBF inside a ZIP archiveOlivier Bertrand2020-07-131-1/+0
| | | | | | | | | | | | | | | | | | | | | modified: storage/connect/filamdbf.cpp modified: storage/connect/filamdbf.h modified: storage/connect/filamzip.cpp modified: storage/connect/filamzip.h modified: storage/connect/ha_connect.cc modified: storage/connect/plgxml.cpp modified: storage/connect/tabdos.cpp modified: storage/connect/tabdos.h modified: storage/connect/tabfix.h - Add/Init Level class member modified: storage/connect/mongo.cpp modified: storage/connect/mongo.h modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h - Typo modified: storage/connect/connect.cc
* - Use delayed load for the MongoDB C Drive on WindowsOlivier Bertrand2018-01-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | modified: storage/connect/CMakeLists.txt modified: storage/connect/cmgoconn.cpp modified: storage/connect/ha_connect.cc - Add FORCE to the connect_type_conv enum values This will translate binary values to TYPE_STRING modified: storage/connect/checklvl.h modified: storage/connect/ha_connect.cc modified: storage/connect/odbconn.cpp - Change the connect_xtrace variable to from int to set modified: storage/connect/array.cpp modified: storage/connect/blkfil.cpp modified: storage/connect/block.h modified: storage/connect/cmgoconn.cpp modified: storage/connect/colblk.cpp modified: storage/connect/connect.cc modified: storage/connect/filamap.cpp modified: storage/connect/filamdbf.cpp modified: storage/connect/filamfix.cpp modified: storage/connect/filamgz.cpp modified: storage/connect/filamtxt.cpp modified: storage/connect/filamvct.cpp modified: storage/connect/filamzip.cpp modified: storage/connect/filter.cpp modified: storage/connect/global.h modified: storage/connect/ha_connect.cc modified: storage/connect/javaconn.cpp modified: storage/connect/jdbconn.cpp modified: storage/connect/jmgfam.cpp modified: storage/connect/jmgoconn.cpp modified: storage/connect/json.cpp modified: storage/connect/jsonudf.cpp modified: storage/connect/mongo.cpp modified: storage/connect/mycat.cc modified: storage/connect/myconn.cpp modified: storage/connect/odbconn.cpp modified: storage/connect/plgdbutl.cpp modified: storage/connect/plugutil.cpp modified: storage/connect/reldef.cpp modified: storage/connect/tabcol.cpp modified: storage/connect/tabdos.cpp modified: storage/connect/tabext.cpp modified: storage/connect/tabfix.cpp modified: storage/connect/tabfmt.cpp modified: storage/connect/tabjdbc.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/table.cpp modified: storage/connect/tabmul.cpp modified: storage/connect/tabmysql.cpp modified: storage/connect/tabodbc.cpp modified: storage/connect/tabpivot.cpp modified: storage/connect/tabsys.cpp modified: storage/connect/tabtbl.cpp modified: storage/connect/tabutil.cpp modified: storage/connect/tabvct.cpp modified: storage/connect/tabwmi.cpp modified: storage/connect/tabxml.cpp modified: storage/connect/user_connect.cc modified: storage/connect/valblk.cpp modified: storage/connect/value.cpp modified: storage/connect/xindex.cpp - Restore connect_enable_mongo variable (but undocumented) modified: storage/connect/ha_connect.cc modified: storage/connect/mycat.cc modified: storage/connect/mysql-test/connect/r/json_java_2.result modified: storage/connect/mysql-test/connect/r/json_java_3.result modified: storage/connect/mysql-test/connect/r/json_mongo_c.result modified: storage/connect/mysql-test/connect/r/mongo_c.result modified: storage/connect/mysql-test/connect/r/mongo_java_2.result modified: storage/connect/mysql-test/connect/r/mongo_java_3.result modified: storage/connect/mysql-test/connect/r/tbl_thread.result modified: storage/connect/mysql-test/connect/t/mongo.inc modified: storage/connect/mysql-test/connect/t/mongo_test.inc modified: storage/connect/mysql-test/connect/t/tbl_thread.test
* - Fix a bug in TYPVAL<PSZ> compute causing it sometime not to be executedOlivier Bertrand2017-10-161-2/+3
| | | | | | | | | | | | | This was the cause of the bug in CalculateArray modified: storage/connect/jsonudf.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/value.cpp - Avoid some compiler warnings modified: storage/connect/mongo.cpp modified: storage/connect/tabjmg.cpp modified: storage/connect/tabtbl.cpp modified: storage/connect/tabutil.cpp
* - Update version numberOlivier Bertrand2017-10-151-6/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | modified: storage/connect/ha_connect.cc - Include MONGO in all Java enabled distributions Mongo will be enabled only for 10.2 and 10.3 modified: storage/connect/CMakeLists.txt - Change JDBC_SUPPORT to JAVA_SUPPORT which also replaces MONGO_SUPPORT MONGO_SUPPORT is now just used to enable the MONGO table type modified: storage/connect/filter.cpp modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h modified: storage/connect/mongo.cpp modified: storage/connect/mycat.cc modified: storage/connect/plgdbutl.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h - Move MakeSelector function from FILTER to mongo.cpp modified: storage/connect/filter.cpp modified: storage/connect/filter.h modified: storage/connect/cmgoconn.cpp modified: storage/connect/jmgoconn.cpp modified: storage/connect/mongo.cpp - Do mongo_init only on first use of the MongoDB C Driver This will permit to delay load the mongo lib on Windows modified: storage/connect/cmgoconn.cpp modified: storage/connect/cmgoconn.h modified: storage/connect/ha_connect.cc - Replace NEW_VAR by a test on MYSQL_VERSION_ID modified: storage/connect/ha_connect.cc - Suppress enable_mongo session variable modified: storage/connect/ha_connect.cc modified: storage/connect/mycat.cc - Make some function headers identical in .h and .cc file (replacing const char* by PCSZ) modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h - Change a parameter type from uchar* to const uchar* (for ScanRecord and CheckRecord) modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h - Changes on LIKE and NOT LIKE does not fix a bug yet modified: storage/connect/ha_connect.cc - Suppress PIVOT_SUPPORT (PIVOT type is unconditionnal) modified: storage/connect/ha_connect.cc modified: storage/connect/mycat.cc - Change the strz function from inline to static modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h - export the JavaConn class and the MgoColumns and IsNum functions modified: storage/connect/javaconn.h modified: storage/connect/json.h modified: storage/connect/mongo.h - Fix MDEV-13924 modified: storage/connect/jdbconn.cpp - Make a temporary fix for the compiler bug in CalculateArray modified: storage/connect/jsonudf.cpp modified: storage/connect/tabjson.cpp - Typo modified: storage/connect/jdbccat.h modified: storage/connect/reldef.h modified: storage/connect/tabext.h modified: storage/connect/tabjmg.cpp modified: storage/connect/tabxml.h modified: storage/connect/valblk.h modified: storage/connect/value.h modified: storage/connect/xtable.h - Fix a bug in MONGO tests by changing 'MONGO' to $TYPE modified: storage/connect/mysql-test/connect/t/mongo_test.inc - Record test results to reflect all changes modified: storage/connect/mysql-test/connect/r/json_java_2.result modified: storage/connect/mysql-test/connect/r/json_java_3.result modified: storage/connect/mysql-test/connect/r/json_mongo_c.result modified: storage/connect/mysql-test/connect/r/mongo_c.result modified: storage/connect/mysql-test/connect/r/mongo_java_2.result modified: storage/connect/mysql-test/connect/r/mongo_java_3.result
* - Fix gcc compile error: crosses initialization of ‘const char* drv’Olivier Bertrand2017-08-091-4/+5
| | | | | | | | | | | modified: storage/connect/mongo.cpp - Fix test failures modified: storage/connect/mysql-test/connect/r/mysql.result modified: storage/connect/mysql-test/connect/r/mysql_new.result modified: storage/connect/mysql-test/connect/r/tbl_thread.result modified: storage/connect/mysql-test/connect/t/tbl_thread.test modified: storage/connect/tabjson.cpp
* - Add FBLOCK when opening ODBC, JSON and MONGO tables.Olivier Bertrand2017-08-051-36/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This to have automatic closing in case of thrown error. modified: storage/connect/cmgoconn.cpp modified: storage/connect/cmgoconn.h modified: storage/connect/filamzip.cpp modified: storage/connect/javaconn.cpp modified: storage/connect/javaconn.h modified: storage/connect/odbconn.cpp modified: storage/connect/odbconn.h modified: storage/connect/plgdbsem.h modified: storage/connect/plgdbutl.cpp - Fix INCOL bug when inserting several lines to MONGO tables modified: storage/connect/cmgoconn.cpp modified: storage/connect/tabcmg.cpp modified: storage/connect/tabjmg.cpp - MONGO_SUPPORT is set for both MongoDB drivers CMGO_SUPPORT is set only when the C driver is available modified: storage/connect/CMakeLists.txt modified: storage/connect/filter.h modified: storage/connect/filter.h modified: storage/connect/ha_connect.cc modified: storage/connect/mongo.cpp modified: storage/connect/mycat.cc modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h - Separate enums JCATINFO and modified: storage/connect/javaconn.h modified: storage/connect/jdbconn.cpp - Fix crash when executing JDBC catfunc=driver modified: storage/connect/jdbconn.cpp - Report an error when the Mongo driver if explicitly specified and not supported. modified: storage/connect/mongo.cpp - Fix bug causing catalog JSON tables to fail modified: storage/connect/tabjson.cpp - Protect by mutex the Ready variable of the TBLTBM table type modified: storage/connect/tabtbl.cpp - Put testing of Thread TBL tables in a separate test With added case and modified old case that could avoid the test to fail. modified: storage/connect/mysql-test/connect/r/tbl.result modified: storage/connect/mysql-test/connect/t/tbl.test new file: storage/connect/mysql-test/connect/r/tbl_thread.result new file: storage/connect/mysql-test/connect/t/tbl_thread.test - jmongo3.test no more exists deleted: storage/connect/mysql-test/connect/t/jmongo3.test - Add new tests for the MONGO feature (disabled) modified: storage/connect/mysql-test/connect/disabled.def new file: storage/connect/mysql-test/connect/r/json_java_2.result new file: storage/connect/mysql-test/connect/r/json_java_3.result new file: storage/connect/mysql-test/connect/r/json_mongo_c.result new file: storage/connect/mysql-test/connect/r/mongo_c.result new file: storage/connect/mysql-test/connect/r/mongo_java_2.result new file: storage/connect/mysql-test/connect/r/mongo_java_3.result new file: storage/connect/mysql-test/connect/std_data/Mongo2.jar new file: storage/connect/mysql-test/connect/std_data/cities.json new file: storage/connect/mysql-test/connect/t/json_java_2.test new file: storage/connect/mysql-test/connect/t/json_java_3.test new file: storage/connect/mysql-test/connect/t/json_mongo_c.test new file: storage/connect/mysql-test/connect/t/mongo.inc new file: storage/connect/mysql-test/connect/t/mongo_c.test new file: storage/connect/mysql-test/connect/t/mongo_java_2.test new file: storage/connect/mysql-test/connect/t/mongo_java_3.test new file: storage/connect/mysql-test/connect/t/mongo_test.inc
* -- Fix wrong setting of PipeOlivier Bertrand2017-07-281-2/+1
| | | | | | | | | | | | modified: storage/connect/tabcmg.cpp -- Fix declaration of drv in MGOColumns modified: storage/connect/mongo.cpp -- Add jmongo3 test (disabled) modified: storage/connect/mysql-test/connect/disabled.def added: storage/connect/mysql-test/connect/std_data/Mongo3.jar added: storage/connect/mysql-test/connect/t/jmongo3.test
* -- Finalize work on MongoDB accessOlivier Bertrand2017-07-181-3/+256
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement discovery for the MongoDB Java Driver Create classes to minimize code and avoid dupicates Rearrange and rename implied files modified: storage/connect/CMakeLists.txt renamed: storage/connect/mongofam.cpp -> storage/connect/cmgfam.cpp renamed: storage/connect/mongofam.h -> storage/connect/cmgfam.h modified: storage/connect/cmgoconn.h modified: storage/connect/javaconn.h modified: storage/connect/jdbconn.cpp modified: storage/connect/jmgoconn.cpp modified: storage/connect/jmgoconn.h modified: storage/connect/mongo.cpp modified: storage/connect/mongo.h renamed: storage/connect/tabmgo.cpp -> storage/connect/tabcmg.cpp renamed: storage/connect/tabmgo.h -> storage/connect/tabcmg.h modified: storage/connect/tabjmg.cpp modified: storage/connect/tabjmg.h modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h -- Trace Sarea allocation and freeing modified: storage/connect/connect.cc modified: storage/connect/plugutil.cpp modified: storage/connect/user_connect.cc -- Null Json values where not mark as null in JSNX::SetJsonValue This was added in TYPE_NULL (declared as TYPE_VOID) modified: storage/connect/json.cpp modified: storage/connect/json.h modified: storage/connect/jsonudf.cpp -- Null JValues are ignored in JSNX::CalculateArray Also done in tabjson.cpp for JSONCOL::CalculateArray modified: storage/connect/json.h modified: storage/connect/jsonudf.cpp modified: storage/connect/tabjson.cpp -- Null JSON values now represented by connect_json_null session variable modified: storage/connect/json.cpp modified: storage/connect/json.h modified: storage/connect/jsonudf.cpp modified: storage/connect/tabjson.cpp -- JVALUE has size = 1 modified: storage/connect/json.h -- Fix by vuvova because Debian compilation failure. modified: storage/connect/value.cpp
* - Suppress warning when creating mongo JSON tablesOlivier Bertrand2017-07-031-1/+5
| | | | | | | | | | modified: storage/connect/ha_connect.cc - Add catalog table for Mongo java driver tables modified: storage/connect/mongo.cpp modified: storage/connect/mongo.h modified: storage/connect/tabjmg.cpp modified: storage/connect/tabjmg.h
* - Add Support of the MongoDB Java Driver.Olivier Bertrand2017-07-021-0/+99
modified: storage/connect/CMakeLists.txt modified: storage/connect/JavaWrappers.jar modified: storage/connect/colblk.h modified: storage/connect/filter.cpp modified: storage/connect/filter.h modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h modified: storage/connect/jdbccat.h modified: storage/connect/jdbconn.cpp modified: storage/connect/jdbconn.h modified: storage/connect/mongofam.cpp modified: storage/connect/mongofam.h modified: storage/connect/mycat.cc modified: storage/connect/mycat.h modified: storage/connect/tabext.h modified: storage/connect/tabjdbc.cpp modified: storage/connect/tabjdbc.h modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h modified: storage/connect/tabmgo.cpp modified: storage/connect/tabmgo.h created: storage/connect/Mongo2Interface.java created: storage/connect/Mongo3Interface.java created: storage/connect/cmgoconn.cpp created: storage/connect/cmgoconn.h created: storage/connect/javaconn.cpp created: storage/connect/javaconn.h created: storage/connect/jmgfam.cpp created: storage/connect/jmgfam.h created: storage/connect/jmgoconn.cpp created: storage/connect/jmgoconn.h created: storage/connect/mongo.cpp created: storage/connect/mongo.h created: storage/connect/tabjmg.cpp created: storage/connect/tabjmg.h - tdbp not initialized when catched exception in CntGetTDB (connect.cc line 188) modified: storage/connect/connect.h - CheckCleanup should sometimes doing cleanup on pure info Sometimes MariaDB loops on info to get the size of all tables in a database. This can sometimes fail by exhausted memory. CheckCleanup now have a force boolean parameter (defaulting to false) modified: storage/connect/ha_connect.cc modified: storage/connect/user_connect.cc modified: storage/connect/user_connect.h Change the copyright of some source files modified: storage/connect/connect.cc modified: storage/connect/connect.h modified: storage/connect/engmsg.h modified: storage/connect/global.h modified: storage/connect/ha_connect.cc modified: storage/connect/ha_connect.h modified: storage/connect/msgid.h modified: storage/connect/mycat.cc modified: storage/connect/mycat.h modified: storage/connect/os.h modified: storage/connect/osutil.c modified: storage/connect/osutil.h modified: storage/connect/user_connect.cc modified: storage/connect/user_connect.h