summaryrefslogtreecommitdiff
path: root/src/include/mb
Commit message (Collapse)AuthorAgeFilesLines
* Fix machine-dependent crash in sqlchar_to_unicode(). Get rid ofTom Lane2006-12-241-1/+3
| | | | | | | bletcherous and unsafe manipulation of global encoding setting. Clean up libxml reporting mechanism a bit (it still looks like a dangling-pointer crash waiting to happen, though, not to mention being far less than sane from a localization standpoint).
* pgindent run for 8.2.Bruce Momjian2006-10-041-23/+23
|
* Change the backend to reject strings containing invalidly-encoded multibyteTom Lane2006-05-211-24/+44
| | | | | | | | | | | | | | | | | | | | characters in all cases. Formerly we mostly just threw warnings for invalid input, and failed to detect it at all if no encoding conversion was required. The tighter check is needed to defend against SQL-injection attacks as per CVE-2006-2313 (further details will be published after release). Embedded zero (null) bytes will be rejected as well. The checks are applied during input to the backend (receipt from client or COPY IN), so it no longer seems necessary to check in textin() and related routines; any string arriving at those functions will already have been validated. Conversion failure reporting (for characters with no equivalent in the destination encoding) has been cleaned up and made consistent while at it. Also, fix a few longstanding errors in little-used encoding conversion routines: win1251_to_iso, win866_to_iso, euc_tw_to_big5, euc_tw_to_mic, mic_to_euc_tw were all broken to varying extents. Patches by Tatsuo Ishii and Tom Lane. Thanks to Akio Ishida and Yasuo Ohgaki for identifying the security issues.
* Add support for Windows codepages 1253, 1254, 1255, and 1257 and cleanPeter Eisentraut2006-02-181-2/+6
| | | | | | | | | | | | | | | | | | | | | up a bunch of the support utilities. In src/backend/utils/mb/Unicode remove nearly duplicate copies of the UCS_to_XXX perl script and replace with one version to handle all generic files. Update the Makefile so that it knows about all the map files. This produces a slight difference in some of the map files, using a uniform naming convention and not mapping the null character. In src/backend/utils/mb/conversion_procs create a master utf8<->win codepage function like the ISO 8859 versions instead of having a separate handler for each conversion. There is an externally visible change in the name of the win1258 to utf8 conversion. According to the documentation notes, it was named incorrectly and this changes it to a standard name. Running the Unicode mapping perl scripts has shown some additional mapping changes in koi8r and iso8859-7.
* Previous commit message should have been:Bruce Momjian2005-12-241-2/+2
| | | | Add comment marker for PG_ENCODING_BE_LAST.
* AddBruce Momjian2005-12-241-2/+3
|
* Alignment cleanup.Bruce Momjian2005-12-241-43/+43
|
* Standard pgindent run for 8.1.Bruce Momjian2005-10-151-18/+16
|
* Suppress signed-vs-unsigned-char warnings.Tom Lane2005-09-241-14/+14
|
* Actually, this macro had worse problems than a bogus name ...Tom Lane2005-08-051-3/+2
|
* Fix misspelled macro name. Doesn't appear to be used anywhere yet,Tom Lane2005-08-051-2/+2
| | | | so no one noticed.
* Support 3 and 4-byte unicode characters.Bruce Momjian2005-06-151-1/+3
| | | | John Hansen
* Add support for Win1252 encoding.Bruce Momjian2005-03-141-2/+3
| | | | Roland Volkmann
* Rename canonical encodings, per Peter:Bruce Momjian2005-03-071-8/+8
| | | | | | | | | UNICODE => UTF8 ALT => WIN866 WIN => WIN1251 TCVN => WIN1258 The old codes continue to work.
* Back out addition of Win1252 encoding.Bruce Momjian2004-12-041-2/+1
|
* Back out fix for Unicode characters above 0x10000Bruce Momjian2004-12-031-11/+1
|
* Fix for Unicode characters above 0x10000.Bruce Momjian2004-12-021-1/+11
| | | | John Hansen
* Add Charset WIN1252 support.Bruce Momjian2004-12-021-1/+2
| | | | Roland Volkmann
* Allow WIN1250 as server encoding.Peter Eisentraut2004-09-171-3/+3
|
* Pgindent run for 8.0.Bruce Momjian2004-08-291-2/+3
|
* Add PQmbdsplen() which returns the "display length" of a character.Tatsuo Ishii2004-03-151-1/+6
| | | | | Still some works needed: - UTF-8, MULE_INTERNAL always returns 1
* make sure the $Id tags are converted to $PostgreSQL as well ...PostgreSQL Daemon2003-11-291-1/+1
|
* pgindent run.Bruce Momjian2003-08-041-2/+2
|
* Error message editing in utils/adt. Again thanks to Joe Conway for doingTom Lane2003-07-271-2/+2
| | | | the bulk of the heavy lifting ...
* In src/include/mb/pg_wchar.h we have:Bruce Momjian2003-06-021-2/+2
| | | | | | | | | | | | | | | | | | | | | | #define PG_ENCODING_BE_LAST PG_ISO_8859_8 #define PG_ENCODING_FE_LAST PG_WIN1256 but the last client encoding in the enum list is actually PG_GB18030 and it seems that #define PG_ENCODING_IS_CLIEN_ONLY(_enc) \ (((_enc) > PG_ENCODING_BE_LAST && (_enc) <= PG_ENCODING_FE_LAST) can never be true. I think the define should read #define PG_ENCODING_FE_LAST PG_GB18030 On the other hand, perhaps no-one cares, because PG_ENCODING_IS_CLIEN_ONLY is never used. -- Oliver Elphick Oliver.Elphick@lfix.co.uk
* Fix for GUC client_encoding variable not being handledTatsuo Ishii2003-02-191-1/+2
| | | | | | | | correctly. See following thread for more details. Subject: [HACKERS] client_encoding directive is ignored in postgresql.conf From: Tatsuo Ishii <t-ishii@sra.co.jp> Date: Wed, 29 Jan 2003 22:24:04 +0900 (JST)
* pgindent run.Bruce Momjian2002-09-041-16/+16
|
* Remove all traces of multibyte and locale options. Clean up commentsPeter Eisentraut2002-09-031-6/+6
| | | | referring to "multibyte" where it really means character encoding.
* Add Cyrillic and other encodings for encoding conversion.Tatsuo Ishii2002-08-141-1/+5
| | | | Patches submitted by Kaori Inaba (i-kaori@sra.co.jp).
* Fix typo. Remove #ifdef MULTIBYTETatsuo Ishii2002-07-291-35/+25
|
* I have committed many support files for CREATE CONVERSION. DefaultTatsuo Ishii2002-07-181-32/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | conversion procs and conversions are added in initdb. Currently supported conversions are: UTF-8(UNICODE) <--> SQL_ASCII, ISO-8859-1 to 16, EUC_JP, EUC_KR, EUC_CN, EUC_TW, SJIS, BIG5, GBK, GB18030, UHC, JOHAB, TCVN EUC_JP <--> SJIS EUC_TW <--> BIG5 MULE_INTERNAL <--> EUC_JP, SJIS, EUC_TW, BIG5 Note that initial contents of pg_conversion system catalog are created in the initdb process. So doing initdb required is ideal, it's possible to add them to your databases by hand, however. To accomplish this: psql -f your_postgresql_install_path/share/conversion_create.sql your_database So I did not bump up the version in cataversion.h. TODO: Add more conversion procs Add [CASCADE|RESTRICT] to DROP CONVERSION Add tuples to pg_depend Add regression tests Write docs Add SQL99 CONVERT command? -- Tatsuo Ishii
* Add GB18030 support. Contributed by Bill Huang <bill_huanghb@ybb.ne.jp>Tatsuo Ishii2002-06-131-2/+2
| | | | (ODBC support has not been committed yet. left for Hiroshi...)
* > Tatsuo Ishii wrote:Bruce Momjian2002-03-051-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > > > > It was made to cope with encoding such as an Asian bloc in 7.2Beta2. > > > > > > > > Added ServerEncoding > > > > Korean (JOHAB), Thai (WIN874), > > > > Vietnamese (TCVN), Arabic (WIN1256) > > > > > > > > Added ClientEncoding > > > > Simplified Chinese (GBK), Korean (UHC) > > > > > > > > > > > > > http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2b2.newencoding.diff.tar.gz > > > > (608K) > > > > > > Looks good. I need some people to review this for me. > > > > For me they look good too. The only missing part is a > > documentation. I will ask him to write it up. If he couldn't, I will > > do it for him. > > > The diff is 3mb > > > but appears to address only additions to multibyte. I have attached a > > > list of files it modifies. Also, look at the sizes of the mb/ > > > directory. It is getting large: > > > > > > 4 ./CVS > > > 6 ./Unicode/CVS > > > 3433 ./Unicode > > > 6197 . > > > > Yes. We definitely need the on-the-fly encoding addition capability: > > i.e. CREATE CHRACTER SET in the future... > > -- > > Tatsuo Ishii > > > > Address chainge. http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2.newencoding.diff.gz Add PsqlODBC and document ...etc patch. Eiji Tokuya
* New pgindent run with fixes suggested by Tom. Patch manually reviewed,Bruce Momjian2001-11-051-4/+4
| | | | initdb/regression tests pass.
* Another pgindent run. Fixes enum indenting, and improves #endifBruce Momjian2001-10-281-35/+36
| | | | spacing. Also adds space for one-line comments.
* pgindent run on all C files. Java run to follow. initdb/regressionBruce Momjian2001-10-251-119/+121
| | | | tests pass.
* Ok, here is the modified encoding table (column1 is the standard name,Tatsuo Ishii2001-10-161-13/+13
| | | | | | | | | | | | | | | | | | | | | | | 2 is our "official" name, and 3 is alias). If there's no objection, I will change them. ASCII SQL_ASCII UTF-8 UNICODE UTF_8 MULE-INTERNAL MULE_INTERNAL ISO-8859-1 LATIN1 ISO_8859_1 ISO-8859-2 LATIN2 ISO_8859_2 ISO-8859-3 LATIN3 ISO_8859_3 ISO-8859-4 LATIN4 ISO_8859_4 ISO-8859-5 ISO_8859_5 ISO-8859-6 ISO_8859_6 ISO-8859-7 ISO_8859_7 ISO-8859-8 ISO_8859_8 ISO-8859-9 LATIN5 ISO_8859_9 ISO-8859-10 LATIN6 ISO_8859_10 ISO-8859-13 LATIN7 ISO_8859_13 ISO-8859-14 LATIN8 ISO_8859_14 ISO-8859-15 LATIN9 ISO_8859_15 ISO-8859-16 LATIN10 ISO_8859_16
* Add support for ISO-8859-6 to 16Tatsuo Ishii2001-10-111-11/+98
|
* Add pg_database_encoding_max_length() function.Tatsuo Ishii2001-09-231-1/+2
|
* Fix type_maximum_size() to give the right answer in MULTIBYTE cases.Tom Lane2001-09-211-14/+24
| | | | Avoid use of prototype-less function pointers in MB code.
* Implement following item in TODO:Tatsuo Ishii2001-09-111-1/+5
| | | | * Reject character sequences those are not valid in their charset
* Commit Karel's patch.Tatsuo Ishii2001-09-061-67/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ------------------------------------------------------------------- Subject: Re: [PATCHES] encoding names From: Karel Zak <zakkr@zf.jcu.cz> To: Peter Eisentraut <peter_e@gmx.net> Cc: pgsql-patches <pgsql-patches@postgresql.org> Date: Fri, 31 Aug 2001 17:24:38 +0200 On Thu, Aug 30, 2001 at 01:30:40AM +0200, Peter Eisentraut wrote: > > - convert encoding 'name' to 'id' > > I thought we decided not to add functions returning "new" names until we > know exactly what the new names should be, and pending schema Ok, the patch not to add functions. > better > > ...(): encoding name too long Fixed. I found new bug in command/variable.c in parse_client_encoding(), nobody probably never see this error: if (pg_set_client_encoding(encoding)) { elog(ERROR, "Conversion between %s and %s is not supported", value, GetDatabaseEncodingName()); } because pg_set_client_encoding() returns -1 for error and 0 as true. It's fixed too. IMHO it can be apply. Karel PS: * following files are renamed: src/utils/mb/Unicode/KOI8_to_utf8.map --> src/utils/mb/Unicode/koi8r_to_utf8.map src/utils/mb/Unicode/WIN_to_utf8.map --> src/utils/mb/Unicode/win1251_to_utf8.map src/utils/mb/Unicode/utf8_to_KOI8.map --> src/utils/mb/Unicode/utf8_to_koi8r.map src/utils/mb/Unicode/utf8_to_WIN.map --> src/utils/mb/Unicode/utf8_to_win1251.map * new file: src/utils/mb/encname.c * removed file: src/utils/mb/common.c -- Karel Zak <zakkr@zf.jcu.cz> http://home.zf.jcu.cz/~zakkr/ C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz
* Add conver/convert2 functions. They are similar to the SQL99's convert.Tatsuo Ishii2001-08-151-1/+3
|
* TODO item:Tatsuo Ishii2001-07-151-1/+2
| | | | * Make n of CHAR(n)/VARCHAR(n) the number of letters, not bytes
* BTW it does not add encodign it just patches existing one (KOI8) toBruce Momjian2001-05-031-2/+3
| | | | | | | support two - KOI8-R and KOI8-U (latter is superset of the former if not to take to the account pseudographics) Andy Rysin
* pgindent run. Make it all clean.Bruce Momjian2001-03-221-14/+15
|
* Restructure the key include files per recent pghackers discussion: thereTom Lane2001-02-101-2/+1
| | | | | | | | | | | are now separate files "postgres.h" and "postgres_fe.h", which are meant to be the primary include files for backend .c files and frontend .c files respectively. By default, only include files meant for frontend use are installed into the installation include directory. There is a new make target 'make install-all-headers' that adds the whole content of the src/include tree to the installed fileset, for use by people who want to develop server-side code without keeping the complete source tree on hand. Cleaned up a whole lot of crufty and inconsistent header inclusions.
* Extend CREATE DATABASE to allow selection of a template database to beTom Lane2000-11-141-3/+1
| | | | | | | | | | cloned, rather than always cloning template1. Modify initdb to generate two identical databases rather than one, template0 and template1. Connections to template0 are disallowed, so that it will always remain in its virgin as-initdb'd state. pg_dumpall now dumps databases with restore commands that say CREATE DATABASE foo WITH TEMPLATE = template0. This allows proper behavior when there is user-added data in template1. initdb forced!
* Add support for code conversion between Unicode and other encodings.Tatsuo Ishii2000-10-301-5/+9
| | | | | | Supported encodings are: EUC_JP, EUC_CN, EUC_KR, EUC_TW, Shift JIS, Big5, ISO8859-[1-5]. TODO: testings! and documentations...
* Remove gcc-only macro definitionTatsuo Ishii2000-10-271-2/+2
|
View Lorry Controller Status