diff options
Diffstat (limited to 'Docs/net_doc.txt')
| -rw-r--r-- | Docs/net_doc.txt | 945 |
1 files changed, 0 insertions, 945 deletions
diff --git a/Docs/net_doc.txt b/Docs/net_doc.txt deleted file mode 100644 index 8a25ef41d06..00000000000 --- a/Docs/net_doc.txt +++ /dev/null @@ -1,945 +0,0 @@ - MySQL Client - Server Protocol Ducumentation - - - -Introduction ------------- - - -This paper has an objective of a through description of the -client - server protocol which is embodied in MySQL. Particularly, -this paper aims to document and describe: - -- manner in which MySQL server detects client connection requests and - creates connection -- manner in which MySQL client C API call connects to server - the - entire protocol of sending / receiving data by MySQL server and C API - code -- manner in which queries are sent by client C API calls to server -- manner in which query results are sent by server -- manner in which query results are resolved by server -- sending and receiving of error messages - - -This paper does not have the goal or describing nor documenting other -related MySQL issues, like usage of thread libraries, MySQL standard -library set, MySQL strings library and other MySQL specific libraries, -type definitions and utilities. - -Issues that are covered by this paper are contained in the following -source code files: - -- client/net.c and sql/net_serv.c, the two being identical -- client/libmysql.c (not entire file is covered) -- include/mysql_com.h -- include/mysql.h -- sql/mysqld.cc (not entire file is covered) -- sql/net_pkg.cc -- sql/sql_base.cc (not entire file is covered) -- sql/sql_select.cc (not entire file is covered) -- sql/sql_parse.cc (not entire file is covered) - -Beside this introduction this paper presents basic definitions, -constants, structures and global variables, all related functions in -server and in C API. Textual description of the entire protocol -functioning is described in the last chapter of this paper. - - -Constants, structures and global variables ------------------------------------------- - -This chapter will describe all constants, structures and -global variables relevant to client - server protocol. - -Constants - -They are important as they contain default values, the ones -that are valied if options are not set in any other way. Beside that -MySQL source code does not contain a single non-defined constant in -it's code. This description of constants does not include -configuration and conditional compilation #defines. - -NAME_LEN - field and table name length, current value 64 -HOSTNAME_LENGTH - length of the host name, current value 64 -USERNAME_LENGTH - user name length, current vluae 16 -MYSQL_PORT - default TCP/IP port number, current value 3306 -MYSQL_UNIX_ADDR - full path of the default Unix socket file, current value - "/tmp/mysql.sock" -MYSQL_NAMEDPIPE - full path of the default NT pipe file, current value - "MySQL" -MYSQL_SERVICENAME - name of the MySQL Service on NT, current value "MySql" -NET_HEADER_SIZE - size of the network header, when no - compression is used, current value 4 -COMP_HEADER_SIZE - additional size of network header when - compression is used, current value 3 - -What follows are set of constants, defined in source only, which -define capabilities of the client built with that version of C -API. Simply, when some new feature is added in client, that client -feature is defined, so that server can detect what capabilities a -client program has. - -CLIENT_LONG_PASSWORD - client supports new more secure passwords -CLIENT_LONG_FLAG - client uses longer flags -CLIENT_CONNECT_WITH_DB - client can specify db on connect -CLIENT_COMPRESS - client can use compression protocol -CLIENT_ODBC - ODBC client -CLIENT_LOCAL_FILES - client can use LOAD DATA INFILE LOCAL -CLIENT_IGNORE_SPACE - client can Ignore spaces before '(' -CLIENT_CHANGE_USER - client supports the mysql_change_user() - -What follows are other constants, pertaining to timeouts and sizes - -MYSQL_ERRMSG_SIZE - maximum size of error message string, current value 200 -NET_READ_TIMEOUT - read timeout, current value 30 sec. -NET_WRITE_TIMEOUT - write timeout, current value 60 sec. -NET_WAIT_TIMEOUT - wait for new query timeout, current value 8*60*60 - sec. i.e. 8 hours -packet_error - value returned in case of socket errors, current - value -1 -TES_BLOCKING - used in debug mode for setting up blocking testing -RETRY COUNT - number of times network read and write will be - retried, current value 1 - -There are also error messages for last_errno, which depict system -errors, and are used on the server only. - -ER_NET_PACKAGE_TOO_LARGE - packet is larger then max_allowed_packet -ER_OUT_OF_RESOURCES - practically no more memory -ER_NET_ERROR_ON_WRITE - error in writing to NT Named Pipe -ER_NET_WRITE_INTERRUPTED - some signal or interrupt happened - during write -ER_NET_READ_ERROR_FROM_PIPE - error in reading from NT Named Pipe -ER_NET_FCNTL_ERROR - error in trying to set fcntl on socket - descriptor -ER_NET_PACKETS_OUT_OF_ORDER - packet numbers on client and - server side differ -ER_NET_UNCOMPRESS_ERROR - error in uncompress of compressed packet - - - Structs and eunms - - -struct NET - -this is MySQL's network handle structure, used in all client / server -read/write functions. On the server it is initialized and preserved in -each thread. On the client, it is a part of MYSQL struct, which is -MySQL handle used in all C API functions. This structure uniquely -identifies a connection, either on the server or client side. -This structure consists of the following -fields: - - Vio* vio; - explained above - HANDLE hPipe - Handle for NT Named Pipe file - my_socket fd - file descriptor used for both tcp socket and Unix socket file - int fcntl - contains info on fcntl options used on fd. Mostly - used for saving info if blocking is used or not - unsigned char *buff - network buffer used for storing data for - reading from / writing to socket - unsigned char,*buff_end - points to the end of buff - unsigned char *write_pos - present writing position in buff - unsigned char *read_pos - present reading postiion in - buff. This pointer is used for - reading data after calling - my_net_read function and function - that are just it's wrappers - char last_error[MYSQL_ERRMSG_SIZE] - holds last error message - unsigned int last_errno - holds last error code of the network - protocol. It's ossible values are - listed in above constants. It is - used only on the server side - unsigned int max_packet - holds current value of buff size - unsigned int timeout - stores read timeout value for that connection - unsigned int pkt_nr - stores a value of the current packet - number in a batch of packets. Used - primarily for detection of protocol - errors resulting in a mismatch - my_bool error - holds either 1 or 0 depending on the error condition - my_bool return_errno - if it's value != 0 then there is an - error in protocol mismatch between - client and server - my_bool compress - if true compression is used in the protocol - unsigned long remain_in_buf - used only in reading compressed - packets. Explained in my_net_read - unsigned long length - used only for storing a length of the - read packet. Explained in my_net_read - unsigned long buf_length - used only in reading compressed - packets. Explained in my_net_read - unsigned long where_b - used only in reading compressed - packets. Explained in my_net_read - short int more - used for reporting in mysql_list_processes - char save_char - used in reading compressed packets for saving - chars in order to make zero-delimited - strings. Explained in my_net_read - -Few typedefs will be defined for easier understanding of the text that -follows. - - typedef char **MYSQL_ROW - data containing one row of values - typedef unsigned int MYSQL_FIELD_OFFSET - offset in bytes of - the current field - typedef MYSQL_ROWS *MYSQL_ROW_OFFSET - offset in bytes of - the current row - - - struct MYSQL_FIELD - contains all info on the attributes of a -specific column in a result set, plus info on lengths of the column in -a result set. This struct is tagged as st_mysql_field. This structure -consists of the following fields: - - char *name - name of column - char *table - table of column if column was a field and not - expression or constant - char *def - default value (set by mysql_list_fields) - enum enum_field_types type - see above - unsigned int length - width of column in a current row - unsigned int max_length - maximum width of that column in - entire result set - unsigned int flags - corresponding to Extra in DESCRIBE - unsigned int decimals - number of decimals in field - - - struct MYSQL_ROWS - a node for each row in the single linked -list forming entire result set. This struct is tagged as -st_mysql_rows, and has two fields: - - struct st_mysql_rows *next - pointer to a next one - MYSQL_ROW data - see above - - - struct MYSQL_DATA - contains all rows from result set. It is -tagged as st_mysql_data and has following fields: - - my_ulonglong rows - how many rows - unsigned int fields - how many columns - MYSQL_ROWS *data - see above. This is a first node of the - linked list - MEM_ROOT alloc - MEM_ROOT is MySQL memory allocation - structure, and this field is used to store - all fields and rows. - - - struct st_mysql_options - holds various client options, and -contains following fields: - - unsigned int connect_timeout - time in sec. for cennection - unsigned int client_flag - used to cold client capabilities - my_bool compress - boolean for compression - my_bool named_pipe - is Named Pipe used on NT - unsigned int port - what TCP port is used - char *host - host to connect to - char *init_command - command to be executed upon connection - char *user - account name on MySQL server - char *password - password for the above - char *unix_socket - full path for Unix socket file - char *db - default database - char *my_cnf_file - optional configuration file - char *my_cnf_group - optional header for options - - - struct MYSQL - MySQL client's handle. Required for any -operation issed from client to server. Tagged as st_mysql and having -following fields: - - NET net - see above - char *host - host on which MySQL server is running - char *user - MySQL user name - char *passwd - password for above - char *unix_socket- full path of Unix socket file - char *server_version - version of the server - char *host_info - contains info on how has connection been - established, TCP port, socket or Named Pipe - char *info - used to store information on the query results, - like number of rows affected etc. - char *db - current database - unsigned int port -= TCP port in use - unsigned int client_flag - client capabilities - unsigned int server_capabilities - server capabilities - unsigned int protocol_version - version of the protocl - unsigned int field_count - used for storing number of fields - immidiately upon execution of a - query, but before fetching rows - unsigned long thread_id - server thread to which this connection is attached - my_ulonglong affected_rows - used for storing number of rows - immidiately upon execution of a - query, but before fetching rows - my_ulonglong insert_id - fetching LAST_INSERT_ID() through - client C API - my_ulonglong extra_info - used by mysqlshow - unsigned long packet_length - saving size of the first packet - upon execution of a query - enum mysql_status status - see above - MYSQL_FIELD *fields - see above - MEM_ROOT field_alloc - memory used for storing previous field - (fields) - my_bool free_me - boolean that flags if MYSQL was allocated in - mysql_init - my_bool reconnect - used to automatically reconnect - struct st_mysql_options options - see above - char scramble_buff[9] - key for scrambling password before - sending it to server - - - struct MYSQL_RES - tagged as st_mysql_res and used to store -entire result set from a single query. Contains following fields: - - my_ulonglong row_count - number of rows - unsigned int field_count - number of columns - unsigned int current_field - cursor for fetching fields - MYSQL_FIELD *fields - see above - MYSQL_DATA *data - see above, and used in buffered reads, - i.e. mysql_store_result only - MYSQL_ROWS *data_cursor - pointing to the field of above "data" - MEM_ROOT field_alloc - memory allocation for above "fields" - MYSQL_ROW row - used for storing row by row in unbuffered - reads, i.e. in mysql_use_result - MYSQL_ROW current_row - cursor to the current row for buffered - reads - unsigned long *lengths - column lengths of current row - MYSQL *handle - see above, used in unbuffered reads, i.e. in - mysql_use_resultq - my_bool eof - used my mysql_fetch_row as a marker for end of data - - - - - Global variables - - - unsigned long max_allowed_packet - maximum allowable value of - network buffer. Default - value - 1 Mb - - unsigned long net_buffer_length - default, starting value of - network buffer - 8 Kb - - unsigned long bytes_sent - total number of bytes written since - startup of the server - - unsigned long bytes_received - total number of bytes read - since startup of the server - - - Synopsis of the basic client - server protocol - ---------------------------------------------- - - Purpose of this chapter is to provide a complete picture of -the basic client - server protocol implemented in MySQL. It was felt -it is necessary after writting descriptions for all of the functions -involved in basic protocol. There are at present 11 functions -involved, with several structures, many constants etc, which are all -described in detail. But as a forest could not be seen from the trees, -so a concept of the protocol could not be deciphered easily from a -thourough documentation on minutae. - - Althouch concept of the protocol was not changed with the -introduction of vio system, embodied in violate.cc source file and VIO -system, the introduction of these has changed a code substantially. Before -VIO was introduced, functions for reading from / writing to network -connection had to deal with various network standards. So, these functions -depended on whether TCP port or Unix socket file or NT Named Pipe file is -used. This is all changed now and single vio_ functions are called, while -all this diversity is covered by vio_ functions. - - In MySQL a specific buffered network input / output transport -model has been implemented. Although each operating system mah have -it's own buffering for network connections, MySQL has added it's own -buffering model. This same for each of the three transport protocol -types that are used in MySQL client - server communications, which are -tcp sockets (on all systems), Unix socket files on Unix and Unix-like -operating systems and Named Pipe files on NT. Alghouth tcp sockets -are omnipresent, the later two types have been added for local -connections. Those two connection types can be used in local modes -only, i.e. when both client and server reside on the same host, and -are introduced because they enable better speeds for local -connections. This is especially usefull for WWW type of -applications. Startup options of MySQl server allow that either tcp -sockets or local connection (OS dependent) can be disallowed. - - In order to be able to implement buffered input / output MySQL -allocates a buffer. A starting size of this buffer is determined by a -value of the global variable net_buffer_length, which can be changed -at MySQL server startup. This is, as explained only a startup length -of MySQL network buffer. As a signle item that has to be read / -written can be larger then that value, MySQL will increase buffer size -as long as that size reaches value of global variable -max_aallowed_packet, which is also settable at server startup. Maximum -value of this variable is limited by the way MySQL stores / reads -sizes of packets to be sent / read, which means by the way MySQL -formats packages. - - Basically each packet consists of two parts, a header and -data. In the case when compression is not used, header consists of 4 -bytes of which 3 contain a length of the packet to be sent and one -holds a packet number. When compression is used there are -onother 3 bytes which store a size of uncompressed data. Because of -the way MySQL packs length into 3 bytes, plus due to the usage of some -special values in the most significant byte, maximum size of -max_allowed_packet is limited to 24 Mb at present. So, if compression -is not used, at first 4 bytes are written to the buffer and then data -itself. As MySQL buffers I/O logical packets are packet together until -packets fill up entire size of the buffer. That size no less then -net_buffer_size, but no greater then max_allowed_packet. So, actuall -writting to the network is done when this buffer is filled up. As -frequently sequence of buffers make a logicall unit, like a result -set, then at the end of sending data, even if buffer is not full, data -is written, i.e. flushed to the connection with a call of the -net_flush function. In order to maintain that no single packet can be -larger then this value, checks are made throughout a code, so that not -signle field or command could exceed that value. - - In order to maintain coherency in consicutive packets, each -packet is numbered and their number stored as a part of a header, as -explained above. Packets start with 0, so whenever a logical packet is -written, that number is incremented. On the other side when packets -are read, value that is fetched is compared with a value stored and if -there is no mismatch that value is incremented too. Packet number is -reset on the client side when unwanted connections are removed from -the connection and on the server side when a new command hsa been -started. - - - So, before writting, a buffer contains a sequence of logical -packets, consisting of header plus data consequtively. In the case -that compression is used, packet numbers are not stored in each header -of the logical packets, but a whole buffer, or a part of it if -flushing is done, containing one or more logical packets are -compressed. In that case a new larger header, is formed, and all -logical packets contained in the buffer are compressed together. This -way only one packet is formed which makes several logical packets, -which improves both speed and compression ratio. On the other side, -when this large compressed packet is read, it is furst uncompressed, -and then logical packets are sent, one by one, to the calling -functions. - - - All this functionality is described in detail in the following -chapter. It does not contain functions that form logical packets, or -that read and write to connections but also functions that are used -for initialisation, clearing of connections. There are functions at -higher level dealing with sending fields, rows, establishing -connections, sending commands, but those are not explained in the -following chapter. - - - Functions utilized in client - server protocol - ---------------------------------------------- - - First of all, functions are described that are involved in -praparing/ reading / writing data over TCP port , socket or named pipe -file, and functions directly related to those. All of these functions -are used both in server and client. Server and client specific code -segments will be documented in each function description. Each MySQl -function checks for errors in memory allocation / freeing, as well as -in every OS call, like the one dealing with files and sockets, and for -errors in indeginous MySQL function calls. This is expected, but has -to be said, as not to repeat it in every function description. - - Older versions of MySQL have utilized the following macros for -reading from / writing to socket. - - raw_net_read - calls OS function recv function that reads N - bytes from a socket into a buffer. Number of - bytes read is returned. - - raw_net_write - calls OS funciton send to write N bytes from - a buffer to socket. Number of bytes written - is returned. - - These macros are replaced with VIO (Virtual I/O) functions. - - - Function name: my_net_init - - Parameters: struct NET *, enum_net_type, struct Vio - - Return value : 1 for error, 0 for success - - Function purpose: To initialize properly all NET fields, - allocate memory and set socket options - - Function description - - First of all, buff field of NET struct is allocated to the -size of net_buffer_lenght, and on failure function exits with 0. All -fields in NET are set to their default or starting values. As -net_buffer_length and max_allowed_packet are configurable, -max_allowed_packet is set equal to net_buffer_length it the later one -is greater. max_packet if set for that NET to net_buffer_lenght, and -buff_end points to buff end. vio feild is set to the second parameter. -If it is a real connection, which is a case when second parameter is -not null, then fd field is set by calling vio_fd function.read_pos and -write_pos to buff, while remaining integers are set to 0. If function -is run on the MySQL server on Unix and server is started in a test -mode that would require testing of blocking, then vio_blocking -function is called. Last, fast throughput mode is set by a call to -vio_fastsend function. - - - - Function name: net_end - - Parameters: struct NET * - - Return value : void - - Function purpose: To release memory alloceted to buff - - - - Function name: net_realloc (private, static function) - - Parameters: struct NET, unlong (unsigned long) - - Return value : 1 for error, 0 for success - - Function purpose: To change memory allocated to buff - - Function description - - New length of buff field of NET struct is passed as second -parameter. It is first checked versus max_allowd_packet and if greater -error is returned. New length is aligned to 4096 boundary. Then , buff -is reallocated, buff_end, max_packet and write_pas reset to the same -values as in my_net_init. - - - - Function name: net_clear (used on client side only) - - Parameters: struct NET * - - Return value : void - - Function purpose: To read unread packets - - Function description - - This function is used on client side only, and is executed -only if a program is not started in test mode. This function reads -unread packets without processing them. First, non-blocking mode is -set on systems that have not non-blocking mode defined. This is -performed by checking a mode with vio_is_blocking function. and -setting non-blocking mode by vio_blocking function. If this operation -was successfull, then packets are read by vio_read function, to which -vio field of NET is passed together with buff and max_packet field -values. field of the same struct at a length of max_packet. If -blocking was active before reading is performed, blocking is set with -vio_blocking function. AFter reading has been performed, pkt_nr is -reset to 0 and write_pos reset to buff. In order to clarify some -matters non-blocking mode enables executing program to dissociate from -a connection, so that error in connection would not hang entire -program or it's thread. - - Function name: net_flush - - Parameters: struct NET * - - Return value : 1 for error, 0 for success - - Function purpose: To write remaining bytes in buff to socket - - Function description - - net_real_write (described below) is performed is write_pos -differs from buff, both being fields of the only parameter. write_pos -is reset to buff. This function has to be used, as MySQL uses buffered -writes, as it will be more explained in a function net_write_buff. - - - Function name: my_net_write - - Parameters: struct NET *, const char *, ulong - - Return value : 1 for error, 0 for success - - Function purpose: Write a logical packet in a first parameter - of second parameter length - - Function description - - The purpose of this function is to prepare a logical packet -such that entire content of data, pointed to by second parametar and -in length of third parameter is sent to the other side. In case of -server, it is used for sending result sets, and in case of client it -is used for sending local data. This function foremost prepares a -header for the packet. Normal, header consists of 4 bytes, of which -first 3 bytes contain a length of the packet, thereby limiting a -maximum allowable length of a packet to 16 Mb, while a fourth byte -contains a packet number, which is used when one large packet has to -be divided into sequence of packets. This way each sub-packet gets -it's number which should be matched on the other side. When -compression is used another three bytes are added to packet header, -thus packet header is in that case increased to 7 bytes. Additional -three bytes are used to save a length of compressed data. As in -connection that uses compression option, code packs packets together,, -a header prepared by this function is later not used in writting to / -reading from network, but only to distinguish logical packets within a -buffered read operation. - - - This function, first stores a value for third parameter into a -first 3 bytes of local char variable of NET_HEADER_SIZE size by usage -of function int3store. Then, at this point, if compression is not -used, pkt_nr is increased, and it's value stored in the last byte of -the said local char[] variable. If compression is used 0 is stored in -both values. Then those four bytes are sent to other side by the usage -of the function net_write_buff(to be explained later on), and -successfull, entire packet in second parameter of the length described -in third parameter is sent by the usage of the same function. - - - Function name: net_write_command - - Parameters: struct NET *, char, const char *, ulong - - Return value : 1 for error, 0 for success - - Function purpose: Send a command with a packet as in previous - function - - Function description - - This function is very similar to the previous one. The only -difference is that first packet is enlarged by one byte, so that a -command precedes a packet to be sent. This is implemented by -increasing fist packet by one byte, which contains a command code. As -command codes do not use a range of values that are used by chararcter -sets, so when the other side receives a packet, first byte after -header contains a command code. This function is used by client for -sending all commands and queries, and by server in connection process -and for sending errors. - - - Function name: net_write_buff (private, static function) - - Parameters: struct NET *, const char *, uint - - Return value : 1 for error, 0 for success - - Function purpose: To write a packet of vany size by cutting it - and using next function for writing it - - Function description - - This function was created after compression feature has been -added to MySQL. This function supposes that packets have already been -properly formatted, regarding packet header etc. Principal reason for -this function existst because a packet that is sent by client or -server does not have to be less then max_packet. So this function -first calculeates how much data has been left in a buff, by getting a -difference between buff_end and write_pos and storing it to local -variable left_length. Then a loop is run as long as a length to be -sent is greater then length of left bytes (left_length). In a loop -data from second parameter is copied to buff at write_pos, as much as -it can be, i.e. by left_length. Then net_real_write function is called -(see below) with NET, buff, and max_packet parameters. This function -is the lowest level fucntion that writes data over established -connection. In a loop, write_pos is reset to buff, a pointer to data -(second parameter) is moved by teh amount of data sent (left_length), -length of data to be sent (third parameter) is decreased by the amount -sent (left_length) and left_length is reset to max_packet value, which -ends a loop. This logic was necessary, as there could have been some -data yet unsent (write_pos != buf), while data to be sent could be as -large as necessary, thus requiring many loops. At the end of function, -remaining data in second parameter are copied to buff at write_pos, by -the remaining length of data to be sent (third parameter). So, in the -next call of this function remaining data will be sent, as buff is -used in a call to net_real_write. It is very important to note that if -a packet to be sent is less then a number of bytes that are still -available in buff, then there will be no writing over network, but -only logical packets will be added one after another. This will -accelerate network traffic, plus if a compression is used, the -expected compression rate would be higher. That is why server or -client functions that sends data uses at the end of data net_flush -function described above. - - - Function name: net_real_write - - Parameters: struct NET *, const char *, ulong - - Return value : 1 for error, 0 for success - - Function purpose: To write data to a socket or pipe, with - compression if used - - Function description - - First, more field is set to 2, to enable reporting in -mysql_list_processes. Then if compression is enabled on that -connection, a new local buffer (variable b) is initialized to the -length of total header (normal header + compression header) and if no -memory is available error is returned. This buffer (b) is used for -holding a fineal, compressed packet to be written ove -connection. Furthermore in compressiion initialization, Second -parameter at length of third parameter is copied to the local buffer -b, and MySQL's wrapped zlib's compression function is run at total -header offset of the local buffer. Please, do note that this function -does not test effectiveness of compression. If compression is turned -on in some connection, it is used all of the time. Also, it is very -important to be cognizant of the fact that this algorithm makes -possible that a single compressed packet contains several logical -packets. In this way compression rate is increased and network -throughput is increased as well. However, this algorithm has -consequences on the other sided, that reads compressed packet, which -is covered in my_net_read function. After compression is done, a full -compression header is properly formed with a packet number, -compressed and uncompressed lengths. At the end of compression code, -third parameter is increased by total header length, as the original -header is not used (see above), and second parameter, pointer to data, -is set to point to local buffer b, in order that a further flow of -function is independent of compression. . If a function is executed -on server side, a thread alarm initialized and if non-blocking is -active set at NET_WRITE_TIMEOUT. Two local (char *) pointers are -initialized, pos at beginning of second parameter, and end at end of -data. Then a loop is run as long as all data is written, which means -as long as pos != end. First vio_write function is called, with -parameters of vio field, pos and size of data (end - pos). Number of -bytes written over connection is saved in local variable (length). If -error is returned local bool variable (interrupted) is set according -to the return value of the vio_should_retry called with vio field as -parameter. This bool variable indicates whether writing was -interrupted in some way or not. Firther, error from vio_write is -differently treated on Unix versus other OS's (Win32 or OS/2). On Unix -an alarm is set if one is not in use, no bytes have been written and -there has been no interruption. Also, in that case, if connection is -not in blocking mode, a sub - loop is run as long as blocking is not -set with vio_blocking function. Withing a loop another run of above -vio_write is run based on return value of vio_is_retry function, --provided number of repeated writes is less then RETRY_COUNT. If that -is not a case, error field of struct NET. is set to 1 and function -exits. At the exit of sub-llop number of reruns already executed is -reset to zero and another run of above vio_write function is -attempted. If a function is run on Win32 and OS/2, and in the case -that function flow was not interrupted and thread alarm is not in use, -again a main loop is continued until pos != end. In the case that this -function is executed on thread safe client program, a communication -flow is tested on EINTR, caused by context switching, by use of -vio_errno function, in which case a loop is continued. At the end of -processing of the error from vio_write, error field of struct NET is -set, and if on server last_errno field is set to -ER_NET_WRITE_INTERRUPTED in the case thatb local bool variable -(interrupted) is true or to ER_NET_ERROR_ON_WRITE. Before the end of -loop, in order to make possible evaluation of the loop condition, pos -is increased by a value writen in last iteration (length). Also global -variable bytes_sent is increased by the same value, for status -purposes. At the end of the functions more fields is reset, in case -of compression, combression buffer (b) memory is released and if -thread is still in use, it is ended and blocking state is reset to -it's original state, and function returns error is all bytes are not -written. - - - - Function name: my_real_read (private, static function) - - Parameters: struct NET *, ulong * - - Return value : length of bytes read - - Function purpose: low level network connection read function - - Function description - - This function has made as a separate one when compression was -introduced in MySQL client - server protocol . It contains basic, low -level network reading functionality, while all dealings with -compressed packets are handled in next function. Compression in this -function is only handled in as much to unfold a length of uncompressed -data. First blocking state of connection is saved in local bool -variable net_blocking, and field more is set 1 for deteiled reporting -in mysqld_list_processes. A new thread alarm is initialized, in order -to enable read timout handling, and if on server and a connection can -block a program, the alarm is set at a value of timeout field. Local -pointer is set to the position of the next logical packet, with it's -header skipped, which is at field where_b offset from buff. Next, a -two time run code is entered. A loop is run exactly two times becase -first time number of bytes to be fetched (remain) are set to the -header size, which is different when compression is used or not used -on the connection. After first fetch has been done, number of packets -that will be received in second iteration is well known, as fetched -header contains a size of packet, packet number ,and in a case of -compression a size of uncompressed packet. Then as long, as there are -bytes to read a loop is entered with ffirst reading data from network -connection with vio_read function, called with parameters of field -vio, current position and remaining number of bytes, which value is -hold by local variable (remain) initialized at a value of header size, -which differs if compression is used. Number of bytes read are -returned in local length variable. If error is returned local bool -variable (interrupted) is set according to the return value of the -vio_should_retry called with vio field as parameter. This bool -variable indicates whether reading was interrupted in some way or not. -Firther, error from vio_read is differently treated on Unix versus -other OS's (Win32 or OS/2). On Unix an alarm is set if one is not in -use, no bytes have been read and there has been no interruption. Also, -in that case, if connection is not in blocking mode, a sub - loop is -run as long as blocking is not set with vio_blocking function. Withing -a loop another run of above vio_read is run based on return value of -vio_is_retry function, -provided number of repeated writes is less -then RETRY_COUNT. If that is not a case, error field of struct NET. is -set to 1 and function exits. At the exit of sub-llop number of reruns -already executed is reset to zero and another run of above vio_read -function is attempted. If a function is run on Win32 and OS/2, and in -the case that function flow was not interrupted and thread alarm is -not in use, again a main loop is continued as long as there are bytes -remaining. In the case that this function is executed on thread safe -client program, then if a another run should be made, which is decided -by the output of vio_should_retry function, in which case a loop is -continued. At the end of processing of the error from vio_read, error -field of struct NET is set, and if on server last_errno field is set -to ER_NET_READ_INTERRUPTED in the case thatb local bool variable -(interrupted) is true or to ER_NET_ERROR_ON_READ. In case of such an -error this function exits and returns error. In a case when there is -no error, number of remaining bytes (remain) is decreased by a number -of bytes read, which should be zero, but in case it is not the entire -code is still in while (remain > 0) loop, which will be exited -immediately if it is. This has been done to accomodate errors in the -traffic level and for the very slow connections. Current position in -field buff is also moved by the amount of bytes read by vio_read -funciton, and global variable bytes_received is increased by the same -value in a thread safe manner. When a loop that is run until necessary -bytes are read (remain) is finished, then if external loop is in it's -first run, of the two, packet sequencing is tested on consistency by -comparing a number contained at 4th byte in header with pkt_nre -field. Header location is found at where_b offset to field_b. Usage of -where_b is obligatory due to the possible compression usage. If there -is no compression on a connection, then where_b is always 0. If there -is a discrepancy, then first byte of the header is checked whether it -is equal to 255, because when error is sent by a server, or by a -client if it is sending data (like in LOAD DATA INFILE LOCAL...), then -first byte in header is set to 255. If it is not 255, then an error on -packets being out of order is printed. In any case, on server, -last_errno field is set to ER_NET_PACKETS_OUT_OF_ORDER and a function -returns with the error, i.e. value returned is packet_error. If a -check on serial number of packet is successful, pkt_nr field is -incremented in order to enable checking packet order with next packet -and if compression is used, uncompressed length is extracted from a -proper position in header and returned in the second parameter of this -function. Length of the packet is saved, for the purpose of a proper -return value from this function. Still in the first iteration of the -main loop, a check must be made if field buff could accomodate entire -package that comes, in it's compressed or uncompressed form. This is -done in such a way, because zlib's compress and uncompress functions -use a same memory area for compression / uncompression. Necessary -field buff length is equal to current offset where data are (where_b -which is zero for non-compression), plus the larger value of -compressed or uncompressed package to be read in a second run. If this -value is larger then a current length of field buff, which is read -from field max_packet, then feild buff has to be reallocated. IF -reallocation with net_realloc function fails, function is returned -with error. Before a second loop is started, length to be read is set -to the length of expected data and current position (pos) is set at -where_b offset from field buff. At the end of function, if alarm is -set, which is a case if it is run on server or on a client if a -function is interrupted and another run of vio_read is attempted, -alarm is ended and blocking state is resotred from the saved local bool -variable net_blocking. Function returns number of bytes read or the -error (packet_error). - - - - - Function name: my_net_read - - Parameters: struct NET * - - Return value : length of bytes read - - Function purpose: Highest level general purpose reading function - - Function description - - First, if compression is not used, my_real_read is called, -with struct NET * a first parameter, and pointer to local ulong -complen as a second parameter, but it's value is not used here. -Number of bytes read is returned in local ulong variable len. read_pos -field is set to an offset of value of where_b field from field -buff. where_b field actually denotes where in field buff is a current -packet. If returned number of bytes read (local variable len) does not -signal that an error in packet trnasmission occured, i.e. it is not -set to packet_error, then a string contained in read_pos is zero -terminated. Simply, the end of a string starting at read_pos, and -ending at read_pos + len, is set to zero. This is done in that way, -because mysql_use_result expects a zero terminated string, and -function returns with a value local variable len. This ends this -function in the case that compression is not used and a remaining code -is executed only if compression is enabled on the connection. In -order to explain how is compressed packet logically cut into -meningfull packets, a full meaning of several NET feilds should be -explained. First of all, fields in NET are used and not local -variables, as all values should be saved between consecutive calls of -this function. Simply, this function is called in order to return -logical packets, but this function does not need to call my_real_read -function everytime, because when a large packet is uncompressed, it -may, but not necessarily so, contain several logical -packets. Therefore, in oreder to preserve data on logical packets -local variables are not used. Instead fields in NET struct are -used. Field remain_in_buf denotes how many bytes of entire -uncompressed packets is still contained withing buff. field buf_length -saves a value of the length of entire uncompressed packet. field -save_char is used to save a character at a position where a packet -ends, which character has to be replaced with a zero, '\0', in order -to make a logical packet zero delimited, for mysql_use_result. Field -length stores a value of the length of compressed packet. Field -read_pos as usual, points to the current reading position. This char * -pointer is used by all fucntion calling this function in order to -fetch their data. Field buff is not used for that purpose, but -read_pos is used instead. This change was introduced with compression, -when algorihtm accomodated grouping of several packets together. Now, -that meanings of all relevant NET fields is complained, we can proceed -with a flow of this functinn in case when compression is -active. First, if there are remaining portions of compressed packet in -a field buff, saved character value is set at a position where zero -char '\0' was inserted to enable a string do be zero delimited for -mysql_use_result. Then a loop is started. In the first part of the -loop, if there are remaining bytes, local uchar *pos variable is set -at a current position in field buff where a new packet starts. This -position is an (buf_length - remain_in_buf) offset in field buff. As -it is possible that next logical packet is not read to the full length -in the remaining of the field buf, several things had to be -inspected. It should be noted that data that is read from -net_rweal_read contains only logical packets containing 4 byte headers -only, being 4 byte headers prepared by my_net_write or -net_write_command. But, when written, logical packet could be so -divided that only a part of header is read in. Therefore after pointer -to the start of the next packet has been saved, a check is made -whether number of remaining bytes in buffer is less then 4, being 3 -bytes for length and one byte for packet number. If it is greater, -then a length of the logical packet is extracted and saved a length -field. Then a check is made whether entire packet is contained withing -a buf, i.e. a check is made that a logical packet is fully contained -in a buffer. In that case, number of bytes remaining in buffer is -decreased by a full length of logical packet ( 4 + length field), -read_pos is moved forward by 4 bytes to skip header and be set at a -beginning of data in logical packet, length field is saved for a value -to be returned in function and a loop is exited. In a case that -entire logical packet is not contained within a buffer, then if length -of the entire buffer differs from remaining length of logical packet, -it (logical packet) is moved to the beginning of the field buff. If -length of the entire buffer equals the remaining length of logical -packet, where_b and buf_length fields are set to 0. This is done so -that in both cases buffer is ready to accept next part of packet. In -order to get a next part of a packet, still within a loop, -my_real_read function is called and length of compressed packet is -returned to a local len variable, and length of compressed data is -returned in complen variable. In a case of non-compression value of -complen is zero. If packet_error is from my_real_read function, this -function returns also with packet_error. If it is not a packet_error -my_uncompress function is called to uncompress data. It is called with -offset of where_b data from field buff, as it is a postion where -compressed packet starts, and with len and complen values, being -lengths of compressed and uncompressed data. If there is no -compression, 0 is returned for uncompressed size from my_real_read -function, and my_uncompress wrapper function is made to skip zlib -uncompress in that case. If error is returned fom my_uncompress, -error field is set to 1, if on server last_errno is set to -ER_NET_UNCOMPRESS_ERROR and loop is exited and function returns with -packet_error. If not, buf_length and reamin_in_buf fields are set to -the uncompressed size of buffer and a loop is continued. When a loop -is exited save_char field is used to save a char at end of a logical -packet, which is an offset of field len from position in field buff -pointed by field read_os, in order that zero char is set at the same -position, for mysql_use_result. Function returns a length of the -logical packet without it's header. |