diff options
42 files changed, 420 insertions, 264 deletions
diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index 99aae0d22f..8a35a408b7 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -44,8 +44,8 @@ The module defines the following items: the method call ``login(user, passwd, acct)`` is made (where *passwd* and *acct* default to the empty string when not given). The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the - connection attempt (if is not specified, or passed as None, the global - default timeout setting will be used). + connection attempt (if is not specified, the global default timeout setting + will be used). .. attribute:: all_errors @@ -123,10 +123,8 @@ followed by ``lines`` for the text version or ``binary`` for the binary version. made. The optional *timeout* parameter specifies a timeout in seconds for the - connection attempt. If is not specified, or passed as None, the object - timeout is used (the timeout that you passed when instantiating the class); - if the object timeout is also None, the global default timeout setting will - be used. + connection attempt. If no *timeout* is passed, the global default timeout + setting will be used. .. method:: FTP.getwelcome() diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst index 6903b6a903..9c895633c9 100644 --- a/Doc/library/http.client.rst +++ b/Doc/library/http.client.rst @@ -33,7 +33,7 @@ The module provides the following classes: be raised if the status line can't be parsed as a valid HTTP/1.0 or 1.1 status line. If the optional *timeout* parameter is given, blocking operations (like connection attempts) will timeout after that many seconds - (if it is not given or ``None``, the global default timeout setting is used). + (if it is not given, the global default timeout setting is used). For example, the following calls all create instances that connect to the server at the same host and port:: diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index 893296fade..0b2c033af5 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -29,8 +29,8 @@ A single class is provided by the :mod:`poplib` module: This class implements the actual POP3 protocol. The connection is created when the instance is initialized. If *port* is omitted, the standard POP3 port (110) is used. The optional *timeout* parameter specifies a timeout in seconds for the - connection attempt (if not specified, or passed as None, the global default - timeout setting will be used). + connection attempt (if not specified, the global default timeout setting will + be used). .. class:: POP3_SSL(host[, port[, keyfile[, certfile]]]) diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index 602edb6a3f..31f1ad6e11 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -25,8 +25,8 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). with those parameters during initialization. An :exc:`SMTPConnectError` is raised if the specified host doesn't respond correctly. The optional *timeout* parameter specifies a timeout in seconds for blocking operations - like the connection attempt (if not specified, or passed as None, the global - default timeout setting will be used). + like the connection attempt (if not specified, the global default timeout + setting will be used). For normal use, you should only require the initialization/connect, :meth:`sendmail`, and :meth:`quit` methods. An example is included below. @@ -42,8 +42,8 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). and *certfile* are also optional, and can contain a PEM formatted private key and certificate chain file for the SSL connection. The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the - connection attempt (if not specified, or passed as None, the global default - timeout setting will be used). + connection attempt (if not specified, the global default timeout setting + will be used). .. class:: LMTP([host[, port[, local_hostname]]]) diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst index 8fefce6942..15e66c70a4 100644 --- a/Doc/library/socket.rst +++ b/Doc/library/socket.rst @@ -197,12 +197,11 @@ The module :mod:`socket` exports the following constants and functions: .. function:: create_connection(address[, timeout]) - Connects to the *address* received (as usual, a ``(host, port)`` pair), with an - optional timeout for the connection. Especially useful for higher-level - protocols, it is not normally used directly from application-level code. - Passing the optional *timeout* parameter will set the timeout on the socket - instance (if it is not given or ``None``, the global default timeout setting is - used). + Convenience function. Connect to *address* (a 2-tuple ``(host, port)``), + and return the socket object. Passing the optional *timeout* parameter will + set the timeout on the socket instance before attempting to connect. If no + *timeout* is supplied, the global default timeout setting returned by + :func:`getdefaulttimeout` is used. .. function:: getaddrinfo(host, port[, family[, socktype[, proto[, flags]]]]) diff --git a/Doc/library/telnetlib.rst b/Doc/library/telnetlib.rst index 8c8ddbb6ef..4fd88962cd 100644 --- a/Doc/library/telnetlib.rst +++ b/Doc/library/telnetlib.rst @@ -28,6 +28,11 @@ Character), EL (Erase Line), GA (Go Ahead), SB (Subnegotiation Begin). :class:`Telnet` represents a connection to a Telnet server. The instance is initially not connected by default; the :meth:`open` method must be used to establish a connection. Alternatively, the host name and optional port + and timeout can be passed to the constructor, in which case the connection to + the server will be established before the constructor returns. The optional + *timeout* parameter specifies a timeout in seconds for the connection attempt (if + not specified, the global default timeout setting will be used). + number can be passed to the constructor, to, in which case the connection to the server will be established before the constructor returns. The optional *timeout* parameter specifies a timeout in seconds for blocking operations @@ -123,8 +128,7 @@ Telnet Objects Connect to a host. The optional second argument is the port number, which defaults to the standard Telnet port (23). The optional *timeout* parameter specifies a timeout in seconds for blocking operations like the connection - attempt (if not specified, or passed as None, the global default timeout - setting will be used). + attempt (if not specified, the global default timeout setting will be used). Do not try to reopen an already connected instance. diff --git a/Doc/library/urllib2.rst b/Doc/library/urllib2.rst index 0462222e86..06dbb445a3 100644 --- a/Doc/library/urllib2.rst +++ b/Doc/library/urllib2.rst @@ -27,9 +27,9 @@ The :mod:`urllib2` module defines the following functions: returns a string in this format. The optional *timeout* parameter specifies a timeout in seconds for blocking - operations like the connection attempt (if not specified, or passed as - ``None``, the global default timeout setting will be used). This actually - only works for HTTP, HTTPS, FTP and FTPS connections. + operations like the connection attempt (if not specified, the global default + timeout setting will be used). This actually only works for HTTP, HTTPS, + FTP and FTPS connections. This function returns a file-like object with two additional methods: @@ -404,9 +404,9 @@ OpenerDirector Objects the same as those of :func:`urlopen` (which simply calls the :meth:`open` method on the currently installed global :class:`OpenerDirector`). The optional *timeout* parameter specifies a timeout in seconds for blocking - operations like the connection attempt (if not specified, or passed as - ``None``, the global default timeout setting will be used; this actually only - works for HTTP, HTTPS, FTP and FTPS connections). + operations like the connection attempt (if not specified, the global default + timeout setting will be usedi). The timeout feature actually works only for + HTTP, HTTPS, FTP and FTPS connections). .. method:: OpenerDirector.error(proto[, arg[, ...]]) diff --git a/Include/pyport.h b/Include/pyport.h index 0727d1073b..0095cc4649 100644 --- a/Include/pyport.h +++ b/Include/pyport.h @@ -431,6 +431,13 @@ extern int gethostname(char *, int); extern char * _getpty(int *, int, mode_t, int); #endif +/* On QNX 6, struct termio must be declared by including sys/termio.h + if TCGETA, TCSETA, TCSETAW, or TCSETAF are used. sys/termio.h must + be included before termios.h or it will generate an error. */ +#ifdef HAVE_SYS_TERMIO_H +#include <sys/termio.h> +#endif + #if defined(HAVE_OPENPTY) || defined(HAVE_FORKPTY) #if !defined(HAVE_PTY_H) && !defined(HAVE_LIBUTIL_H) /* BSDI does not supply a prototype for the 'openpty' and 'forkpty' diff --git a/Lib/ftplib.py b/Lib/ftplib.py index eec20f581d..4955727147 100644 --- a/Lib/ftplib.py +++ b/Lib/ftplib.py @@ -44,6 +44,7 @@ try: from socket import getfqdn; socket.getfqdn = getfqdn; del getfqdn except ImportError: import socket +from socket import _GLOBAL_DEFAULT_TIMEOUT __all__ = ["FTP","Netrc"] @@ -71,7 +72,6 @@ all_errors = (Error, IOError, EOFError) # Line terminators (we always output CRLF, but accept any of CRLF, CR, LF) CRLF = '\r\n' - # The class itself class FTP: @@ -110,14 +110,15 @@ class FTP: # Initialize host to localhost, port to standard ftp port # Optional arguments are host (for connect()), # and user, passwd, acct (for login()) - def __init__(self, host='', user='', passwd='', acct='', timeout=None): + def __init__(self, host='', user='', passwd='', acct='', + timeout=_GLOBAL_DEFAULT_TIMEOUT): self.timeout = timeout if host: self.connect(host) if user: self.login(user, passwd, acct) - def connect(self, host='', port=0, timeout=None): + def connect(self, host='', port=0, timeout=-999): '''Connect to host. Arguments are: - host: hostname to connect to (string, default previous host) - port: port to connect to (integer, default previous port) @@ -126,7 +127,7 @@ class FTP: self.host = host if port > 0: self.port = port - if timeout is not None: + if timeout != -999: self.timeout = timeout self.sock = socket.create_connection((self.host, self.port), self.timeout) self.af = self.sock.family diff --git a/Lib/heapq.py b/Lib/heapq.py index 71a12e7bf4..380fe1268b 100644 --- a/Lib/heapq.py +++ b/Lib/heapq.py @@ -167,7 +167,7 @@ def heapreplace(heap, item): def heappushpop(heap, item): """Fast version of a heappush followed by a heappop.""" - if heap and item > heap[0]: + if heap and heap[0] < item: item, heap[0] = heap[0], item _siftup(heap, 0) return item @@ -240,10 +240,11 @@ def _siftdown(heap, startpos, pos): while pos > startpos: parentpos = (pos - 1) >> 1 parent = heap[parentpos] - if parent <= newitem: - break - heap[pos] = parent - pos = parentpos + if newitem < parent: + heap[pos] = parent + pos = parentpos + continue + break heap[pos] = newitem # The child indices of heap index pos are already heaps, and we want to make @@ -294,7 +295,7 @@ def _siftup(heap, pos): while childpos < endpos: # Set childpos to index of smaller child. rightpos = childpos + 1 - if rightpos < endpos and heap[rightpos] <= heap[childpos]: + if rightpos < endpos and not heap[childpos] < heap[rightpos]: childpos = rightpos # Move the smaller child up. heap[pos] = heap[childpos] diff --git a/Lib/http/client.py b/Lib/http/client.py index de27c17b4c..c6e40e1cbb 100644 --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -664,7 +664,8 @@ class HTTPConnection: debuglevel = 0 strict = 0 - def __init__(self, host, port=None, strict=None, timeout=None): + def __init__(self, host, port=None, strict=None, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): self.timeout = timeout self.sock = None self._buffer = [] @@ -996,7 +997,7 @@ else: default_port = HTTPS_PORT def __init__(self, host, port=None, key_file=None, cert_file=None, - strict=None, timeout=None): + strict=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): HTTPConnection.__init__(self, host, port, strict, timeout) self.key_file = key_file self.cert_file = cert_file diff --git a/Lib/poplib.py b/Lib/poplib.py index aa4600c4a8..bd82841ef1 100644 --- a/Lib/poplib.py +++ b/Lib/poplib.py @@ -76,7 +76,8 @@ class POP3: """ - def __init__(self, host, port=POP3_PORT, timeout=None): + def __init__(self, host, port=POP3_PORT, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): self.host = host self.port = port self.sock = socket.create_connection((host, port), timeout) diff --git a/Lib/smtplib.py b/Lib/smtplib.py index a3cc65d295..e67f0d6520 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -220,7 +220,8 @@ class SMTP: ehlo_resp = None does_esmtp = 0 - def __init__(self, host='', port=0, local_hostname=None, timeout=None): + def __init__(self, host='', port=0, local_hostname=None, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): """Initialize a new instance. If specified, `host' is the name of the remote host to which to @@ -744,7 +745,8 @@ if _have_ssl: certificate chain file for the SSL connection. """ def __init__(self, host='', port=0, local_hostname=None, - keyfile=None, certfile=None, timeout=None): + keyfile=None, certfile=None, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): self.keyfile = keyfile self.certfile = certfile SMTP.__init__(self, host, port, local_hostname, timeout) diff --git a/Lib/socket.py b/Lib/socket.py index eb876731e3..fcbc545d5a 100644 --- a/Lib/socket.py +++ b/Lib/socket.py @@ -265,13 +265,17 @@ def getfqdn(name=''): return name -def create_connection(address, timeout=None): - """Connect to address (host, port) with an optional timeout. +_GLOBAL_DEFAULT_TIMEOUT = object() - Provides access to socketobject timeout for higher-level - protocols. Passing a timeout will set the timeout on the - socket instance (if not present, or passed as None, the - default global timeout setting will be used). +def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT): + """Connect to *address* and return the socket object. + + Convenience function. Connect to *address* (a 2-tuple ``(host, + port)``) and return the socket object. Passing the optional + *timeout* parameter will set the timeout on the socket instance + before attempting to connect. If no *timeout* is supplied, the + global default timeout setting returned by :func:`getdefaulttimeout` + is used. """ msg = "getaddrinfo returns an empty list" @@ -281,7 +285,7 @@ def create_connection(address, timeout=None): sock = None try: sock = socket(af, socktype, proto) - if timeout is not None: + if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) sock.connect(sa) return sock diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py index 8327aa195b..d276a336c9 100644 --- a/Lib/sqlite3/test/dbapi.py +++ b/Lib/sqlite3/test/dbapi.py @@ -292,6 +292,15 @@ class CursorTests(unittest.TestCase): self.cu.execute("update test set name='bar'") self.failUnlessEqual(self.cu.rowcount, 2) + def CheckRowcountSelect(self): + """ + pysqlite does not know the rowcount of SELECT statements, because we + don't fetch all rows after executing the select statement. The rowcount + has thus to be -1. + """ + self.cu.execute("select 5 union select 6") + self.failUnlessEqual(self.cu.rowcount, -1) + def CheckRowcountExecutemany(self): self.cu.execute("delete from test") self.cu.executemany("insert into test(name) values (?)", [(1,), (2,), (3,)]) diff --git a/Lib/telnetlib.py b/Lib/telnetlib.py index 33029e1047..56c5d702b6 100644 --- a/Lib/telnetlib.py +++ b/Lib/telnetlib.py @@ -184,13 +184,13 @@ class Telnet: """ - def __init__(self, host=None, port=0, timeout=None): + def __init__(self, host=None, port=0, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): """Constructor. When called without arguments, create an unconnected instance. - With a hostname argument, it connects the instance; a port - number is optional. - + With a hostname argument, it connects the instance; port number + and timeout are optional. """ self.debuglevel = DEBUGLEVEL self.host = host @@ -208,23 +208,21 @@ class Telnet: if host is not None: self.open(host, port, timeout) - def open(self, host, port=0, timeout=None): + def open(self, host, port=0, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): """Connect to a host. The optional second argument is the port number, which defaults to the standard telnet port (23). Don't try to reopen an already connected instance. - """ self.eof = 0 if not port: port = TELNET_PORT self.host = host self.port = port - if timeout is not None: - self.timeout = timeout - self.sock = socket.create_connection((host, port), self.timeout) + self.timeout = timeout + self.sock = socket.create_connection((host, port), timeout) def __del__(self): """Destructor -- close the connection.""" diff --git a/Lib/test/test_datetime.py b/Lib/test/test_datetime.py index 2048fa0a42..4b8ae82df7 100644 --- a/Lib/test/test_datetime.py +++ b/Lib/test/test_datetime.py @@ -845,9 +845,13 @@ class TestDate(HarmlessMixedComparison, unittest.TestCase): self.assertRaises(TypeError, t.strftime, "one", "two") # too many args self.assertRaises(TypeError, t.strftime, 42) # arg wrong type + # test that unicode input is allowed (issue 2782) + self.assertEqual(t.strftime("%m"), "03") + # A naive object replaces %z and %Z w/ empty strings. self.assertEqual(t.strftime("'%z' '%Z'"), "'' ''") + def test_format(self): dt = self.theclass(2007, 9, 10) self.assertEqual(dt.__format__(''), str(dt)) diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py index b91588a310..35e4746a8c 100644 --- a/Lib/test/test_ftplib.py +++ b/Lib/test/test_ftplib.py @@ -54,35 +54,52 @@ class GeneralTests(TestCase): # connects ftp = ftplib.FTP(HOST) self.evt.wait() - ftp.sock.close() + ftp.close() def testTimeoutDefault(self): - # default - ftp = ftplib.FTP(HOST) + # default -- use global socket timeout + self.assert_(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + ftp = ftplib.FTP("localhost") + finally: + socket.setdefaulttimeout(None) + self.assertEqual(ftp.sock.gettimeout(), 30) + self.evt.wait() + ftp.close() + + def testTimeoutNone(self): + # no timeout -- do not use global socket timeout + self.assert_(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + ftp = ftplib.FTP("localhost", timeout=None) + finally: + socket.setdefaulttimeout(None) self.assertTrue(ftp.sock.gettimeout() is None) self.evt.wait() - ftp.sock.close() + ftp.close() def testTimeoutValue(self): # a value ftp = ftplib.FTP(HOST, timeout=30) self.assertEqual(ftp.sock.gettimeout(), 30) self.evt.wait() - ftp.sock.close() + ftp.close() def testTimeoutConnect(self): ftp = ftplib.FTP() ftp.connect(HOST, timeout=30) self.assertEqual(ftp.sock.gettimeout(), 30) self.evt.wait() - ftp.sock.close() + ftp.close() def testTimeoutDifferentOrder(self): ftp = ftplib.FTP(timeout=30) ftp.connect(HOST) self.assertEqual(ftp.sock.gettimeout(), 30) self.evt.wait() - ftp.sock.close() + ftp.close() def testTimeoutDirectAccess(self): ftp = ftplib.FTP() @@ -90,18 +107,6 @@ class GeneralTests(TestCase): ftp.connect(HOST) self.assertEqual(ftp.sock.gettimeout(), 30) self.evt.wait() - ftp.sock.close() - - def testTimeoutNone(self): - # None, having other default - previous = socket.getdefaulttimeout() - socket.setdefaulttimeout(30) - try: - ftp = ftplib.FTP(HOST, timeout=None) - finally: - socket.setdefaulttimeout(previous) - self.assertEqual(ftp.sock.gettimeout(), 30) - self.evt.wait() ftp.close() diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index ff4cf9bd71..054825a5a3 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -215,27 +215,32 @@ class TimeoutTest(TestCase): # This will prove that the timeout gets through HTTPConnection # and into the socket. - # default - httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT) - httpConn.connect() - self.assertTrue(httpConn.sock.gettimeout() is None) - httpConn.close() - - # a value - httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT, timeout=30) - httpConn.connect() + # default -- use global socket timeout + self.assert_(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT) + httpConn.connect() + finally: + socket.setdefaulttimeout(None) self.assertEqual(httpConn.sock.gettimeout(), 30) httpConn.close() - # None, having other default - previous = socket.getdefaulttimeout() + # no timeout -- do not use global socket default + self.assert_(socket.getdefaulttimeout() is None) socket.setdefaulttimeout(30) try: httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT, timeout=None) httpConn.connect() finally: - socket.setdefaulttimeout(previous) + socket.setdefaulttimeout(None) + self.assertEqual(httpConn.sock.gettimeout(), None) + httpConn.close() + + # a value + httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT, timeout=30) + httpConn.connect() self.assertEqual(httpConn.sock.gettimeout(), 30) httpConn.close() diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index ae29cda667..f4c27bb389 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -736,7 +736,7 @@ class MathTests(unittest.TestCase): OverflowError), ([2.**1023, 2.**1023, -1e307], OverflowError), ([1e16, 1., 1e-16], 10000000000000002.0), - ([1e16-2., 1.-2.**53, -(1e16-2.), -(1.-2.**53)], 0.0), + ([1e16-2., 1.-2.**-53, -(1e16-2.), -(1.-2.**-53)], 0.0), ] for i, (vals, s) in enumerate(test_values): diff --git a/Lib/test/test_poplib.py b/Lib/test/test_poplib.py index 14b46e3162..f9c52f911f 100644 --- a/Lib/test/test_poplib.py +++ b/Lib/test/test_poplib.py @@ -40,28 +40,29 @@ class GeneralTests(TestCase): pop.sock.close() def testTimeoutDefault(self): - # default - pop = poplib.POP3(HOST, self.port) - self.assertTrue(pop.sock.gettimeout() is None) - pop.sock.close() - - def testTimeoutValue(self): - # a value - pop = poplib.POP3(HOST, self.port, timeout=30) + self.assertTrue(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + pop = poplib.POP3("localhost", self.port) + finally: + socket.setdefaulttimeout(None) self.assertEqual(pop.sock.gettimeout(), 30) pop.sock.close() def testTimeoutNone(self): - # None, having other default - previous = socket.getdefaulttimeout() + self.assertTrue(socket.getdefaulttimeout() is None) socket.setdefaulttimeout(30) try: pop = poplib.POP3(HOST, self.port, timeout=None) finally: - socket.setdefaulttimeout(previous) - self.assertEqual(pop.sock.gettimeout(), 30) + socket.setdefaulttimeout(None) + self.assertTrue(pop.sock.gettimeout() is None) pop.sock.close() + def testTimeoutValue(self): + pop = poplib.POP3("localhost", self.port, timeout=30) + self.assertEqual(pop.sock.gettimeout(), 30) + pop.sock.close() def test_main(verbose=None): diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index 3d38d89d27..81302520e6 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -54,41 +54,43 @@ class GeneralTests(TestCase): def testBasic1(self): # connects smtp = smtplib.SMTP(HOST, self.port) - smtp.sock.close() + smtp.close() def testBasic2(self): # connects, include port in host name smtp = smtplib.SMTP("%s:%s" % (HOST, self.port)) - smtp.sock.close() + smtp.close() def testLocalHostName(self): # check that supplied local_hostname is used smtp = smtplib.SMTP(HOST, self.port, local_hostname="testhost") self.assertEqual(smtp.local_hostname, "testhost") - smtp.sock.close() + smtp.close() def testTimeoutDefault(self): - # default - smtp = smtplib.SMTP(HOST, self.port) - self.assertTrue(smtp.sock.gettimeout() is None) - smtp.sock.close() - - def testTimeoutValue(self): - # a value - smtp = smtplib.SMTP(HOST, self.port, timeout=30) + self.assertTrue(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + smtp = smtplib.SMTP(HOST, self.port) + finally: + socket.setdefaulttimeout(None) self.assertEqual(smtp.sock.gettimeout(), 30) - smtp.sock.close() + smtp.close() def testTimeoutNone(self): - # None, having other default - previous = socket.getdefaulttimeout() + self.assertTrue(socket.getdefaulttimeout() is None) socket.setdefaulttimeout(30) try: smtp = smtplib.SMTP(HOST, self.port, timeout=None) finally: - socket.setdefaulttimeout(previous) + socket.setdefaulttimeout(None) + self.assertTrue(smtp.sock.gettimeout() is None) + smtp.close() + + def testTimeoutValue(self): + smtp = smtplib.SMTP(HOST, self.port, timeout=30) self.assertEqual(smtp.sock.gettimeout(), 30) - smtp.sock.close() + smtp.close() # Test server thread using the specified SMTP server class diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index d6645822e5..e62788ee27 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -929,8 +929,25 @@ class NetworkConnectionAttributesTest(SocketTCPTest, ThreadableTest): testTimeoutDefault = _justAccept def _testTimeoutDefault(self): - self.cli = socket.create_connection((HOST, self.port)) - self.assertTrue(self.cli.gettimeout() is None) + # passing no explicit timeout uses socket's global default + self.assert_(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(42) + try: + self.cli = socket.create_connection((HOST, self.port)) + finally: + socket.setdefaulttimeout(None) + self.assertEquals(self.cli.gettimeout(), 42) + + testTimeoutNone = _justAccept + def _testTimeoutNone(self): + # None timeout means the same as sock.settimeout(None) + self.assert_(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + self.cli = socket.create_connection((HOST, self.port), timeout=None) + finally: + socket.setdefaulttimeout(None) + self.assertEqual(self.cli.gettimeout(), None) testTimeoutValueNamed = _justAccept def _testTimeoutValueNamed(self): @@ -942,17 +959,6 @@ class NetworkConnectionAttributesTest(SocketTCPTest, ThreadableTest): self.cli = socket.create_connection((HOST, self.port), 30) self.assertEqual(self.cli.gettimeout(), 30) - testTimeoutNone = _justAccept - def _testTimeoutNone(self): - previous = socket.getdefaulttimeout() - socket.setdefaulttimeout(30) - try: - self.cli = socket.create_connection((HOST, self.port), timeout=None) - finally: - socket.setdefaulttimeout(previous) - self.assertEqual(self.cli.gettimeout(), 30) - - class NetworkConnectionBehaviourTest(SocketTCPTest, ThreadableTest): def __init__(self, methodName='runTest'): diff --git a/Lib/test/test_telnetlib.py b/Lib/test/test_telnetlib.py index c24d41d02d..4ae6222f6c 100644 --- a/Lib/test/test_telnetlib.py +++ b/Lib/test/test_telnetlib.py @@ -40,34 +40,36 @@ class GeneralTests(TestCase): telnet.sock.close() def testTimeoutDefault(self): - # default - telnet = telnetlib.Telnet(HOST, self.port) - self.assertTrue(telnet.sock.gettimeout() is None) - telnet.sock.close() - - def testTimeoutValue(self): - # a value - telnet = telnetlib.Telnet(HOST, self.port, timeout=30) - self.assertEqual(telnet.sock.gettimeout(), 30) - telnet.sock.close() - - def testTimeoutDifferentOrder(self): - telnet = telnetlib.Telnet(timeout=30) - telnet.open(HOST, self.port) + self.assertTrue(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(30) + try: + telnet = telnetlib.Telnet("localhost", self.port) + finally: + socket.setdefaulttimeout(None) self.assertEqual(telnet.sock.gettimeout(), 30) telnet.sock.close() def testTimeoutNone(self): # None, having other default - previous = socket.getdefaulttimeout() + self.assertTrue(socket.getdefaulttimeout() is None) socket.setdefaulttimeout(30) try: telnet = telnetlib.Telnet(HOST, self.port, timeout=None) finally: - socket.setdefaulttimeout(previous) + socket.setdefaulttimeout(None) + self.assertTrue(telnet.sock.gettimeout() is None) + telnet.sock.close() + + def testTimeoutValue(self): + telnet = telnetlib.Telnet("localhost", self.port, timeout=30) self.assertEqual(telnet.sock.gettimeout(), 30) telnet.sock.close() + def testTimeoutOpen(self): + telnet = telnetlib.Telnet() + telnet.open("localhost", self.port, timeout=30) + self.assertEqual(telnet.sock.gettimeout(), 30) + telnet.sock.close() def test_main(verbose=None): diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py index b14510fce2..26b51cb77c 100644 --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -568,6 +568,7 @@ class Pathname_Tests(unittest.TestCase): # . Facundo # # def server(evt): +# import socket, time # serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # serv.settimeout(3) # serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) @@ -592,6 +593,7 @@ class Pathname_Tests(unittest.TestCase): # class FTPWrapperTests(unittest.TestCase): # # def setUp(self): +# import ftplib, time, threading # ftplib.FTP.port = 9093 # self.evt = threading.Event() # threading.Thread(target=server, args=(self.evt,)).start() @@ -603,31 +605,37 @@ class Pathname_Tests(unittest.TestCase): # def testBasic(self): # # connects # ftp = urllib.ftpwrapper("myuser", "mypass", "localhost", 9093, []) -# ftp.ftp.sock.close() +# ftp.close() # -# def testTimeoutDefault(self): -# # default -# ftp = urllib.ftpwrapper("myuser", "mypass", "localhost", 9093, []) -# self.assertTrue(ftp.ftp.sock.gettimeout() is None) -# ftp.ftp.sock.close() -# -# def testTimeoutValue(self): -# # a value -# ftp = urllib.ftpwrapper("myuser", "mypass", "localhost", 9093, [], timeout=30) +# def testTimeoutNone(self): +# # global default timeout is ignored +# import socket +# self.assert_(socket.getdefaulttimeout() is None) +# socket.setdefaulttimeout(30) +# try: +# ftp = urllib.ftpwrapper("myuser", "mypass", "localhost", 9093, []) +# finally: +# socket.setdefaulttimeout(None) # self.assertEqual(ftp.ftp.sock.gettimeout(), 30) -# ftp.ftp.sock.close() +# ftp.close() # -# def testTimeoutNone(self): -# # None, having other default -# previous = socket.getdefaulttimeout() +# def testTimeoutDefault(self): +# # global default timeout is used +# import socket +# self.assert_(socket.getdefaulttimeout() is None) # socket.setdefaulttimeout(30) # try: # ftp = urllib.ftpwrapper("myuser", "mypass", "localhost", 9093, []) # finally: -# socket.setdefaulttimeout(previous) +# socket.setdefaulttimeout(None) # self.assertEqual(ftp.ftp.sock.gettimeout(), 30) -# ftp.ftp.close() +# ftp.close() # +# def testTimeoutValue(self): +# ftp = urllib.ftpwrapper("myuser", "mypass", "localhost", 9093, [], +# timeout=30) +# self.assertEqual(ftp.ftp.sock.gettimeout(), 30) +# ftp.close() diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 33868002fc..c2d594b400 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -3,6 +3,7 @@ from test import support import os import io +import socket import urllib2 from urllib2 import Request, OpenerDirector @@ -546,14 +547,15 @@ class HandlerTests(unittest.TestCase): class NullFTPHandler(urllib2.FTPHandler): def __init__(self, data): self.data = data - def connect_ftp(self, user, passwd, host, port, dirs, timeout=None): + def connect_ftp(self, user, passwd, host, port, dirs, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): self.user, self.passwd = user, passwd self.host, self.port = host, port self.dirs = dirs self.ftpwrapper = MockFTPWrapper(self.data) return self.ftpwrapper - import ftplib, socket + import ftplib data = "rheum rhaponicum" h = NullFTPHandler(data) o = h.parent = MockOpener() @@ -686,7 +688,7 @@ class HandlerTests(unittest.TestCase): self.req_headers = [] self.data = None self.raise_on_endheaders = False - def __call__(self, host, timeout=None): + def __call__(self, host, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): self.host = host self.timeout = timeout return self diff --git a/Lib/test/test_urllib2net.py b/Lib/test/test_urllib2net.py index 990d3da536..752406c446 100644 --- a/Lib/test/test_urllib2net.py +++ b/Lib/test/test_urllib2net.py @@ -188,46 +188,58 @@ class OtherNetworkTests(unittest.TestCase): class TimeoutTest(unittest.TestCase): def test_http_basic(self): + self.assertTrue(socket.getdefaulttimeout() is None) u = _urlopen_with_retry("http://www.python.org") self.assertTrue(u.fp.raw.fp._sock.gettimeout() is None) - def test_http_NoneWithdefault(self): - prev = socket.getdefaulttimeout() + def test_http_default_timeout(self): + self.assertTrue(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(60) + try: + u = _urlopen_with_retry("http://www.python.org") + finally: + socket.setdefaulttimeout(None) + self.assertEqual(u.fp.raw.fp._sock.gettimeout(), 60) + + def test_http_no_timeout(self): + self.assertTrue(socket.getdefaulttimeout() is None) socket.setdefaulttimeout(60) try: u = _urlopen_with_retry("http://www.python.org", timeout=None) - self.assertTrue(u.fp.raw.fp._sock.gettimeout(), 60) finally: - socket.setdefaulttimeout(prev) + socket.setdefaulttimeout(None) + self.assertTrue(u.fp.raw.fp._sock.gettimeout() is None) - def test_http_Value(self): + def test_http_timeout(self): u = _urlopen_with_retry("http://www.python.org", timeout=120) self.assertEqual(u.fp.raw.fp._sock.gettimeout(), 120) - def test_http_NoneNodefault(self): - u = _urlopen_with_retry("http://www.python.org", timeout=None) - self.assertTrue(u.fp.raw.fp._sock.gettimeout() is None) - FTP_HOST = "ftp://ftp.mirror.nl/pub/mirror/gnu/" def test_ftp_basic(self): + self.assertTrue(socket.getdefaulttimeout() is None) u = _urlopen_with_retry(self.FTP_HOST) self.assertTrue(u.fp.fp.raw._sock.gettimeout() is None) - def test_ftp_NoneWithdefault(self): - prev = socket.getdefaulttimeout() + def test_ftp_default_timeout(self): + self.assertTrue(socket.getdefaulttimeout() is None) socket.setdefaulttimeout(60) try: - u = _urlopen_with_retry(self.FTP_HOST, timeout=None) - self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60) + u = _urlopen_with_retry(self.FTP_HOST) finally: - socket.setdefaulttimeout(prev) + socket.setdefaulttimeout(None) + self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60) - def test_ftp_NoneNodefault(self): - u = _urlopen_with_retry(self.FTP_HOST, timeout=None) + def test_ftp_no_timeout(self): + self.assertTrue(socket.getdefaulttimeout() is None) + socket.setdefaulttimeout(60) + try: + u = _urlopen_with_retry(self.FTP_HOST, timeout=None) + finally: + socket.setdefaulttimeout(None) self.assertTrue(u.fp.fp.raw._sock.gettimeout() is None) - def test_ftp_Value(self): + def test_ftp_timeout(self): u = _urlopen_with_retry(self.FTP_HOST, timeout=60) self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60) diff --git a/Lib/urllib.py b/Lib/urllib.py index 2005baf194..fb93604902 100644 --- a/Lib/urllib.py +++ b/Lib/urllib.py @@ -776,7 +776,8 @@ def noheaders(): class ftpwrapper: """Class used by open_ftp() for cache of open FTP connections.""" - def __init__(self, user, passwd, host, port, dirs, timeout=None): + def __init__(self, user, passwd, host, port, dirs, + timeout=socket._GLOBAL_DEFAULT_TIMEOUT): self.user = user self.passwd = passwd self.host = host diff --git a/Lib/urllib2.py b/Lib/urllib2.py index 948c6c3c35..244d737819 100644 --- a/Lib/urllib2.py +++ b/Lib/urllib2.py @@ -115,7 +115,7 @@ from urllib import localhost, url2pathname, getproxies __version__ = sys.version[:3] _opener = None -def urlopen(url, data=None, timeout=None): +def urlopen(url, data=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): global _opener if _opener is None: _opener = build_opener() @@ -357,7 +357,7 @@ class OpenerDirector: if result is not None: return result - def open(self, fullurl, data=None, timeout=None): + def open(self, fullurl, data=None, timeout=socket._GLOBAL_DEFAULT_TIMEOUT): # accept a URL or a Request object if isinstance(fullurl, str): req = Request(fullurl, data) diff --git a/Modules/_ctypes/libffi/configure b/Modules/_ctypes/libffi/configure index ee8f63650e..aaa7914190 100755 --- a/Modules/_ctypes/libffi/configure +++ b/Modules/_ctypes/libffi/configure @@ -20406,6 +20406,9 @@ case "$host" in i?86-*-solaris2.1[0-9]*) TARGET=X86_64; TARGETDIR=x86 ;; + i*86-*-nto-qnx*) + TARGET=X86; TARGETDIR=x86 + ;; i?86-*-*) TARGET=X86; TARGETDIR=x86 ;; diff --git a/Modules/_ctypes/libffi/configure.ac b/Modules/_ctypes/libffi/configure.ac index 0a85146b1a..57c161befe 100644 --- a/Modules/_ctypes/libffi/configure.ac +++ b/Modules/_ctypes/libffi/configure.ac @@ -86,6 +86,9 @@ case "$host" in i?86-*-solaris2.1[[0-9]]*) TARGET=X86_64; TARGETDIR=x86 ;; + i*86-*-nto-qnx*) + TARGET=X86; TARGETDIR=x86 + ;; i?86-*-*) TARGET=X86; TARGETDIR=x86 ;; diff --git a/Modules/_ctypes/malloc_closure.c b/Modules/_ctypes/malloc_closure.c index 29e9f4c4b2..b7d58c00bd 100644 --- a/Modules/_ctypes/malloc_closure.c +++ b/Modules/_ctypes/malloc_closure.c @@ -44,7 +44,11 @@ static void more_core(void) } #else if (!_pagesize) { +#ifdef _SC_PAGESIZE + _pagesize = sysconf(_SC_PAGESIZE); +#else _pagesize = getpagesize(); +#endif } #endif diff --git a/Modules/_heapqmodule.c b/Modules/_heapqmodule.c index 97ccd86e4d..a5a56053d6 100644 --- a/Modules/_heapqmodule.c +++ b/Modules/_heapqmodule.c @@ -8,6 +8,25 @@ annotated by François Pinard, and converted to C by Raymond Hettinger. #include "Python.h" +/* Older implementations of heapq used Py_LE for comparisons. Now, it uses + Py_LT so it will match min(), sorted(), and bisect(). Unfortunately, some + client code (Twisted for example) relied on Py_LE, so this little function + restores compatability by trying both. +*/ +static int +cmp_lt(PyObject *x, PyObject *y) +{ + int cmp; + cmp = PyObject_RichCompareBool(x, y, Py_LT); + if (cmp == -1 && PyErr_ExceptionMatches(PyExc_AttributeError)) { + PyErr_Clear(); + cmp = PyObject_RichCompareBool(y, x, Py_LE); + if (cmp != -1) + cmp = 1 - cmp; + } + return cmp; +} + static int _siftdown(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos) { @@ -28,12 +47,12 @@ _siftdown(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos) while (pos > startpos){ parentpos = (pos - 1) >> 1; parent = PyList_GET_ITEM(heap, parentpos); - cmp = PyObject_RichCompareBool(parent, newitem, Py_LE); + cmp = cmp_lt(newitem, parent); if (cmp == -1) { Py_DECREF(newitem); return -1; } - if (cmp == 1) + if (cmp == 0) break; Py_INCREF(parent); Py_DECREF(PyList_GET_ITEM(heap, pos)); @@ -68,15 +87,14 @@ _siftup(PyListObject *heap, Py_ssize_t pos) /* Set childpos to index of smaller child. */ rightpos = childpos + 1; if (rightpos < endpos) { - cmp = PyObject_RichCompareBool( - PyList_GET_ITEM(heap, rightpos), + cmp = cmp_lt( PyList_GET_ITEM(heap, childpos), - Py_LE); + PyList_GET_ITEM(heap, rightpos)); if (cmp == -1) { Py_DECREF(newitem); return -1; } - if (cmp == 1) + if (cmp == 0) childpos = rightpos; } /* Move the smaller child up. */ @@ -214,10 +232,10 @@ heappushpop(PyObject *self, PyObject *args) return item; } - cmp = PyObject_RichCompareBool(item, PyList_GET_ITEM(heap, 0), Py_LE); + cmp = cmp_lt(PyList_GET_ITEM(heap, 0), item); if (cmp == -1) return NULL; - if (cmp == 1) { + if (cmp == 0) { Py_INCREF(item); return item; } @@ -270,6 +288,7 @@ nlargest(PyObject *self, PyObject *args) { PyObject *heap=NULL, *elem, *iterable, *sol, *it, *oldelem; Py_ssize_t i, n; + int cmp; if (!PyArg_ParseTuple(args, "nO:nlargest", &n, &iterable)) return NULL; @@ -312,7 +331,12 @@ nlargest(PyObject *self, PyObject *args) else goto sortit; } - if (PyObject_RichCompareBool(elem, sol, Py_LE)) { + cmp = cmp_lt(sol, elem); + if (cmp == -1) { + Py_DECREF(elem); + goto fail; + } + if (cmp == 0) { Py_DECREF(elem); continue; } @@ -362,12 +386,12 @@ _siftdownmax(PyListObject *heap, Py_ssize_t startpos, Py_ssize_t pos) while (pos > startpos){ parentpos = (pos - 1) >> 1; parent = PyList_GET_ITEM(heap, parentpos); - cmp = PyObject_RichCompareBool(newitem, parent, Py_LE); + cmp = cmp_lt(parent, newitem); if (cmp == -1) { Py_DECREF(newitem); return -1; } - if (cmp == 1) + if (cmp == 0) break; Py_INCREF(parent); Py_DECREF(PyList_GET_ITEM(heap, pos)); @@ -402,15 +426,14 @@ _siftupmax(PyListObject *heap, Py_ssize_t pos) /* Set childpos to index of smaller child. */ rightpos = childpos + 1; if (rightpos < endpos) { - cmp = PyObject_RichCompareBool( - PyList_GET_ITEM(heap, childpos), + cmp = cmp_lt( PyList_GET_ITEM(heap, rightpos), - Py_LE); + PyList_GET_ITEM(heap, childpos)); if (cmp == -1) { Py_DECREF(newitem); return -1; } - if (cmp == 1) + if (cmp == 0) childpos = rightpos; } /* Move the smaller child up. */ @@ -434,6 +457,7 @@ nsmallest(PyObject *self, PyObject *args) { PyObject *heap=NULL, *elem, *iterable, *los, *it, *oldelem; Py_ssize_t i, n; + int cmp; if (!PyArg_ParseTuple(args, "nO:nsmallest", &n, &iterable)) return NULL; @@ -477,7 +501,12 @@ nsmallest(PyObject *self, PyObject *args) else goto sortit; } - if (PyObject_RichCompareBool(los, elem, Py_LE)) { + cmp = cmp_lt(elem, los); + if (cmp == -1) { + Py_DECREF(elem); + goto fail; + } + if (cmp == 0) { Py_DECREF(elem); continue; } diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c index 243b0b66cc..29d3ff4ee2 100644 --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -101,10 +101,7 @@ int pysqlite_cursor_init(pysqlite_Cursor* self, PyObject* args, PyObject* kwargs self->arraysize = 1; - self->rowcount = PyLong_FromLong(-1L); - if (!self->rowcount) { - return -1; - } + self->rowcount = -1L; Py_INCREF(Py_None); self->row_factory = Py_None; @@ -130,7 +127,6 @@ void pysqlite_cursor_dealloc(pysqlite_Cursor* self) Py_XDECREF(self->row_cast_map); Py_XDECREF(self->description); Py_XDECREF(self->lastrowid); - Py_XDECREF(self->rowcount); Py_XDECREF(self->row_factory); Py_XDECREF(self->next_row); @@ -418,12 +414,12 @@ PyObject* _pysqlite_query_execute(pysqlite_Cursor* self, int multiple, PyObject* int statement_type; PyObject* descriptor; PyObject* second_argument = NULL; - long rowcount = 0; int allow_8bit_chars; if (!pysqlite_check_thread(self->connection) || !pysqlite_check_connection(self->connection)) { return NULL; } + /* Make shooting yourself in the foot with not utf-8 decodable 8-bit-strings harder */ allow_8bit_chars = ((self->connection->text_factory != (PyObject*)&PyUnicode_Type) && (self->connection->text_factory != (PyObject*)&PyUnicode_Type && pysqlite_OptimizedUnicode)); @@ -498,10 +494,11 @@ PyObject* _pysqlite_query_execute(pysqlite_Cursor* self, int multiple, PyObject* if (operation == NULL) goto error; - /* reset description */ + /* reset description and rowcount */ Py_DECREF(self->description); Py_INCREF(Py_None); self->description = Py_None; + self->rowcount = -1L; func_args = PyTuple_New(1); if (!func_args) { @@ -723,7 +720,10 @@ PyObject* _pysqlite_query_execute(pysqlite_Cursor* self, int multiple, PyObject* case STATEMENT_DELETE: case STATEMENT_INSERT: case STATEMENT_REPLACE: - rowcount += (long)sqlite3_changes(self->connection->db); + if (self->rowcount == -1L) { + self->rowcount = 0L; + } + self->rowcount += (long)sqlite3_changes(self->connection->db); } Py_DECREF(self->lastrowid); @@ -757,13 +757,9 @@ error: Py_XDECREF(parameters_list); if (PyErr_Occurred()) { - Py_DECREF(self->rowcount); - self->rowcount = PyLong_FromLong(-1L); + self->rowcount = -1L; return NULL; } else { - Py_DECREF(self->rowcount); - self->rowcount = PyLong_FromLong(rowcount); - Py_INCREF(self); return (PyObject*)self; } @@ -1053,7 +1049,7 @@ static struct PyMemberDef cursor_members[] = {"description", T_OBJECT, offsetof(pysqlite_Cursor, description), READONLY}, {"arraysize", T_INT, offsetof(pysqlite_Cursor, arraysize), 0}, {"lastrowid", T_OBJECT, offsetof(pysqlite_Cursor, lastrowid), READONLY}, - {"rowcount", T_OBJECT, offsetof(pysqlite_Cursor, rowcount), READONLY}, + {"rowcount", T_LONG, offsetof(pysqlite_Cursor, rowcount), READONLY}, {"row_factory", T_OBJECT, offsetof(pysqlite_Cursor, row_factory), 0}, {NULL} }; diff --git a/Modules/_sqlite/cursor.h b/Modules/_sqlite/cursor.h index d916ca5fc6..54d816db91 100644 --- a/Modules/_sqlite/cursor.h +++ b/Modules/_sqlite/cursor.h @@ -37,7 +37,7 @@ typedef struct PyObject* row_cast_map; int arraysize; PyObject* lastrowid; - PyObject* rowcount; + long rowcount; PyObject* row_factory; pysqlite_Statement* statement; diff --git a/Modules/datetimemodule.c b/Modules/datetimemodule.c index 0ad5b91062..f37290c041 100644 --- a/Modules/datetimemodule.c +++ b/Modules/datetimemodule.c @@ -1201,17 +1201,17 @@ wrap_strftime(PyObject *object, PyObject *format, PyObject *timetuple, PyObject *Zreplacement = NULL; /* py string, replacement for %Z */ PyObject *freplacement = NULL; /* py string, replacement for %f */ - const char *pin;/* pointer to next char in input format */ - Py_ssize_t flen;/* length of input format */ - char ch; /* next char in input format */ + const char *pin; /* pointer to next char in input format */ + Py_ssize_t flen; /* length of input format */ + char ch; /* next char in input format */ PyObject *newfmt = NULL; /* py string, the output format */ char *pnew; /* pointer to available byte in output format */ - int totalnew; /* number bytes total in output format buffer, - exclusive of trailing \0 */ - int usednew; /* number bytes used so far in output format buffer */ + size_t totalnew; /* number bytes total in output format buffer, + exclusive of trailing \0 */ + size_t usednew; /* number bytes used so far in output format buffer */ - const char *ptoappend;/* pointer to string to append to output buffer */ + const char *ptoappend; /* ptr to string to append to output buffer */ Py_ssize_t ntoappend; /* # of bytes to append to output buffer */ assert(object && format && timetuple); @@ -1335,7 +1335,7 @@ wrap_strftime(PyObject *object, PyObject *format, PyObject *timetuple, assert(ptoappend != NULL); assert(ntoappend > 0); while (usednew + ntoappend > totalnew) { - int bigger = totalnew << 1; + size_t bigger = totalnew << 1; if ((bigger >> 1) != totalnew) { /* overflow */ PyErr_NoMemory(); goto Done; @@ -2444,8 +2444,8 @@ date_strftime(PyDateTime_Date *self, PyObject *args, PyObject *kw) * timetuple() method appropriate to self's class. */ PyObject *result; - PyObject *format; PyObject *tuple; + PyObject *format; static char *keywords[] = {"format", NULL}; if (! PyArg_ParseTupleAndKeywords(args, kw, "U:strftime", keywords, @@ -3211,8 +3211,8 @@ static PyObject * time_strftime(PyDateTime_Time *self, PyObject *args, PyObject *kw) { PyObject *result; - PyObject *format; PyObject *tuple; + PyObject *format; static char *keywords[] = {"format", NULL}; if (! PyArg_ParseTupleAndKeywords(args, kw, "U:strftime", keywords, @@ -3232,7 +3232,8 @@ time_strftime(PyDateTime_Time *self, PyObject *args, PyObject *kw) if (tuple == NULL) return NULL; assert(PyTuple_Size(tuple) == 9); - result = wrap_strftime((PyObject *)self, format, tuple, Py_None); + result = wrap_strftime((PyObject *)self, format, tuple, + Py_None); Py_DECREF(tuple); return result; } diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 5c5def2333..16c829ac25 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -373,14 +373,20 @@ FUNC1(tanh, tanh, 0, value semantics across iterations (i.e. handling -Inf + Inf). Note 2: No provision is made for intermediate overflow handling; - therefore, sum([1e+308, 1e-308, 1e+308]) returns result 1e+308 while + therefore, sum([1e+308, 1e-308, 1e+308]) returns 1e+308 while sum([1e+308, 1e+308, 1e-308]) raises an OverflowError due to the overflow of the first partial sum. - Note 3: Aggressively optimizing compilers can potentially eliminate the - residual values needed for accurate summation. For instance, the statements - "hi = x + y; lo = y - (hi - x);" could be mis-transformed to - "hi = x + y; lo = 0.0;" which defeats the computation of residuals. + Note 3: The itermediate values lo, yr, and hi are declared volatile so + aggressive compilers won't algebraicly reduce lo to always be exactly 0.0. + Also, the volatile declaration forces the values to be stored in memory as + regular doubles instead of extended long precision (80-bit) values. This + prevents double rounding because any addition or substraction of two doubles + can be resolved exactly into double-sized hi and lo values. As long as the + hi value gets forced into a double before yr and lo are computed, the extra + bits in downstream extended precision operations (x87 for example) will be + exactly zero and therefore can be losslessly stored back into a double, + thereby preventing double rounding. Note 4: A similar implementation is in Modules/cmathmodule.c. Be sure to update both when making changes. @@ -457,7 +463,8 @@ math_sum(PyObject *self, PyObject *seq) { PyObject *item, *iter, *sum = NULL; Py_ssize_t i, j, n = 0, m = NUM_PARTIALS; - double x, y, hi, lo=0.0, ps[NUM_PARTIALS], *p = ps; + double x, y, t, ps[NUM_PARTIALS], *p = ps; + volatile double hi, yr, lo; iter = PyObject_GetIter(seq); if (iter == NULL) @@ -483,10 +490,12 @@ math_sum(PyObject *self, PyObject *seq) for (i = j = 0; j < n; j++) { /* for y in partials */ y = p[j]; + if (fabs(x) < fabs(y)) { + t = x; x = y; y = t; + } hi = x + y; - lo = fabs(x) < fabs(y) - ? x - (hi - y) - : y - (hi - x); + yr = hi - x; + lo = y - yr; if (lo != 0.0) p[i++] = lo; x = hi; @@ -506,38 +515,41 @@ math_sum(PyObject *self, PyObject *seq) } } + hi = 0.0; if (n > 0) { hi = p[--n]; if (Py_IS_FINITE(hi)) { /* sum_exact(ps, hi) from the top, stop when the sum becomes inexact. */ while (n > 0) { - x = p[--n]; - y = hi; + x = hi; + y = p[--n]; + assert(fabs(y) < fabs(x)); hi = x + y; - assert(fabs(x) < fabs(y)); - lo = x - (hi - y); + yr = hi - x; + lo = y - yr; if (lo != 0.0) break; } - /* Little dance to allow half-even rounding across multiple partials. - Needed so that sum([1e-16, 1, 1e16]) will round-up to two instead - of down to zero (the 1e16 makes the 1 slightly closer to two). */ + /* Make half-even rounding work across multiple partials. Needed + so that sum([1e-16, 1, 1e16]) will round-up the last digit to + two instead of down to zero (the 1e-16 makes the 1 slightly + closer to two). With a potential 1 ULP rounding error fixed-up, + math.sum() can guarantee commutativity. */ if (n > 0 && ((lo < 0.0 && p[n-1] < 0.0) || (lo > 0.0 && p[n-1] > 0.0))) { y = lo * 2.0; x = hi + y; - if (y == (x - hi)) + yr = x - hi; + if (y == yr) hi = x; } } - else { /* raise corresponding error */ + else { /* raise exception corresponding to a special value */ errno = Py_IS_NAN(hi) ? EDOM : ERANGE; if (is_error(hi)) goto _sum_error; } } - else /* default */ - hi = 0.0; sum = PyFloat_FromDouble(hi); _sum_error: diff --git a/PCbuild/pyproject.vsprops b/PCbuild/pyproject.vsprops index ccf579dcab..d1e5d86f70 100644 --- a/PCbuild/pyproject.vsprops +++ b/PCbuild/pyproject.vsprops @@ -45,6 +45,10 @@ Value="$(SolutionDir)\python.exe" /> <UserMacro + Name="externalsDir" + Value="..\.." + /> + <UserMacro Name="bsddbDir" Value="$(bsddb44Dir)" /> @@ -54,7 +58,7 @@ /> <UserMacro Name="bsddb44Dir" - Value="..\..\db-4.4.20\build_win32" + Value="$(externalsDir)\db-4.4.20\build_win32" /> <UserMacro Name="bsddb44DepLibs" @@ -62,7 +66,7 @@ /> <UserMacro Name="bsddb45Dir" - Value="..\..\db-4.5.20.x\build_windows" + Value="$(externalsDir)\db-4.5.20.x\build_windows" /> <UserMacro Name="bsddb45DepLibs" @@ -70,23 +74,23 @@ /> <UserMacro Name="sqlite3Dir" - Value="..\..\sqlite-source-3.3.4" + Value="$(externalsDir)\sqlite-source-3.3.4" /> <UserMacro Name="bz2Dir" - Value="..\..\bzip2-1.0.3" + Value="$(externalsDir)\bzip2-1.0.3" /> <UserMacro Name="opensslDir" - Value="..\..\openssl-0.9.8g" + Value="$(externalsDir)\openssl-0.9.8g" /> <UserMacro Name="tcltkDir" - Value="..\..\tcltk" + Value="$(externalsDir)\tcltk" /> <UserMacro Name="tcltk64Dir" - Value="..\..\tcltk64" + Value="$(externalsDir)\tcltk64" /> <UserMacro Name="tcltkLib" diff --git a/Python/mysnprintf.c b/Python/mysnprintf.c index 4d3770d894..3173863c46 100644 --- a/Python/mysnprintf.c +++ b/Python/mysnprintf.c @@ -54,18 +54,28 @@ int PyOS_vsnprintf(char *str, size_t size, const char *format, va_list va) { int len; /* # bytes written, excluding \0 */ -#ifndef HAVE_SNPRINTF +#ifdef HAVE_SNPRINTF +#define _PyOS_vsnprintf_EXTRA_SPACE 1 +#else +#define _PyOS_vsnprintf_EXTRA_SPACE 512 char *buffer; #endif assert(str != NULL); assert(size > 0); assert(format != NULL); + /* We take a size_t as input but return an int. Sanity check + * our input so that it won't cause an overflow in the + * vsnprintf return value or the buffer malloc size. */ + if (size > INT_MAX - _PyOS_vsnprintf_EXTRA_SPACE) { + len = -666; + goto Done; + } #ifdef HAVE_SNPRINTF len = vsnprintf(str, size, format, va); #else /* Emulate it. */ - buffer = PyMem_MALLOC(size + 512); + buffer = PyMem_MALLOC(size + _PyOS_vsnprintf_EXTRA_SPACE); if (buffer == NULL) { len = -666; goto Done; @@ -75,7 +85,7 @@ PyOS_vsnprintf(char *str, size_t size, const char *format, va_list va) if (len < 0) /* ignore the error */; - else if ((size_t)len >= size + 512) + else if ((size_t)len >= size + _PyOS_vsnprintf_EXTRA_SPACE) Py_FatalError("Buffer overflow in PyOS_snprintf/PyOS_vsnprintf"); else { @@ -86,8 +96,10 @@ PyOS_vsnprintf(char *str, size_t size, const char *format, va_list va) str[to_copy] = '\0'; } PyMem_FREE(buffer); -Done: #endif - str[size-1] = '\0'; +Done: + if (size > 0) + str[size-1] = '\0'; return len; +#undef _PyOS_vsnprintf_EXTRA_SPACE } diff --git a/configure.in b/configure.in index fd6f68d1d1..baa305c1a6 100644 --- a/configure.in +++ b/configure.in @@ -217,7 +217,7 @@ case $ac_sys_system/$ac_sys_release in # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined, # even though select is a POSIX function. Reported by J. Ribbens. # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish. - OpenBSD/2.* | OpenBSD/3.@<:@0123456789@:>@ | OpenBSD/4.@<:@0@:>@) + OpenBSD/2.* | OpenBSD/3.@<:@0123456789@:>@ | OpenBSD/4.@<:@0123@:>@) define_xopen_source=no # OpenBSD undoes our definition of __BSD_VISIBLE if _XOPEN_SOURCE is # also defined. This can be overridden by defining _BSD_SOURCE @@ -265,6 +265,11 @@ case $ac_sys_system/$ac_sys_release in Darwin/@<:@789@:>@.*) define_xopen_source=no ;; + # On QNX 6.3.2, defining _XOPEN_SOURCE prevents netdb.h from + # defining NI_NUMERICHOST. + QNX/6.3.2) + define_xopen_source=no + ;; esac @@ -549,6 +554,10 @@ then LINKCC="\$(srcdir)/Modules/makexp_aix Modules/python.exp $exp_extra \$(LIBRARY); $LINKCC";; Monterey64*) LINKCC="$LINKCC -L/usr/lib/ia64l64";; + QNX*) + # qcc must be used because the other compilers do not + # support -N. + LINKCC=qcc;; esac fi AC_MSG_RESULT($LINKCC) @@ -1091,7 +1100,7 @@ unistd.h utime.h \ sys/audioio.h sys/bsdtty.h sys/epoll.h sys/event.h sys/file.h sys/loadavg.h \ sys/lock.h sys/mkdev.h sys/modem.h \ sys/param.h sys/poll.h sys/select.h sys/socket.h sys/statvfs.h sys/stat.h \ -sys/time.h \ +sys/termio.h sys/time.h \ sys/times.h sys/types.h sys/un.h sys/utsname.h sys/wait.h pty.h libutil.h \ sys/resource.h netpacket/packet.h sysexits.h bluetooth.h \ bluetooth/bluetooth.h linux/tipc.h) @@ -1517,7 +1526,7 @@ then fi fi ;; - Linux*|GNU*) LDSHARED='$(CC) -shared';; + Linux*|GNU*|QNX*) LDSHARED='$(CC) -shared';; BSD/OS*/4*) LDSHARED="gcc -shared";; FreeBSD*) if [[ "`$CC -dM -E - </dev/null | grep __ELF__`" != "" ]] @@ -1641,6 +1650,13 @@ then then LINKFORSHARED='-Wl,--out-implib=$(LDLIBRARY)' fi;; + QNX*) + # -Wl,-E causes the symbols to be added to the dynamic + # symbol table so that they can be found when a module + # is loaded. -N 2048K causes the stack size to be set + # to 2048 kilobytes so that the stack doesn't overflow + # when running test_compile.py. + LINKFORSHARED='-Wl,-E -N 2048K';; esac fi AC_MSG_RESULT($LINKFORSHARED) diff --git a/pyconfig.h.in b/pyconfig.h.in index bbcc1d332c..d5ec2b0166 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -691,6 +691,9 @@ /* Define to 1 if you have the <sys/stat.h> header file. */ #undef HAVE_SYS_STAT_H +/* Define to 1 if you have the <sys/termio.h> header file. */ +#undef HAVE_SYS_TERMIO_H + /* Define to 1 if you have the <sys/times.h> header file. */ #undef HAVE_SYS_TIMES_H @@ -939,7 +939,7 @@ class PyBuildExt(build_ext): missing.append('resource') # Sun yellow pages. Some systems have the functions in libc. - if platform not in ['cygwin', 'atheos']: + if platform not in ['cygwin', 'atheos', 'qnx6']: if (self.compiler.find_library_file(lib_dirs, 'nsl')): libs = ['nsl'] else: |