diff options
author | Fred Drake <fdrake@acm.org> | 2002-06-18 20:32:09 +0000 |
---|---|---|
committer | Fred Drake <fdrake@acm.org> | 2002-06-18 20:32:09 +0000 |
commit | df726031dce3fe0b767ba1ed547c6cda81001c5d (patch) | |
tree | 4c52c1b2ee8201fc1e85e66fb333b09cd5b2bb84 | |
parent | 8304f5431fc3b4810a5fa5e1e086684ac4ea7bc7 (diff) | |
download | cpython-df726031dce3fe0b767ba1ed547c6cda81001c5d.tar.gz |
Add description of the deadlock problem with child processes and pipes, and
hints about how to work around it.
Closes SF bug #530637.
-rw-r--r-- | Doc/lib/libos.tex | 5 | ||||
-rw-r--r-- | Doc/lib/libpopen2.tex | 62 |
2 files changed, 67 insertions, 0 deletions
diff --git a/Doc/lib/libos.tex b/Doc/lib/libos.tex index b699f2afad..42145f5f3f 100644 --- a/Doc/lib/libos.tex +++ b/Doc/lib/libos.tex @@ -314,6 +314,11 @@ specified, it specifies the buffer size for the I/O pipes. objects should be opened in binary or text mode. The default value for \var{mode} is \code{'t'}. +For a discussion of possible dead lock conditions related to the use +of these functions, see ``\ulink{Flow Control +Issues}{popen2-flow-control.html}'' +(section~\ref{popen2-flow-control}). + \begin{funcdesc}{popen2}{cmd\optional{, mode\optional{, bufsize}}} Executes \var{cmd} as a sub-process. Returns the file objects \code{(\var{child_stdin}, \var{child_stdout})}. diff --git a/Doc/lib/libpopen2.tex b/Doc/lib/libpopen2.tex index add0af130c..69181df4a8 100644 --- a/Doc/lib/libpopen2.tex +++ b/Doc/lib/libpopen2.tex @@ -107,3 +107,65 @@ This will always be \code{None} for \class{Popen4} instances. \begin{memberdesc}{pid} The process ID of the child process. \end{memberdesc} + + +\subsection{Flow Control Issues \label{popen2-flow-control}} + +Any time you are working with any form of inter-process communication, +control flow needs to be carefully thought out. This remains the case +with the file objects provided by this module (or the \refmodule{os} +module equivalents). + +% Example explanation and suggested work-arounds substantially stolen +% from Martin von Löwis: +% http://mail.python.org/pipermail/python-dev/2000-September/009460.html + +When reading output from a child process that writes a lot of data to +standard error while the parent is reading from the child's standard +out, a dead lock can occur. A similar situation can occur with other +combinations of reads and writes. The essential factors are that more +than \constant{_PC_PIPE_BUF} bites are being written by one process in +a blocking fashion, while the other process is reading from the other +process, also in a blocking fashion. + +There are several ways to deal with this situation. + +The simplest application change, in many cases, will be to follow this +model in the parent process: + +\begin{verbatim} +import popen2 + +r, w, e = popen2.popen3('python slave.py') +e.readlines() +r.readlines() +r.close() +e.close() +w.close() +\end{verbatim} + +with code like this in the child: + +\begin{verbatim} +import os +import sys + +# note that each of these print statements +# writes a single long string + +print >>sys.stderr, 400 * 'this is a test\n' +os.close(sys.stderr.fileno()) +print >>sys.stdout, 400 * 'this is another test\n' +\end{verbatim} + +In particular, note that \code{sys.stderr} must be closed after +writing all data, or \method{readlines()} won't return. Also note +that \function{os.close()} must be used, as \code{sys.stderr.close()} +won't close \code{stderr} (otherwise assigning to \code{sys.stderr} +will silently close it, so no further errors can be printed). + +Applications which need to support a more general approach should +integrate I/O over pipes with their \function{select()} loops, or use +separate threads to read each of the individual files provided by +whichever \function{popen*()} function or \class{Popen*} class was +used. |