Allow overriding completion in `auto_prepend_file`

Currently, it's possible to override `php -a`s completion
functionality to provide an alternative to the C implementation,
with `readline_completion_function()`.

However, that surprisingly gets overridden when called from
`auto_prepend_file`, because those scripts get run before the interactive shell
is started. I believe that not overriding it would be more consistent
with what happens when you override the completion function **after** the
interactive shell.

CLI is the only built-in API that uses this (See discussion in GH-5872).
I believe MINIT and RINIT will only run once when invoked with `php -a`.

Add documentation about the architecture of how php uses readline/libedit

Closes GH-5872

5 files changed, 39 insertions, 3 deletions

diff --git a/UPGRADING b/UPGRADING
index a71c8959c3..f38f149236 100644
--- a/UPGRADING
+++ b/UPGRADING
@@ -932,6 +932,12 @@ PHP 8.0 UPGRADE NOTES
 - PGSQL / PDO PGSQL:
   . The PGSQL and PDO PGSQL extensions now require at least libpq 9.1.
 
+- Readline:
+  . Calling readline_completion_function() before the interactive prompt starts
+    (e.g. in auto_prepend_file) will now override the default interactive prompt
+	completion function. Previously, readline_completion_function() only worked
+	when called after starting the interactive prompt.
+
 - SimpleXML:
   . SimpleXMLElement now implements RecursiveIterator and absorbed the
     functionality of SimpleXMLIterator. SimpleXMLIterator is an empty extension
diff --git a/ext/readline/README.md b/ext/readline/README.md
new file mode 100644
index 0000000000..5dec48e665
--- /dev/null
+++ b/ext/readline/README.md
@@ -0,0 +1,20 @@
+readline
+========
+
+Provides generic line editing, history, and tokenization functions.
+See https://www.php.net/manual/en/book.readline.php
+
+Implementation Details
+----------------------
+
+C variables starting with `rl_*` are declared by the readline library
+(or are macros referring to variables from the libedit library).
+See http://web.mit.edu/gnu/doc/html/rlman_2.html
+
+This should only be used in the CLI SAPI.
+Historically, the code lived in sapi/cli,
+but many distributions build readline as a shared extension.
+Therefore, that code was split into ext/readline so that this can dynamically
+be loaded. With other SAPIs, readline is/should be disabled.
+
+`readline_cli.c` implements most of the interactive shell(`php -a`).
diff --git a/ext/readline/readline.c b/ext/readline/readline.c
index d569efe820..ec1d1ff255 100644
--- a/ext/readline/readline.c
+++ b/ext/readline/readline.c
@@ -437,7 +437,7 @@ static void _readline_long_zval(zval *ret, long l)
 	ZVAL_LONG(ret, l);
 }
 
-static char **_readline_completion_cb(const char *text, int start, int end)
+char **php_readline_completion_cb(const char *text, int start, int end)
 {
 	zval params[3];
 	char **matches = NULL;
@@ -479,7 +479,8 @@ PHP_FUNCTION(readline_completion_function)
 	zval_ptr_dtor(&_readline_completion);
 	ZVAL_COPY(&_readline_completion, &fci.function_name);
 
-	rl_attempted_completion_function = _readline_completion_cb;
+	/* NOTE: The rl_attempted_completion_function variable (and others) are part of the readline library, not php */
+	rl_attempted_completion_function = php_readline_completion_cb;
 	if (rl_attempted_completion_function == NULL) {
 		RETURN_FALSE;
 	}
diff --git a/ext/readline/readline_cli.c b/ext/readline/readline_cli.c
index c1ec134fff..a463a89db4 100644
--- a/ext/readline/readline_cli.c
+++ b/ext/readline/readline_cli.c
@@ -603,7 +603,14 @@ static int readline_shell_run(void) /* {{{ */
 #else
 	spprintf(&history_file, MAX_PATH, "%s/.php_history", getenv("USERPROFILE"));
 #endif
-	rl_attempted_completion_function = cli_code_completion;
+	/* Install the default completion function for 'php -a'.
+	 *
+	 * But if readline_completion_function() was called by PHP code prior to the shell starting
+	 * (e.g. with 'php -d auto_prepend_file=prepend.php -a'),
+	 * then use that instead of PHP's default. */
+	if (rl_attempted_completion_function != php_readline_completion_cb) {
+		rl_attempted_completion_function = cli_code_completion;
+	}
 #ifndef PHP_WIN32
 	rl_special_prefixes = "$";
 #endif
diff --git a/ext/readline/readline_cli.h b/ext/readline/readline_cli.h
index 81e73d1e4d..7afadded07 100644
--- a/ext/readline/readline_cli.h
+++ b/ext/readline/readline_cli.h
@@ -34,4 +34,6 @@ extern PHP_MINIT_FUNCTION(cli_readline);
 extern PHP_MSHUTDOWN_FUNCTION(cli_readline);
 extern PHP_MINFO_FUNCTION(cli_readline);
 
+char **php_readline_completion_cb(const char *text, int start, int end);
+
 ZEND_EXTERN_MODULE_GLOBALS(cli_readline)