pep484 - sql.selectable

the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.

current thinking is coming into:

1. there are _typing._XYZArgument objects that represent "what the
   user sent"
2. there's the roles, which represent a kind of "filter" for different
   kinds of objects.   These are mostly important as the argument
   we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
   construct uses as its internal representation of the thing.
   This is _typing._XYZElement.

but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not.   I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.

Other change introduced here, there was a deannotate=True
for the left/right of a sql.join().    All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.

CTEs now accommodate for text selects including recursive.

Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module.  This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.

References: #6810
Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0

1 files changed, 91 insertions, 0 deletions

diff --git a/lib/sqlalchemy/util/preloaded.py b/lib/sqlalchemy/util/preloaded.py
new file mode 100644
index 000000000..c861c83b3
--- /dev/null
+++ b/lib/sqlalchemy/util/preloaded.py
@@ -0,0 +1,91 @@
+# util/_preloaded.py
+# Copyright (C) 2005-2022 the SQLAlchemy authors and contributors
+# <see AUTHORS file>
+#
+# This module is part of SQLAlchemy and is released under
+# the MIT License: https://www.opensource.org/licenses/mit-license.php
+
+"""supplies the "preloaded" registry to resolve circular module imports at
+runtime.
+
+"""
+from __future__ import annotations
+
+import sys
+from types import ModuleType
+import typing
+from typing import Any
+from typing import Callable
+from typing import TYPE_CHECKING
+from typing import TypeVar
+
+_FN = TypeVar("_FN", bound=Callable[..., Any])
+
+if TYPE_CHECKING:
+    from sqlalchemy.engine import default as engine_default
+    from sqlalchemy.sql import dml as sql_dml
+    from sqlalchemy.sql import util as sql_util
+
+
+class _ModuleRegistry:
+    """Registry of modules to load in a package init file.
+
+    To avoid potential thread safety issues for imports that are deferred
+    in a function, like https://bugs.python.org/issue38884, these modules
+    are added to the system module cache by importing them after the packages
+    has finished initialization.
+
+    A global instance is provided under the name :attr:`.preloaded`. Use
+    the function :func:`.preload_module` to register modules to load and
+    :meth:`.import_prefix` to load all the modules that start with the
+    given path.
+
+    While the modules are loaded in the global module cache, it's advisable
+    to access them using :attr:`.preloaded` to ensure that it was actually
+    registered. Each registered module is added to the instance ``__dict__``
+    in the form `<package>_<module>`, omitting ``sqlalchemy`` from the package
+    name. Example: ``sqlalchemy.sql.util`` becomes ``preloaded.sql_util``.
+    """
+
+    def __init__(self, prefix="sqlalchemy."):
+        self.module_registry = set()
+        self.prefix = prefix
+
+    def preload_module(self, *deps: str) -> Callable[[_FN], _FN]:
+        """Adds the specified modules to the list to load.
+
+        This method can be used both as a normal function and as a decorator.
+        No change is performed to the decorated object.
+        """
+        self.module_registry.update(deps)
+        return lambda fn: fn
+
+    def import_prefix(self, path: str) -> None:
+        """Resolve all the modules in the registry that start with the
+        specified path.
+        """
+        for module in self.module_registry:
+            if self.prefix:
+                key = module.split(self.prefix)[-1].replace(".", "_")
+            else:
+                key = module
+            if (
+                not path or module.startswith(path)
+            ) and key not in self.__dict__:
+                __import__(module, globals(), locals())
+                self.__dict__[key] = globals()[key] = sys.modules[module]
+
+    if typing.TYPE_CHECKING:
+
+        def __getattr__(self, key: str) -> ModuleType:
+            ...
+
+
+_reg = _ModuleRegistry()
+preload_module = _reg.preload_module
+import_prefix = _reg.import_prefix
+
+if TYPE_CHECKING:
+
+    def __getattr__(key: str) -> ModuleType:
+        ...