rearchitect reflection for batched performance

Rearchitected the schema reflection API to allow some dialects to make use
of high performing batch queries to reflect the schemas of many tables at
once using much fewer queries. The new performance features are targeted
first at the PostgreSQL and Oracle backends, and may be applied to any
dialect that makes use of SELECT queries against system catalog tables to
reflect tables (currently this omits the MySQL and SQLite dialects which
instead make use of parsing the "CREATE TABLE" statement, however these
dialects do not have a pre-existing performance issue with reflection. MS
SQL Server is still a TODO).

The new API is backwards compatible with the previous system, and should
require no changes to third party dialects to retain compatibility;
third party dialects can also opt into the new system by implementing
batched queries for schema reflection.

Along with this change is an updated reflection API that is fully
:pep:`484` typed, features many new methods and some changes.

Fixes: #4379
Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08

1 files changed, 9 insertions, 7 deletions

diff --git a/lib/sqlalchemy/testing/plugin/pytestplugin.py b/lib/sqlalchemy/testing/plugin/pytestplugin.py
index fa7d2ca19..cea07b305 100644
--- a/lib/sqlalchemy/testing/plugin/pytestplugin.py
+++ b/lib/sqlalchemy/testing/plugin/pytestplugin.py
@@ -741,13 +741,18 @@ class PytestFixtureFunctions(plugin_base.FixtureFunctions):
                 fn._sa_parametrize.append((argnames, pytest_params))
                 return fn
             else:
+                _fn_argnames = inspect.getfullargspec(fn).args[1:]
                 if argnames is None:
-                    _argnames = inspect.getfullargspec(fn).args[1:]
+                    _argnames = _fn_argnames
                 else:
                     _argnames = re.split(r", *", argnames)
 
                 if has_exclusions:
-                    _argnames += ["_exclusions"]
+                    existing_exl = sum(
+                        1 for n in _fn_argnames if n.startswith("_exclusions")
+                    )
+                    current_exclusion_name = f"_exclusions_{existing_exl}"
+                    _argnames += [current_exclusion_name]
 
                     @_pytest_fn_decorator
                     def check_exclusions(fn, *args, **kw):
@@ -755,13 +760,10 @@ class PytestFixtureFunctions(plugin_base.FixtureFunctions):
                         if _exclusions:
                             exlu = exclusions.compound().add(*_exclusions)
                             fn = exlu(fn)
-                        return fn(*args[0:-1], **kw)
-
-                    def process_metadata(spec):
-                        spec.args.append("_exclusions")
+                        return fn(*args[:-1], **kw)
 
                     fn = check_exclusions(
-                        fn, add_positional_parameters=("_exclusions",)
+                        fn, add_positional_parameters=(current_exclusion_name,)
                     )
 
                 return pytest.mark.parametrize(_argnames, pytest_params)(fn)