Tag inference work.

This does three major things:
* Enforce the invariant that all strict fields must contain tagged
pointers.
* Try to predict the tag on bindings in order to omit tag checks.
* Allows functions to pass arguments unlifted (call-by-value).

The former is "simply" achieved by wrapping any constructor allocations with
a case which will evaluate the respective strict bindings.

The prediction is done by a new data flow analysis based on the STG
representation of a program. This also helps us to avoid generating
redudant cases for the above invariant.

StrictWorkers are created by W/W directly and SpecConstr indirectly.
See the Note [Strict Worker Ids]

Other minor changes:

* Add StgUtil module containing a few functions needed by, but
  not specific to the tag analysis.

-------------------------
Metric Decrease:
	T12545
	T18698b
	T18140
	T18923
        LargeRecord
Metric Increase:
        LargeRecord
	ManyAlternatives
	ManyConstructors
	T10421
	T12425
	T12707
	T13035
	T13056
	T13253
	T13253-spj
	T13379
	T15164
	T18282
	T18304
	T18698a
	T1969
	T20049
	T3294
	T4801
	T5321FD
	T5321Fun
	T783
	T9233
	T9675
	T9961
	T19695
	WWRec
-------------------------

3 files changed, 73 insertions, 0 deletions

diff --git a/docs/users_guide/9.4.1-notes.rst b/docs/users_guide/9.4.1-notes.rst
index 75e7268cc1..c29d00e9d7 100644
--- a/docs/users_guide/9.4.1-notes.rst
+++ b/docs/users_guide/9.4.1-notes.rst
@@ -65,6 +65,30 @@ Compiler
   that is used boxed in a hot path). Do post a bug report with your example!
   Then wrap the uses of the parameter in ``GHC.Exts.lazy`` for a short-term fix.
 
+- Tag inference has been implemented.
+
+  It's a new backend optimization pass aimed at avoiding
+  redundant evaluatedness checks. The basic pass is always enabled and not optional.
+  When using :ghc-flag:`-fworker-wrapper-cbv` it additionally will generate workers for functions
+  with strict arguments, pushing the evaluation+tagging of the arguments into the wrapper
+  and allowing the worker to simply assume all arguments are fully evaluated and properly
+  tagged. Usually the wrapper will then inline, and if the argument is known to be properly
+  tagged at the call site the wrapper will become a no-op. Giving us a more efficient
+  worker without adding any overhead. If the argument *isn't* known to be evaluated we
+  perform the same amount of work, but do it at call sites instead of inside the called
+  function.
+
+  In general :ghc-flag:`-fworker-wrapper-cbv` is very beneficial and can be safely enabled.
+  However sadly there are two exceptions. It can break rules for code which made assumptions about
+  which functions get a W/W split which now no longer hold.
+  See #20364 for the details. For this reason it isn't enabled by default.
+  For code which has the proper ``INLINABLE`` (:ref:`inlinable-pragma`) and ``INLINE`` (:ref:`inline-pragma`)
+  or that doesn't define any rule-relevant functions this shouldn't happen. The longterm fix here is to
+  apply the proper pragmas.
+  There is also a known issue where a function taking multiple unlifted arguments can cause excessive
+  spilling (#20334). This seems to be an edge case. But if you think you are hitting this case please
+  comment on the ticket so that we can prioritize it accordingly.
+
 - Support for Sun SPARC architecture has been dropped (:ghc-ticket:`16883`).
 
 - A fix for GHC's handling of the XDG Base Directory Specification
diff --git a/docs/users_guide/debugging.rst b/docs/users_guide/debugging.rst
index 30f31a64ef..6802fa71fb 100644
--- a/docs/users_guide/debugging.rst
+++ b/docs/users_guide/debugging.rst
@@ -457,6 +457,19 @@ These flags dump various phases of GHC's STG pipeline.
 
     Show the output of the unarise pass.
 
+.. ghc-flag:: -ddump-stg-cg
+    :shortdesc: Show output after Stg2Stg
+    :type: dynamic
+
+    Show the output of the STG after Stg2Stg. This is the result after
+    applying the Stg2Stg optimization passes.
+
+.. ghc-flag:: -ddump-stg-tags
+    :shortdesc: Show output of the tag inference pass.
+    :type: dynamic
+
+    Show the output of the tag inference pass.
+
 .. ghc-flag:: -ddump-stg-final
     :shortdesc: Show output of last STG pass.
     :type: dynamic
@@ -1060,3 +1073,13 @@ Other
     is useful when debugging because it gives smaller modules and dumps, but the
     compiler will panic if you try to use Typeable instances of things that you
     built with this flag.
+
+.. ghc-flag:: -dtag-inference-checks
+    :shortdesc: Affirm tag inference results are correct at runtime.
+    :type: dynamic
+
+    When tag inference tells as a specific value is supposed to be tagged then
+    generate code to check this at runtime. If the check fails the program will
+    be terminated. This helps narrowing down if an issue is due to tag inference
+    if things go wrong. Which would otherwise be quite difficult.
+
diff --git a/docs/users_guide/using-optimisation.rst b/docs/users_guide/using-optimisation.rst
index b368544696..390b179f33 100644
--- a/docs/users_guide/using-optimisation.rst
+++ b/docs/users_guide/using-optimisation.rst
@@ -1611,6 +1611,32 @@ by saying ``-fno-wombat``.
     while demand analysis is disabled (by :ghc-flag:`-fno-strictness`)
     has no effect.
 
+.. ghc-flag:: -fworker-wrapper-cbv
+    :shortdesc: Enable w/w splits for wrappers whos sole purpose is evaluating arguments.
+    :type: dynamic
+    :category: optimization
+
+    Disabling this flag prevents a W/W split if the only benefit would be call-by-value
+    for some arguments.
+
+    Otherwise this exploits strictness information by passing strict value arguments
+    call-by-value to the functions worker. Even for functions who would
+    otherwise not get a worker.
+
+    This avoids (potentially repeated) checks for evaluatedness of arguments in
+    the rhs of the worker by pushing this check to the call site.
+    If the argument is statically visible to be a value at the call site the
+    overhead for the check disappears completely.
+
+    This can cause slight codesize increases. It will also cause many more functions
+    to get a worker/wrapper split which can play badly with rules (see Ticket #20364)
+    which is why it's currently disabled by default.
+    In particular if you depend on rules firing on functions marked as NOINLINE without
+    marking use sites of these functions as INLINE or INLINEABLE then things will break
+    unless this flag is disabled.
+
+    While WorkerWrapper is disabled this has no effect.
+
 .. ghc-flag:: -fbinary-blob-threshold=⟨n⟩
     :shortdesc: *default: 500K.* Tweak assembly generator for binary blobs.
     :type: dynamic