Make validDerivPred ignore non-visible arguments to a class type constructor

Summary:
GHC choked when trying to derive the following:

```
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE PolyKinds #-}
module Example where

class Category (cat :: k -> k -> *) where
  catId   :: cat a a
  catComp :: cat b c -> cat a b -> cat a c

newtype T (c :: * -> * -> *) a b = MkT (c a b) deriving Category
```

Unlike in #8865, where we were deriving `Category` for a concrete type like
`Either`, in the above example we are attempting to derive an instance of the
form:

```
instance Category * c => Category (T * c) where ...
```

(using `-fprint-explicit-kinds` syntax). But `validDerivPred` is checking if
`sizePred (Category * c)` equals the number of free type variables in
`Category * c`. But note that `sizePred` counts both type variables //and//
type constructors, and `*` is a type constructor! So `validDerivPred`
erroneously rejects the above instance.

The fix is to make `validDerivPred` ignore non-visible arguments to the class
type constructor (e.g., ignore `*` is `Category * c`) by using
`filterOutInvisibleTypes`.

Fixes #11833.

Test Plan: ./validate

Reviewers: goldfire, hvr, simonpj, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2112

GHC Trac Issues: #11833

2 files changed, 10 insertions, 0 deletions

diff --git a/testsuite/tests/deriving/should_compile/T11833.hs b/testsuite/tests/deriving/should_compile/T11833.hs
new file mode 100644
index 0000000000..75d2a2d255
--- /dev/null
+++ b/testsuite/tests/deriving/should_compile/T11833.hs
@@ -0,0 +1,9 @@
+{-# LANGUAGE GeneralizedNewtypeDeriving #-}
+{-# LANGUAGE PolyKinds #-}
+module T11833 where
+
+class Category (cat :: k -> k -> *) where
+  catId   :: cat a a
+  catComp :: cat b c -> cat a b -> cat a c
+
+newtype T (c :: * -> * -> *) a b = MkT (c a b) deriving Category
diff --git a/testsuite/tests/deriving/should_compile/all.T b/testsuite/tests/deriving/should_compile/all.T
index cfbb977abe..07242ec694 100644
--- a/testsuite/tests/deriving/should_compile/all.T
+++ b/testsuite/tests/deriving/should_compile/all.T
@@ -69,3 +69,4 @@ test('T11357', normal, compile, [''])
 test('T11732a', normal, compile, [''])
 test('T11732b', normal, compile, [''])
 test('T11732c', normal, compile, [''])
+test('T11833', normal, compile, [''])