[GlobalISel] Support vector-of-pointers in LLT

This fixes PR32471.

As comment 10 on that bug report highlights
(https://bugs.llvm.org//show_bug.cgi?id=32471#c10), there are quite a
few different defendable design tradeoffs that could be made, including
not representing pointers at all in LLT.

I decided to go for representing vector-of-pointer as a concept in LLT,
while keeping the size of the LLT type 64 bits (this is an increase from
48 bits before). My rationale for keeping pointers explicit is that on
some targets probably it's very handy to have the distinction between
pointer and non-pointer (e.g. 68K has a different register bank for
pointers IIRC). If we keep a scalar pointer, it probably is easiest to
also have a vector-of-pointers to keep LLT relatively conceptually clean
and orthogonal, while we don't have a very strong reason to break that
orthogonality.  Once we gain more experience on the use of LLT, we can
of course reconsider this direction.

Rejecting vector-of-pointer types in the IRTranslator is also an option
to avoid the crash reported in PR32471, but that is only a very
short-term solution; also needs quite a bit of code tweaks in places,
and is probably fragile. Therefore I didn't consider this the best
option.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@300664 91177308-0d34-0410-b5e6-96231b3b80d8

2 files changed, 6 insertions, 5 deletions

diff --git a/lib/CodeGen/GlobalISel/MachineIRBuilder.cpp b/lib/CodeGen/GlobalISel/MachineIRBuilder.cpp
index 8d1a263395a0..54ef7e5c5a1b 100644
--- a/lib/CodeGen/GlobalISel/MachineIRBuilder.cpp
+++ b/lib/CodeGen/GlobalISel/MachineIRBuilder.cpp
@@ -592,7 +592,7 @@ MachineInstrBuilder MachineIRBuilder::buildInsertVectorElement(unsigned Res,
   LLT EltTy = MRI->getType(Elt);
   LLT IdxTy = MRI->getType(Idx);
   assert(ResTy.isVector() && ValTy.isVector() && "invalid operand type");
-  assert(EltTy.isScalar() && IdxTy.isScalar() && "invalid operand type");
+  assert(IdxTy.isScalar() && "invalid operand type");
   assert(ResTy.getNumElements() == ValTy.getNumElements() && "type mismatch");
   assert(ResTy.getElementType() == EltTy && "type mismatch");
 #endif
@@ -612,7 +612,8 @@ MachineInstrBuilder MachineIRBuilder::buildExtractVectorElement(unsigned Res,
   LLT ValTy = MRI->getType(Val);
   LLT IdxTy = MRI->getType(Idx);
   assert(ValTy.isVector() && "invalid operand type");
-  assert(ResTy.isScalar() && IdxTy.isScalar() && "invalid operand type");
+  assert((ResTy.isScalar() || ResTy.isPointer()) && "invalid operand type");
+  assert(IdxTy.isScalar() && "invalid operand type");
   assert(ValTy.getElementType() == ResTy && "type mismatch");
 #endif
 
diff --git a/lib/CodeGen/LowLevelType.cpp b/lib/CodeGen/LowLevelType.cpp
index c4b9068fa905..1c682e72fa49 100644
--- a/lib/CodeGen/LowLevelType.cpp
+++ b/lib/CodeGen/LowLevelType.cpp
@@ -21,10 +21,10 @@ using namespace llvm;
 LLT llvm::getLLTForType(Type &Ty, const DataLayout &DL) {
   if (auto VTy = dyn_cast<VectorType>(&Ty)) {
     auto NumElements = VTy->getNumElements();
-    auto ScalarSizeInBits = VTy->getElementType()->getPrimitiveSizeInBits();
+    LLT ScalarTy = getLLTForType(*VTy->getElementType(), DL);
     if (NumElements == 1)
-      return LLT::scalar(ScalarSizeInBits);
-    return LLT::vector(NumElements, ScalarSizeInBits);
+      return ScalarTy;
+    return LLT::vector(NumElements, ScalarTy);
   } else if (auto PTy = dyn_cast<PointerType>(&Ty)) {
     return LLT::pointer(PTy->getAddressSpace(), DL.getTypeSizeInBits(&Ty));
   } else if (Ty.isSized()) {