glob: resolve DT_UNKNOWN via is_dir

The DT_* values returned by getdents (readdir) are only hints and
not required.  In fact, some Linux filesystems return DT_UNKNOWN
for most entries, regardless of actual type.  This causes make
to mis-match patterns with a trailing slash (via GLOB_ONLYDIR)
(see make's functions/wildcard test case).  Thus, this patch
detects that case and uses is_dir() to make the type known enough
for proper operation.

Performance in non-DT_UNKNOWN cases is not affected.

The lack of DT_* is a well known issue on older XFS installations
(for example, RHEL 7 and 8, Fedora 28) but can be recreated by
creating an XFS filesystem with flags that mimic older behavior:

$ fallocate -l 10G /xfs.fs
$ mkfs.xfs -n ftype=0 -m crc=0 -f /xfs.fs
$ mkdir /xfs
$ mount -o loop /xfs.fs /xfs

1 files changed, 27 insertions, 1 deletions

diff --git a/lib/glob.c b/lib/glob.c
index f8d8a306f2..0da46ac138 100644
--- a/lib/glob.c
+++ b/lib/glob.c
@@ -1381,7 +1381,33 @@ glob_in_dir (const char *pattern, const char *directory, int flags,
               if (flags & GLOB_ONLYDIR)
                 switch (readdir_result_type (d))
                   {
-                  case DT_DIR: case DT_LNK: case DT_UNKNOWN: break;
+                  case DT_DIR: case DT_LNK: break;
+                  case DT_UNKNOWN:
+                    {
+                      /* The filesystem was too lazy to give us a hint,
+                         so we have to do it the hard way.  */
+                      char *fullpath, *p;
+                      bool isdir;
+                      int need = strlen (directory) + strlen (d.name) + 2;
+                      int use_alloca = glob_use_alloca (alloca_used, need);
+                      if (use_alloca)
+                        fullpath = alloca_account (need, alloca_used);
+                      else
+                        {
+                          fullpath = malloc (need);
+                          if (fullpath == NULL)
+                            goto memory_error;
+                        }
+                      p = stpcpy (fullpath, directory);
+                      *p++ = '/';
+                      strcpy (p, d.name);
+                      isdir = is_dir (fullpath, flags, pglob);
+                      if (!use_alloca)
+                        free (fullpath);
+                      if (isdir)
+                        break;
+                      continue;
+                    }
                   default: continue;
                   }