libtool: fix nm test for MSYS/MinGW

The check for the -B option of nm does not work as intended on MSYS/MinGW.
MSYS converts /dev/null to the DOW/Windows "equivanent" special file NUL,
but the MinGW nm treats this file as any empty file. This means that
you might end up with some fallback nm instead of the desired nm. This
is not normally a problem, but if one nm is built without lto support, it
starts to matter.

Fixes sr #108558, reported by LRN.

* m4/libtool.m4 (LT_PATH_NM) [MSYS]: Use a non-existant file instead of
/dev/null when checking if nm supports -B.

Signed-off-by: Peter Rosin <peda@lysator.liu.se>

1 files changed, 7 insertions, 2 deletions

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index 0454030a..320d8b36 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -3509,8 +3509,13 @@ else
 	# Adding the 'sed 1q' prevents false positives on HP-UX, which says:
 	#   nm: unknown option "B" ignored
 	# Tru64's nm complains that /dev/null is an invalid object file
-	case `"$tmp_nm" -B /dev/null 2>&1 | sed '1q'` in
-	*/dev/null* | *'Invalid file or object type'*)
+	# MSYS converts /dev/null to NUL, MinGW nm treats NUL as empty
+	case $build_os in
+	mingw*) lt_bad_file=conftest.nm/nofile ;;
+	*) lt_bad_file=/dev/null ;;
+	esac
+	case `"$tmp_nm" -B $lt_bad_file 2>&1 | sed '1q'` in
+	*$lt_bad_file* | *'Invalid file or object type'*)
 	  lt_cv_path_NM="$tmp_nm -B"
 	  break 2
 	  ;;