Remove screen capture dependency on DirectX debug layer

When creating a Direct3D 11 device, it is possible to enable additional
debug logging from the Direct3D subsystem by passing the
D3D11_CREATE_DEVICE_DEBUG flag to the D3D11CreateDevice function. This
is useful for debugging, but requires the D3D11SDKLayers.dll debug layer
to be installed on the target machine. If the debug layer is not
installed, calls to D3D11CreateDevice fails with the
DXGI_ERROR_SDK_COMPONENT_MISSING status code.

The debug layer is not installed by default on Windows 10 or Windows 11,
and we can therefore not expect that this layer is available on any
Windows computer. We also see that some QtMultimedia tests fail on CI
nodes because the debug layer is not present.

This change fixes the problem by removing the D3D11_CREATE_DEVICE_DEBUG
flag from calls to D3D11CreateDevice. This gives less diagnostics for
developers, but full debug logging can still be enabled through the
DirectX control panel, which can be used to forcefully enable the debug
layer even if the flag is not present when creating the device.

An alternative solution would be to first try to create the device with
the debug layer enabled by passing the D3D11_CREATE_DEVICE_DEBUG flag,
and then retry without this flag if creating the device failed. The
drawback of this approach is that the performance of the screen capture
feature may depend on if the debug layer is installed or not.

Pick-to: 6.5
Change-Id: Icad2965b61de953e3f93c8aeb902f37e5b31c10a
Reviewed-by: Lars Knoll <lars@knoll.priv.no>

2 files changed, 2 insertions, 2 deletions

diff --git a/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_dxgi.cpp b/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_dxgi.cpp
index 614ee4006..3852c8cad 100644
--- a/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_dxgi.cpp
+++ b/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_dxgi.cpp
@@ -282,7 +282,7 @@ static QMaybe<QWindowsIUPointer<ID3D11Device>> createD3D11Device(IDXGIAdapter1 *
 {
     QWindowsIUPointer<ID3D11Device> d3d11dev;
     HRESULT hr = D3D11CreateDevice(adapter, D3D_DRIVER_TYPE_UNKNOWN, nullptr,
-                                   D3D11_CREATE_DEVICE_DEBUG , nullptr, 0, D3D11_SDK_VERSION,
+                                   0, nullptr, 0, D3D11_SDK_VERSION,
                                    d3d11dev.address(), nullptr, nullptr);
     if (SUCCEEDED(hr))
         return d3d11dev;
diff --git a/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_uwp.cpp b/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_uwp.cpp
index d1b7a59d5..7da4a5934 100644
--- a/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_uwp.cpp
+++ b/src/plugins/multimedia/ffmpeg/qffmpegscreencapture_uwp.cpp
@@ -236,7 +236,7 @@ createCaptureFramePool(IDXGIAdapter1 *adapter, const winrt::GraphicsCaptureItem
 {
     winrt::com_ptr<ID3D11Device> d3d11dev;
     HRESULT hr =
-            D3D11CreateDevice(adapter, D3D_DRIVER_TYPE_UNKNOWN, nullptr, D3D11_CREATE_DEVICE_DEBUG,
+            D3D11CreateDevice(adapter, D3D_DRIVER_TYPE_UNKNOWN, nullptr, 0,
                               nullptr, 0, D3D11_SDK_VERSION, d3d11dev.put(), nullptr, nullptr);
     if (FAILED(hr))
         return { "Failed to create ID3D11Device device" + errorString(hr) };