Windows debugger cannot launch the target when used in a VM without any network interface

[xusheng6]

The user tries to debug a binary in a Hype-V VM. The VM has no internet connection. The error message is:

ConnectToDebugServerInternal timeout
Failed to connect process server

On the other hand, things work perfectly when the VM is connected to the Internet.

I suspect this is related to the synbol server handling in DbgEng, but it is still weird because this failure is so early. Previously, I have observed a hang when one uses the debugger on a box that appears to have network, but is not actually connected to the Internet

[xusheng6]

I can also reproduce this on a Virtualbox VM with no network interface configured

[xusheng6]

Also this has nothing to do with Internet, I tested and found that, as long as there is a network interface, even if it is not attached to anything, the debugging will work. A temporary workaround for this issue is to create a network interface for the VM, but do not attach it to any actual network

[xusheng6]

This is more complicated than I have expected. It seems a bug in the DbgEng DLLs. I can reproduce this with the dbgsrv.exe that comes with WinDbg as well. However, this is really not expected -- things should just work in the same way.

I checked and WinDbg is doing the debugging slightly different than our approach. It is running an EngHost.exe (which seems to be an engine host), and then use DebugConnect (https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/dbgeng/nf-dbgeng-debugconnect) to connect to it. DebugConnect does not have the same bug

[xusheng6]

However, using DebugConnect seems to require that the access to the engine to be made from the same thread, which our current implementation is violating.

[LarsAndre]

Context

I ran into the same symptoms on a Windows 10 FLARE-VM using Binary Ninja 5.2.8722 Stable with the DBGENG adapter:

ConnectToDebugServerInternal timeout
Failed to connect process server

In my case the VM did have a working network interface, so I dug into the DbgEng/DbgSrv transport a bit further.

I was able to isolate the problem to the npipe:server=localhost path:

• dbgsrv.exe launched successfully.

• DbgSrv over TCP worked:

  cdb.exe -premote tcp:server=127.0.0.1,port=50080 ...
  

• A local named pipe also worked when using:

  npipe:server=.,pipe=BNTEST
  

• The equivalent connection using:

  npipe:server=localhost,pipe=BNTEST
  

  failed.

• net use \\localhost\IPC$ returned:

  System error 67 has occurred.
  The network name cannot be found.
  

• The SMB client event log (Microsoft-Windows-SMBClient/Connectivity, Event ID 30800) reported:

  The server name cannot be resolved.
  Error: The requested interface is not supported.
  

The root cause turned out to be that the Windows DNS Client (Dnscache) service was disabled:

Get-Service Dnscache

# Status    : Stopped
# StartType : Disabled

Workaround / Fix

Re-enabling it and rebooting fixed the SMB/UNC path:

Set-ItemProperty `
  'HKLM:\SYSTEM\CurrentControlSet\Services\Dnscache' `
  -Name Start `
  -Type DWord `
  -Value 2

Restart-Computer

After reboot:

Dnscache  DNS Client  Running  Automatic

and:

net use \\localhost\IPC$
The command completed successfully.

Most importantly, the DbgEng NPIPE connection using server=localhost then worked and successfully reached the 0:000> debugger prompt.

So for hardened/reversing VMs such as FLARE-VM, where Dnscache may have been intentionally disabled, enabling the DNS Client service appears to be a viable workaround for this DbgEng ConnectProcessServer timeout.

This may not explain the original zero-network-interface case entirely, but it produces the same failure in the same DbgEng NPIPE transport and may be useful for others hitting this issue.