Skip to content

About

Real TCP/IP networking for MSX software running in openMSX via UNAPI

Resources

Stars

13 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

51 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

openMSXnet

First of all, many many thanks to @mcolom, because the idea of this project born thanks to him! Thanks a lot Tiburoncio!

πŸ‘‘ Hall of Bug Hunters πŸ‘‘

πŸ“ Su ExcelentΓ­sima Frutajestad ExcelentΓ­simo FranSX πŸ“

For discovering β€” and persistently re-reporting β€” the Windows DLL-hell bug that broke every fresh install through v0.9.4. The early "ship .exe + a pile of DLLs" approaches kept leaking transitive deps, hitting antivirus false positives, and failing extractions on clean machines. His refusal to let it slide drove the move to upstream openMSX's own recipe: cross-compile staticbindist from Ubuntu so SDL2, Tcl, freetype, glew and friends end up linked directly into openmsx.exe. The Windows ZIP is now a single self-contained binary β€” no DLL hell forever. πŸ™

Submitted bug reports, accepted in lieu of tribute.

build

An MSX TCP/IP UNAPI implementation that exposes the host operating system's network stack to MSX software running under the openMSX emulator.

Looking to install and run, not build from source? See docs/USAGE.md for step-by-step instructions on Windows, Linux and macOS, plus where to download pre-built binaries.

The implementation consists of two cooperating components:

  • A C++ device extension compiled into openMSX, which holds real BSD sockets and a background polling thread on the host side.
  • A Z80 TSR (UNAPINET.COM) that installs in a memory mapper segment under Nextor / MSX-DOS 2, hooks EXTBIO, and exposes a standards-compliant TCP/IP UNAPI 1.1 interface to client programs. Function calls are marshalled through a pair of I/O ports to the C++ device.

Programs that target the UNAPI TCP/IP specification (such as hget, telnet, sntp) run unmodified.

Architecture

MSX program (hget, telnet, sntp, ...)
        β”‚
        β”‚  UNAPI TCP/IP function calls
        β–Ό
UNAPINET.COM    (Z80 TSR in mapper segment)
        β”‚
        β”‚  I/O ports 28h / 29h
        β–Ό
UnapiNet device (C++, inside openMSX)
        β”‚
        β”‚  BSD sockets, threads
        β–Ό
   Host OS network stack

Implementation status

Implemented and tested

Feature Notes
UNAPI discovery via EXTBIO (DE=2222h, ARG="TCP/IP") Programs locate the implementation through Nextor's RAM helper
DNS resolution Asynchronous, backed by host getaddrinfo()
TCP active connections Up to 4 simultaneous, non-blocking connect()
TCP send / receive 64 KiB receive buffer per connection
TCP passive mode (listen) bind() + listen() with non-blocking accept() in the receiver loop
UDP datagrams Up to 4 simultaneous, automatic fallback when bind to a privileged port (<1024) is denied
ICMP echo (ping) Uses Windows IcmpSendEcho API; no admin required
TCPIP_WAIT (fn 29) EI/HALT idiom to release a 50/60 Hz tick
TCPIP_GET_IPINFO Local IP discovered via UDP socket trick to 8.8.8.8

Verified against:

  • telnet (BBS sessions, including ANSI screens with sotanomsxbbs.org)
  • hget (HTTP/1.1 chunked transfer, tested with example.com)
  • sntp (clock synchronisation with pool.ntp.org)
  • tftp (file download from a local TFTP server)
  • ping (ICMP echo request/reply against 8.8.8.8)
  • MSXon clients (multiplayer game clients built with MSXgl's unapi_tcp library β€” tetris.com, lobby.com, etc.)

Not implemented

UNAPI function Reason
TCPIP_TCP_DISCARD (fn 19) No client encountered relies on it.
Raw IP (fn 20-24) Not advertised.
TCPIP_CONFIG_* (fn 25-28) Network configuration is delegated to the host OS.

Known limitations

  • Save states do not preserve socket state; open connections are lost on load.
  • TLS is not supported. Plain HTTP/HTTPS clients work for http:// only.

Repository layout

unapinet/
  UnapiNet.hh           C++ device class declaration
  UnapiNet.cc           C++ device implementation
  unapinet.xml          openMSX device descriptor (claims I/O ports 28h-29h, same range as DenYoNet)

msx/
  unapinet.asm          Z80 TSR (Nestor80 syntax)
  test_unapi.asm        Direct I/O regression test (does not exercise the dispatcher)
  test_hget.asm         End-to-end test mirroring hget's call sequence

Building

The recommended way to obtain a build is not to compile it yourself β€” just download the pre-built ZIP for your platform from Releases. The CI workflow at .github/workflows/build.yml produces those releases for every tagged commit (v*).

If you do want to build from source, the ci/setup-openmsx.sh script automates everything documented below β€” clones openMSX, drops in the extension sources, registers the device class, applies platform patches. After it finishes, you run make in openMSX/.

Prerequisites

  • An openMSX source tree (cloned by the setup script, currently pinned to RELEASE_21_0).
  • A C++17-capable compiler. On Windows this means the MSYS2 MINGW64 toolchain.
  • Nestor80 Z80 assembler, for building UNAPINET.COM.
  • A Nextor 2.1.x ROM at runtime (RAM-helper that the TSR depends on); not needed at build time.

Building from source (canonical, all platforms)

git clone https://github.com/antxiko/openMSXnet.git
cd openMSXnet
bash ci/setup-openmsx.sh                # clones openMSX + applies patches
cd openMSX && make -j$(nproc)

Linux / macOS install their build-time deps via apt / brew:

# Debian / Ubuntu
sudo apt install build-essential pkg-config python3 \
    libsdl2-dev libsdl2-ttf-dev libfreetype-dev libglew-dev \
    libpng-dev libxml2-dev tcl-dev libogg-dev libvorbis-dev \
    libtheora-dev libasound2-dev libjack-jackd2-dev

# macOS (Homebrew)
brew install sdl2 sdl2_ttf freetype glew libpng libxml2 \
    tcl-tk libogg libvorbis theora pkg-config

The resulting binary is dynamically linked against these. End users on the same OS install matching runtime packages and run it directly.

Windows distribution bundle

openmsx.exe on Windows is produced by cross-compiling openMSX's staticbindist target from Ubuntu 24.04 β€” the same recipe the upstream openMSX project uses for its own Windows releases. The CI workflow installs the Ubuntu mingw-w64 cross-toolchain (gcc 13, which compiles the bundled pkg-config + glib sources cleanly, unlike MSYS2's bleeding-edge gcc 16 which trips over C23 keyword changes), configures it for POSIX threads, and runs:

make -j$(nproc) staticbindist \
    OPENMSX_TARGET_CPU=x86_64 \
    OPENMSX_TARGET_OS=mingw-w64

The build downloads SDL2, Tcl, libxml2, freetype, glew, libpng, ogg, vorbis, theora and zlib source tarballs, compiles them as static archives, then statically links every one of them β€” plus libwinpthread and the MinGW C++ runtime β€” into openmsx.exe. The resulting binary's only imports are Windows system DLLs (KERNEL32, USER32, OPENGL32, WS2_32, IPHLPAPI, msvcrt, …). The release ZIP contains the .exe plus share/ and nothing else β€” no MinGW DLLs to bundle, no antivirus landmines, no extraction edge cases.

Patches applied by ci/setup-openmsx.sh

For reference (the script handles all of these automatically):

  • build/main.mk β€” gate -ldl to Linux only (macOS and mingw don't have it).
  • build/msysutils.py β€” Python 3 print() + bytes/str decoding.
  • build/platform-mingw-w64.mk β€” replace -static with selective -static-libgcc -static-libstdc++ (full -static previously conflicted with the dynamic Tcl path; superseded by staticbindist but kept for legacy make builds).
  • build/platform-mingw32.mk β€” add -lws2_32 -liphlpapi for Winsock 2 and IcmpSendEcho.
  • src/DeviceFactory.cc β€” register the UnapiNet device class.
  • Copies unapinet/UnapiNet.{hh,cc} into src/unapinet/ and unapinet/unapinet.xml into share/extensions/.

MSX TSR

N80 msx/unapinet.asm msx/unapinet.com --direct-output-write

The output binary is roughly 2 KiB. Copy it onto a Nextor disk image alongside NEXTOR.SYS and COMMAND2.COM β€” for example with mcopy from the mtools package:

mcopy -o -i path/to/nextor_hd.dsk@@512 msx/unapinet.com ::

Running

openmsx -machine Philips_NMS_8250 \
        -ext Nextor213_IDE \
        -ext unapinet

At the Nextor prompt, run UNAPINET.COM once to install the TSR. From that point on, any UNAPI TCP/IP client will discover the implementation through EXTBIO and route its calls through the bridge.

Bridge protocol

All communication between the Z80 TSR and the C++ device flows through two I/O ports.

Choice of port range. UnapiNet uses ports 28h-29h, the same range as the DenYoNet Ethernet card. Both are UNAPI Ethernet bridges and are never loaded together. Earlier iterations used 7Eh-7Fh (clashed with MoonSound β€” Yamaha OPL4) and C0h-C1h (same range as the Obsonet Ethernet card but also dynamically claimed by the DalSoRi R2 OPL4 cart when its software-controlled register window is enabled).

Port Write Read
28h Command byte; triggers execution Status (00=OK, 01=ERR, 02=DATA available)
29h Parameter byte; appended to a per-command buffer Result byte; auto-advances on each read

The MSX appends parameter bytes to port 29h, then writes a command byte to port 28h. The device processes the command synchronously (DNS is serviced asynchronously through a worker thread but its dispatch is non-blocking), populates the result buffer, and signals DATA available on port 28h. The MSX then reads the result bytes sequentially from port 29h.

Writing to port 29h while a previous result is still pending discards the stale result. This avoids deadlocks if the MSX side fails to drain all bytes from a previous call.

Bridge command set

Cmd Mnemonic Parameters (port 29h) Result (port 29h)
00h PING - 1 byte: ABh
01h DNS_QUERY hostname + 00h 1 byte status [+ 4 bytes IP]
02h DNS_STATUS - 1 byte status [+ 4 bytes IP]
03h TCP_OPEN IP[4] + port[2 LE] 1 byte handle (0 = error)
04h TCP_SEND handle + len[2 LE] + data 1 byte status
05h TCP_RECV handle + maxlen[2 LE] len[2 LE] + data
06h TCP_CLOSE handle 1 byte status
07h TCP_STATE handle state + avail[2 LE] + close_reason
08h TCP_ABORT handle 1 byte status
09h UDP_OPEN local_port[2 LE] 1 byte handle (0 = error)
0Ah UDP_CLOSE handle 1 byte status
0Bh UDP_STATE handle size[2 LE] of next datagram
0Ch UDP_SEND handle + dest_IP[4] + port[2 LE] + len[2 LE] + data 1 byte status
0Dh GET_LOCALIP - 4 bytes IP (network order)
0Eh NET_STATE - 1 byte (2 = open)
0Fh UDP_RECV handle + maxlen[2 LE] src_IP[4] + src_port[2 LE] + len[2 LE] + data
10h QUERY_CAP - 2 bytes: cap0, cap1
11h ICMP_SEND IP[4] + TTL[1] + ID[2 LE] + SEQ[2 LE] + len[2 LE] 1 byte status
12h ICMP_RECV - has_data[1] + [IP[4]+TTL[1]+ID[2]+SEQ[2]+len[2]]

UNAPI dispatch table

Fn UNAPI name Backed by
0 UNAPI_GET_INFO local
1 TCPIP_GET_CAPAB local (hardcoded)
2 TCPIP_GET_IPINFO GET_LOCALIP (idx 1)
3 TCPIP_NET_STATE NET_STATE
4 TCPIP_SEND_ECHO ICMP_SEND
5 TCPIP_RCV_ECHO ICMP_RECV
6 TCPIP_DNS_Q DNS_QUERY
7 TCPIP_DNS_S DNS_STATUS
8 TCPIP_UDP_OPEN UDP_OPEN
9 TCPIP_UDP_CLOSE UDP_CLOSE
10 TCPIP_UDP_STATE UDP_STATE
11 TCPIP_UDP_SEND UDP_SEND
12 TCPIP_UDP_RCV UDP_RECV
13 TCPIP_TCP_OPEN TCP_OPEN (active or passive depending on flags)
14 TCPIP_TCP_CLOSE TCP_CLOSE
15 TCPIP_TCP_ABORT TCP_ABORT
16 TCPIP_TCP_STATE TCP_STATE
17 TCPIP_TCP_SEND TCP_SEND
18 TCPIP_TCP_RCV TCP_RECV
29 TCPIP_WAIT local (ei/halt)
* (others) ERR_NOT_IMP

Implementation notes

Device side (C++)

  • A SocketActivator RAII wrapper drives WSAStartup / WSACleanup on Windows.
  • A background thread (receiverLoop) runs continuously while the device is alive. It iterates over all open TCP and UDP sockets, polls them with select() on a 10 ms cadence, drains any pending data into per-socket queues, and detects connection state transitions (SYN_SENT β†’ ESTABLISHED, remote half-close β†’ CLOSE_WAIT).
  • DNS resolution is offloaded to a detached worker thread so that the emulator does not block during getaddrinfo().
  • connect() is always issued non-blocking; the receiver loop completes the connection state machine.
  • The header deliberately uses intptr_t for socket descriptors so that winsock2.h can stay out of the public header. winsock2.h transitively pulls in windows.h, which defines interface as a preprocessor macro that breaks unrelated openMSX headers.

TSR side (Z80)

  • The TSR follows Konamiman's reference RAM-resident pattern: installer at 0100h allocates a system mapper segment via DOS 2 mapper routines, copies the resident block to 4000h of that segment, and patches EXTBIO to use the RAM helper's segment-call routine.
  • The dispatcher at UNAPI_ENTRY runs each implemented function with ei immediately preceding the final ret. This is required because Nextor's RAM helper does not unconditionally re-enable interrupts on return, and several UNAPI clients rely on *SYSTIMER (0FC9Eh) ticking while they spin.
  • TCPIP_WAIT (function 29) is intercepted ahead of the dispatch table via a fast path. The implementation is ei / halt / ret, which blocks until the next 50/60 Hz interrupt and matches the spec ("block until the next timer interrupt has completed its processing step").

TCPIP_TCP_STATE must not write to (HL)

Per the UNAPI TCP/IP 1.1 spec, function 16 returns connection state via registers (B=state, C=close_reason, HL=available bytes, DE=urgent bytes, IX=output buffer free space). Some clients pass a non-zero HL to indicate they want a full info block written there, but the dominant client library in the wild (MSXgl's unapi_tcp.asm) uses HL purely as the address of its own internal struct that it populates from the returned registers β€” it does not expect the implementation to write into it. Writing 8 bytes of remote/local endpoint info to that pointer corrupts the caller's struct and crashes the application a few frames later when the corrupted fields get interpreted as buffer pointers.

Resolution: FN_TCP_STATE reads the 12-byte response from the bridge, returns the first four bytes through registers, and discards the remote-IP / port / local-port tail. Servers that genuinely need the peer endpoint can recover it from the source address of received data in the application protocol.

TCP_CLOSE serialisation

cmdTcpClose issues shutdown(SD_SEND) and transitions the connection to CLOSE_WAIT. It does not call closesocket() or take the per-connection mutex while transitioning. Both responsibilities are delegated to the receiver loop, which observes recv() == 0 (peer FIN) and tears the socket down. Holding the mutex across both shutdown() and closesocket() deadlocks the emulator if the receiver thread is concurrently calling into the same socket.

References

License

MIT

About

Real TCP/IP networking for MSX software running in openMSX via UNAPI

Resources

Stars

13 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages