Skip to content

fix(onvif): answer digest challenges, retry SOAP 1.1, read nested values - #65

Closed
iBinh wants to merge 2 commits into
OpenIPC:mainfrom
iBinh:fix/onvif-digest-and-soap11
Closed

iBinh wants to merge 2 commits into
OpenIPC:mainfrom
iBinh:fix/onvif-digest-and-soap11

Conversation

@iBinh

@iBinh iBinh commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Three separate causes of one unhelpful failure, found bringing up a Hikvision
dome: every probe died with GetCapabilities: empty SOAP body — HTTP 200, zero
bytes, no fault to explain itself.

  1. A digest challenge is never answered. The client sent a preemptive
    Basic header, but the handler carried no credentials, so a 401 went
    unanswered. Hikvision wants Digest for ONVIF, so the request was simply
    unauthorized and the camera declined to say so. Each (host, user) now gets
    an HttpClient whose handler holds the credentials, which is what lets
    HttpClient satisfy Basic or Digest as the camera asks. PreAuthenticate
    stays off so the camera states its terms first, and the preemptive Basic
    header stays for onvif_simple_server, which enforces Basic at the transport
    and never challenges.

  2. Requests went out as SOAP 1.2 only. Several firmwares are built for 1.1
    and answer 1.2 with nothing. A response with no usable envelope is retried
    once as SOAP 1.1 — text/xml, action moved into its own SOAPAction header.
    A fault counts as an answer, so a camera that says why it refused is not
    asked twice.

  3. Values were read at the wrong depth. GetStreamUri read Uri as a
    direct child of the response, which matches only the flatter shape
    onvif_simple_server sends. The spec nests it as MediaUri/Uri, so a
    compliant camera looked like it had no stream at all. SetPreset's token is
    nested the same way on some firmwares.

When both versions come back empty, the error now names what to check on the
camera — ONVIF switched off, or an account without ONVIF rights — instead of
saying "empty SOAP body", and the response is logged at debug level.

Related

No issue. Fixes the SOAP client introduced in #45; Phase 4 — ONVIF + PTZ.

Type

  • Bug fix
  • Feature
  • Refactor / cleanup
  • Docs / CI
  • Other:

Checklist

  • Builds with 0 warnings (TreatWarningsAsErrors=true). Desktop
    solution and the Android head both clean; the iOS head was not built here
    (workload not installed on this machine) — left to CI.
  • Tests pass (dotnet test); new Core logic has unit tests. Six new tests
    in OpenIPC.Viewer.Devices.Tests; suites green at Core 313 / Devices 11 /
    Video 11.
  • No layering violation — this PR touches Devices and its test project
    only.
  • Scope stays within one phase.
  • README / docs updated if public commands, options, or setup changed —
    none did; no new commands, options or endpoints.

Platforms tested

Built and tested on macOS. No head was actually run: the fix lives below the
UI and is exercised by the stub camera described below.

  • Windows
  • Linux
  • macOS
  • Android
  • iOS
  • CI build only

Screenshots / notes

No UI change.

How it is covered. A stub camera on a real socket, so the client's own HTTP
stack does the work — the 401 handshake, the content types and the SOAPAction
header are exercised rather than mocked:

  • a camera that answers SOAP 1.2 with nothing is retried as 1.1
  • the retry carries the action in its own header
  • a Digest challenge is answered
  • a camera that says nothing at all fails with something actionable
  • a fault is reported as the camera worded it, and never retried as 1.1
  • the stream URI is read from MediaUri/Uri

What is not covered. There is no Hikvision in CI. Each failure mode is
reproduced in the stub as observed on the device, but the end-to-end fix is
confirmed against real hardware only by hand. Worth a second pair of eyes from
anyone with a non-OpenIPC camera to hand.

A Hikvision dome failed every probe with "GetCapabilities: empty SOAP
body" — HTTP 200, zero bytes, no fault to explain itself. Three separate
causes, each of which produces that same unhelpful result.

The client carried no credentials on its handler, so a 401 challenge was
never answered. All it sent was a preemptive Basic header, and Hikvision
wants Digest for ONVIF; the request was simply unauthorized and the
camera declined to say so. Each (host, user) now gets an HttpClient whose
handler holds the credentials, which is what lets HttpClient satisfy
Basic or Digest as the camera asks. PreAuthenticate stays off so the
camera states its terms first, and the preemptive Basic header stays for
onvif_simple_server, which enforces Basic at the transport and never
challenges.

Every request went out as SOAP 1.2 only. Several firmwares are built for
1.1 and answer 1.2 with nothing at all. A response with no usable
envelope is now retried once as SOAP 1.1 — different content type, action
moved into its own SOAPAction header. A fault counts as an answer, so a
camera that says why it refused is not asked twice.

And GetStreamUri read Uri as a direct child of the response, which only
matches the flatter shape onvif_simple_server sends. The spec nests it as
MediaUri/Uri, so a compliant camera looked like it had no stream at all;
SetPreset's token is nested the same way on some firmwares. Both now
search the response instead of assuming its depth.

When both versions come back empty the error names what to check — ONVIF
switched off on the camera, or an account without ONVIF rights, which is
what an empty body from a working camera almost always means — and the
response is logged at debug level.

Covered by a stub camera over a real socket, so the client's own HTTP
stack does the work: the 401 handshake, the content types and the
SOAPAction header are all exercised rather than mocked.
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Fix ONVIF Digest auth, SOAP 1.1 fallback, and nested values

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

Grey Divider

AI Description

• Answer Digest challenges while preserving OpenIPC's preemptive Basic compatibility.
• Retry unusable SOAP 1.2 responses once using SOAP 1.1 semantics.
• Read nested ONVIF values and surface actionable empty-response errors.
Diagram

sequenceDiagram
    actor Caller
    participant Client as ONVIF Client
    participant Handler as HTTP Handler
    participant Camera as ONVIF Camera
    participant Parser as SOAP Parser
    Caller->>Client: Invoke operation
    Client->>Handler: Send SOAP 1.2
    Handler->>Camera: Basic request
    Camera-->>Handler: Optional Digest challenge
    Handler->>Camera: Digest retry
    Camera-->>Client: SOAP response
    alt Empty or unusable body
        Client->>Handler: Send SOAP 1.1
        Handler->>Camera: text/xml and SOAPAction
        Camera-->>Client: SOAP 1.1 response
    end
    Client->>Parser: Parse envelope
    Parser-->>Client: Fault or nested value
    Client-->>Caller: Result or actionable error
Loading
High-Level Assessment

The chosen approach is appropriate: native HttpClient authentication avoids a custom Digest implementation, request-scoped SOAP 1.1 fallback preserves standards-first SOAP 1.2 behavior, and real-socket tests verify protocol details that mocked handlers would miss.

Files changed (3) +400 / -29

Bug fix (1) +120 / -29
SoapOnvifClient.csAdd ONVIF authentication negotiation and SOAP 1.1 fallback +120/-29

Add ONVIF authentication negotiation and SOAP 1.1 fallback

• Creates credential-aware HTTP clients per camera endpoint and user so HttpClient can answer Digest challenges while retaining preemptive Basic support. Retries unusable SOAP 1.2 responses as SOAP 1.1, improves empty-body diagnostics, and reads nested stream URI and preset token values.

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs

Tests (2) +280 / -0
SoapOnvifClientInteropTests.csCover ONVIF firmware interoperability scenarios +147/-0

Cover ONVIF firmware interoperability scenarios

• Adds real-HTTP integration tests for SOAP 1.1 fallback, SOAPAction placement, Digest challenge handling, fault preservation, actionable empty responses, and spec-compliant nested stream URIs.

tests/OpenIPC.Viewer.Devices.Tests/Onvif/SoapOnvifClientInteropTests.cs

StubCamera.csAdd socket-backed ONVIF camera test stub +133/-0

Add socket-backed ONVIF camera test stub

• Introduces a configurable HttpListener-based camera stub that records SOAP versions, actions, and authorization headers. It supports authentication challenges and custom payload/status responses for end-to-end HTTP behavior tests.

tests/OpenIPC.Viewer.Devices.Tests/Onvif/StubCamera.cs

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Mutations retried after execution ✓ Resolved 🐞 Bug ≡ Correctness
Description
CallAsync retries every response without a usable SOAP body, including responses to state-changing
calls such as SetPreset and RemovePreset; if the camera performed the first request but returned
an empty or malformed envelope, the retry can create a duplicate preset or turn a successful removal
into a reported failure. Response usability does not prove that the request was not executed, so
this fallback is unsafe for mutations.
Code

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[R325-329]

+        if (!IsUsable(text))
       {
-            var basic = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{c.Username}:{c.Password}"));
-            req.Headers.Authorization = new AuthenticationHeaderValue("Basic", basic);
+            _logger.LogDebug("ONVIF {Action}: SOAP 1.2 gave HTTP {Status} and {Length} bytes; retrying as SOAP 1.1",
+                action, (int)status, text.Length);
+            (status, text) = await SendAsync(service, action, body, credentials, shift, soap12: false, ct)
Evidence
The new retry condition is shared by every operation. Repository code shows mutating methods
construct and submit ContinuousMove, SetPreset, and RemovePreset through CallAuthedAsync,
which reaches this unconditional fallback; SetPreset omits an existing preset token and returns
the newly produced token to UI/API callers, so repeating it can observably duplicate the mutation.

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[170-185]
src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[222-241]
src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[268-287]
src/OpenIPC.Viewer.Core/Onvif/PtzController.cs[81-85]
src/OpenIPC.Viewer.App/ViewModels/SingleCameraPageViewModel.cs[1234-1251]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The SOAP 1.1 fallback blindly repeats state-changing operations when the first response is unusable, even though the camera may already have applied the request.
## Issue Context
Read operations can generally be retried, but `SetPreset`, `RemovePreset`, and other PTZ commands are not safely repeatable based only on response-body usability. Prefer discovering and caching the working SOAP version through a read-only probe, then send mutations once using that version.
## Fix Focus Areas
- src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[317-330]
- src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[170-241]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Authenticated clients never released ✓ Resolved 🐞 Bug ☼ Reliability
Description
The singleton SoapOnvifClient permanently caches a dedicated HttpClient/SocketsHttpHandler for
every host and credential value, with no eviction or disposal, so camera additions and password
changes retain handlers and old secrets for the lifetime of the process. Concurrent GetOrAdd
misses can also construct losing clients that are never disposed.
Code

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[R94-96]

+        var key = $"{service.Host}:{service.Port}\u0000{c.Username}\u0000{c.Password}";
+        return _authedClients.GetOrAdd(key, _ =>
+            NewClient(new NetworkCredential(c.Username, c.Password ?? string.Empty)));
Evidence
The cache has no bound or removal path, its key changes whenever host, username, or password
changes, and each value owns a newly allocated handler. Both application composition roots keep this
client alive as a singleton, while the class has no disposal implementation.

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[37-37]
src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[60-96]
src/OpenIPC.Viewer.Composition/SharedComposition.cs[94-94]
src/OpenIPC.Viewer.Web/Backend/WebBackend.cs[60-60]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The authenticated `HttpClient` cache grows without eviction or disposal and can also leak clients created by competing `GetOrAdd` factories.
## Issue Context
`SoapOnvifClient` is registered as a singleton in both composition roots. Each cached client owns a dedicated `SocketsHttpHandler` containing credentials, including superseded passwords.
## Fix Focus Areas
- src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs[60-96]
- src/OpenIPC.Viewer.Composition/SharedComposition.cs[94-94]
- src/OpenIPC.Viewer.Web/Backend/WebBackend.cs[60-60]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs Outdated
Comment thread src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs Outdated
…-sent, authed clients recycled

Two holes the review caught, both real.

The SOAP 1.1 fallback retried every call whose response was unusable,
including SetPreset and RemovePreset. An unusable response does not
prove the request was not executed — a camera that ran SetPreset and
answered garbage would get a duplicate preset from the resend, and a
resend after a successful but unreadable remove would fault on the
now-missing preset and report failure for a removal that worked. The
dialect a host speaks is now learned once and remembered: the first time
a host answers 1.2 with nothing usable and 1.1 with something, the flip
is cached and later calls lead with 1.1. Since every authed operation is
preceded by the unauthenticated clock probe on first contact, the
dialect is already known by the time any mutation goes out. Mutations
never cross-dialect retry — they use what the host's reads taught and
fail honestly otherwise. The clock-skew retry stays, for mutations too:
a fault means the camera refused the request, not that it ran it.

And the per-credential HttpClient cache was keyed by host, user and
password together with no eviction: every password a camera has ever had
kept a live handler — old secret included — for the rest of the process,
and two threads missing the cache at once could each construct a client
only one of which was ever stored. Keyed by host:port now, since a
camera has one credential at a time; a lookup that finds a different
credential swaps the entry and disposes the superseded client (a request
in flight on it was sent with the old password and failing anyway), and
a plain lock replaces GetOrAdd so the losing constructor of a concurrent
miss never exists. Growth is bounded by the camera addresses spoken to.

Two tests pin the retry behaviour: the dialect is remembered (exactly
one 1.2 request ever reaches a 1.1-only host), and a SetPreset whose
response is empty is reported as failed after exactly one attempt.
@keyldev
keyldev self-requested a review August 27, 2026 16:46
@keyldev

keyldev commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Good PR. Three causes of one symptom are separated cleanly, and each is covered by a test
against a real socket (HttpListener) rather than a mocked handler — so the 401 handshake,
the content types and the SOAPAction header are actually exercised. The follow-up commit
answering the bot review (dialect learned once from the clock probe, mutations never
cross-dialect retried, authed clients recycled under a lock) addresses the substance, not
just the wording.

1. A VersionMismatch fault defeats the SOAP 1.1 retry — reproduced

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs:457 (IsUsable)

IsUsable treats any fault as a usable answer. But the canonical way for a firmware built
for SOAP 1.1 to reject a 1.2 envelope is exactly a VersionMismatch fault, and gSOAP —
which most ONVIF firmware is built on — sends precisely that.

Wrote a throwaway test: camera answers 1.2 with SOAP-ENV:VersionMismatch, answers 1.1
normally. Result:

OnvifFaultException : ONVIF fault for .../GetCapabilities:
    SOAP version mismatch or invalid SOAP message

No 1.1 retry happens at all, and CallAuthedAsync burns another attempt refreshing the
clock shift on top. So the PR fixes the camera that stays silent but not the camera that
politely explains itself. Fix in IsUsable:

var body = Child(Child(XDocument.Parse(text).Root!, "Body"), null);
if (body is null) return false;
// The one fault that means "ask again differently": a 1.1-only firmware
// rejecting a 1.2 envelope instead of staying silent.
if (body.Name.LocalName == "Fault" &&
    (Descendant(body, "faultcode")?.Value ?? Descendant(body, "Value")?.Value ?? "")
        .Contains("VersionMismatch", StringComparison.Ordinal))
    return false;
return true;

2. A 401 is never named, though the status is threaded in for exactly this

src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs:468 (EmptyBody)

Verified: a camera answering 401 + an HTML body on every attempt (wrong password) produces

ONVIF .../GetCapabilities: the camera returned an empty SOAP body (HTTP 401).
Check that ONVIF is enabled on the camera and that this account may use it.

The body was not empty, and the cause is not "ONVIF is switched off" — it is the wrong
password, which is the single most common real case. status already reaches EmptyBody,
so a branch on Unauthorized/Forbidden is three lines and squarely in the spirit of the PR.

3. _soap11Hosts is keyed by host, the client cache by host:port

SoapOnvifClient.cs:400 vs SoapOnvifClient.cs:107

The client cache correctly includes the port; the dialect verdict does not. Two cameras
behind one NAT address on different ports will share a verdict that belongs to only one of
them. (_shiftByHost had this already, but this key is being chosen fresh here.)

Minor

  • GetTimeShiftAsync passes credentials: null, so ClientFor returns the shared
    credential-less _http. A camera that requires HTTP auth for every request, including
    GetSystemDateAndTime, silently yields shift = 0 and the WS-Security Created stamp
    falls back to host time. Fixing it means separating transport credentials from envelope
    credentials — one parameter currently drives both. Not a regression; the same was true
    before this PR.
  • IsUsable parses the response, then CallAsync parses it again — up to three parses per
    call on the retry path. Returning the parsed XElement would avoid it.
  • SoapOnvifClient is a singleton (SharedComposition.cs:119, WebBackend.cs:60) and is
    not IDisposable, so handlers live for the process lifetime. Growth is bounded by the
    number of camera addresses, so this is tidiness rather than a leak.
  • The digest test accepts any header starting with Digest, so it pins "the client answers
    the challenge", not "the digest is computed correctly". Adequate for the bug — worth
    knowing as the boundary.
  • The description says six new tests; there are eight.

keyldev added a commit that referenced this pull request Sep 22, 2026
# Conflicts:
#	src/OpenIPC.Viewer.Devices/Onvif/SoapOnvifClient.cs
keyldev added a commit that referenced this pull request Sep 22, 2026
- Retry a VersionMismatch fault as SOAP 1.1; name a 401/403 as a bad login
- Key the learned SOAP dialect by host:port, like the client cache
- Candidate probes never flip the dialect; MoveStatus="1" reads as true
- Prefer the FOV relative space when a camera declares both
@keyldev keyldev mentioned this pull request Sep 22, 2026
11 of 20 tasks
@keyldev keyldev closed this Sep 22, 2026
@keyldev

keyldev commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Thanks a lot for this — the work is merged, with your commits and authorship kept, into #68 . The review items are fixed on top there, so I'm closing this one in favour of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants