feat: simulate timeouts and truncated responses

CHANGELOG.md

@@ -82,6 +82,18 @@ Two behaviour changes come with it:
   than leaving the proxy to fail to start for an unstated reason.
 - `Start Mitm Proxy` takes `proxy_auth`, requiring clients to authenticate before the
   proxy serves them.
+- Failure simulation, for the paths an application exercises least. `Simulate Timeout`
+  holds a request and then drops it without contacting the server, which is what a client
+  sees when a service accepts a connection and says nothing - a different test from a
+  service that answers with an error. `Simulate Truncated Response` cuts an answer short
+  while it still claims its full length, which is what a connection dropped mid-answer
+  looks like. A dropped connection is `Block Requests` with `mode=RESET` rather than a
+  keyword of its own.
+
+  Bandwidth throttling is deliberately not supported: mitmproxy hands a response body to
+  a synchronous callback with no way to wait between chunks, so the only implementable
+  version would delay the whole body and deliver it in one piece, which is
+  `Add Response Delay` under a name that would promise more than it does.
 - `Get Proxy Rules` returns the loaded rules in the order they are applied.
 - Blocking rules have an alias, like every other rule, so they are removed the same way.
 - Rules survive a restart of the proxy: the registry outlives it, and only the addon

MitmLibrary/__init__.py

@@ -19,6 +19,7 @@
 from robot.api.deco import keyword, library, not_keyword
 from robot.utils import DotDict, timestr_to_secs
 
+from MitmLibrary.failures import TimeoutAction, TruncateAction
 from MitmLibrary.interceptor import Interceptor
 from MitmLibrary.listener import LibraryListener
 from MitmLibrary.matching import ANY_METHOD, MatchMode, UrlMatcher
@@ -153,6 +154,25 @@ class MitmLibrary:
     | # Send everything on through the network's own proxy
     | Start Mitm Proxy    mode=upstream:http://corporate-proxy:3128
 
+    = Failure simulation =
+    Most rules make a request succeed differently. These make it fail the way a network
+    fails, which is usually the least exercised path in an application:
+
+    - `Simulate Timeout` holds a request and then drops it, so the client waits for an
+      answer that never comes.
+    - `Simulate Truncated Response` cuts an answer short while it still claims to be the
+      full length.
+    - A dropped connection is `Block Requests` with `mode=RESET`, which is where that
+      behaviour lives rather than in a keyword of its own.
+
+    How a client reports any of these depends on the HTTP library it uses, so assert that
+    a request failed rather than on the particular error it raised.
+
+    Bandwidth throttling is not supported. mitmproxy hands a response body to a
+    synchronous callback, with no way to wait between chunks, so the only implementable
+    version would delay the whole body and deliver it in one piece - which is
+    `Add Response Delay`, honestly named.
+
     = Mitm Certificates =
     To test with SSL verification or use a browser without ignoring certificates, you need to set up
     certificates related to mitm. Follow the guide on the
@@ -681,6 +701,88 @@ def redirect_requests_to_host(
             alias, url, method, match, times, RedirectAction(host, port, scheme)
         )
 
+    @keyword
+    def simulate_timeout(
+        self,
+        alias: str,
+        url: str,
+        hold: str = "60s",
+        method: str = ANY_METHOD,
+        match: MatchMode = MatchMode.SUBSTRING,
+        times: int = 0,
+    ) -> None:
+        """Holds matching requests and then drops them, without contacting the server.
+
+        This is what a client sees when a service accepts its connection and then says
+        nothing, and it is a different test from a service that answers with an error: an
+        application that handles a 504 correctly may still wait forever when nothing
+        arrives at all.
+
+        Hold the request for longer than the client's own timeout, so the client is the
+        one that gives up. Other traffic is unaffected while a request is held.
+
+        - `alias`: The handle for this rule. Reusing an alias replaces the rule that
+          already uses it.
+        - `url`: The pattern the request url is compared against. See `match`.
+        - `hold`: How long to hold the request, in Robot Framework time format. It is
+          then dropped, so the client does not wait forever if its own timeout is longer.
+        - `method`: Only match this HTTP method. `ANY` matches every method.
+        - `match`: How `url` is interpreted. See the `Matching` section.
+        - `times`: How often the rule may be applied. `0` means unlimited.
+
+        Example:
+        | Simulate Timeout    hang    /api/orders    hold=30s
+        """
+        self._require_proxy()
+        self._add_rule(
+            alias, url, method, match, times, TimeoutAction(timestr_to_secs(hold), hold)
+        )
+
+    @keyword
+    def simulate_truncated_response(
+        self,
+        alias: str,
+        url: str,
+        keep_bytes: Optional[int] = None,
+        keep_fraction: float = 0.5,
+        method: str = ANY_METHOD,
+        match: MatchMode = MatchMode.SUBSTRING,
+        times: int = 0,
+    ) -> None:
+        """Cuts matching responses short while they still claim to be the full length.
+
+        The response keeps saying how long its body was meant to be, so a client reads
+        what arrived, waits for the rest and eventually gives up. That is what a
+        connection dropped mid-answer looks like, and it is the case that finds parsers
+        which assume a body is either complete or absent.
+
+        How the client reports it differs per HTTP library, so assert that the request
+        failed rather than on a particular error.
+
+        - `alias`: The handle for this rule. Reusing an alias replaces the rule that
+          already uses it.
+        - `url`: The pattern the request url is compared against. See `match`.
+        - `keep_bytes`: How many bytes of the body to keep. Overrides `keep_fraction`.
+        - `keep_fraction`: How much of the body to keep, as a fraction. Half by default.
+        - `method`: Only match this HTTP method. `ANY` matches every method.
+        - `match`: How `url` is interpreted. See the `Matching` section.
+        - `times`: How often the rule may be applied. `0` means unlimited.
+
+        A response with no body, and one already shorter than what would be kept, are
+        left alone and say so in the log.
+
+        A compressed body is cut in its compressed form, so what arrives is not decodable
+        rather than being a valid shorter document. That is the more realistic failure,
+        and the more interesting one to test against.
+
+        Example:
+        | Simulate Truncated Response    cut    /api/orders    keep_bytes=10
+        """
+        self._require_proxy()
+        self._add_rule(
+            alias, url, method, match, times, TruncateAction(keep_bytes, keep_fraction)
+        )
+
     @keyword
     def remove_rule(self, alias: str) -> None:
         """Removes the rule with the given alias.

MitmLibrary/failures.py

@@ -0,0 +1,100 @@
+"""
+This file defines the rules that break traffic on purpose.
+
+Blocking a request answers it, cleanly and immediately. These do the opposite: they make a
+request fail the way a real network fails, so a suite can see what the application under
+test does when a service hangs or an answer arrives half-finished. Those paths are usually
+the least exercised and the most likely to be wrong.
+
+They live apart from the other rules because they are the most version-sensitive part of
+the library: each one depends on a mitmproxy behaviour that is documented but not promised,
+and if one has to be reverted it should be possible to do that without touching the model
+everything else is built on.
+"""
+
+import asyncio
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+from mitmproxy import http
+from robot.api import logger
+
+from MitmLibrary.rules import Action, kill_flow, Phase, Priority
+
+
+@dataclass(frozen=True)
+class TimeoutAction(Action):
+    """Holds a request and then drops it, so the client runs into its own timeout.
+
+    The request is never sent, so the service is not involved at all: this is what a
+    client sees when a service accepts a connection and then says nothing. Holding rather
+    than answering is the point - an application that handles a 504 correctly may still
+    hang forever when nothing arrives.
+    """
+
+    hold_seconds: float = 60.0
+    hold: str = ""
+
+    phase = Phase.REQUEST
+    priority = Priority.TERMINAL
+
+    def apply(self, flow: http.HTTPFlow) -> bool:  # pragma: no cover - async path is used
+        raise NotImplementedError("A timeout can only be applied from the async hook.")
+
+    async def apply_async(self, flow: http.HTTPFlow) -> bool:
+        await asyncio.sleep(self.hold_seconds)
+        kill_flow(flow)
+        return True
+
+    def describe(self) -> Dict[str, Any]:
+        return {"type": "timeout", "hold": self.hold, "hold_seconds": self.hold_seconds}
+
+
+@dataclass(frozen=True)
+class TruncateAction(Action):
+    """Cuts a response short while it still claims to be the full length.
+
+    The `content-length` header keeps saying how long the body was meant to be, so a
+    client reads it, waits for the rest, and eventually gives up. That mismatch is the
+    fault being injected, which is why the body is replaced through `raw_content`:
+    `set_content` would helpfully correct the header and there would be nothing wrong
+    with the response at all.
+    """
+
+    keep_bytes: Optional[int] = None
+    keep_fraction: float = 0.5
+
+    phase = Phase.RESPONSE
+    priority = Priority.MUTATE
+
+    def apply(self, flow: http.HTTPFlow) -> bool:
+        if flow.response is None:
+            return False
+        body = flow.response.raw_content
+        if body is None:
+            # A streamed response has no body to cut here, and a 204 or 304 has none at
+            # all. Saying so beats a rule that silently did nothing.
+            logger.info("There was no body to truncate, so the response is unchanged.")
+            return False
+        keep = self._keep(len(body))
+        if keep >= len(body):
+            logger.info(
+                f"The body is {len(body)} bytes, which is not longer than the "
+                f"{keep} bytes to keep, so the response is unchanged."
+            )
+            return False
+        flow.response.raw_content = body[:keep]
+        return False
+
+    def _keep(self, length: int) -> int:
+        """How many bytes to keep, from either a count or a fraction of the body."""
+        if self.keep_bytes is not None:
+            return max(0, self.keep_bytes)
+        return max(0, int(length * self.keep_fraction))
+
+    def describe(self) -> Dict[str, Any]:
+        return {
+            "type": "truncate",
+            "keep_bytes": self.keep_bytes,
+            "keep_fraction": self.keep_fraction,
+        }

MitmLibrary/interceptor.py

@@ -28,15 +28,20 @@ def set_console_logging(self, value: bool) -> None:
         """Enables or disables reporting each manipulation on the console."""
         self.log_to_console = value
 
-    def request(self, flow: http.HTTPFlow) -> None:
-        """Applies the rules that act before the request is sent."""
+    async def request(self, flow: http.HTTPFlow) -> None:
+        """Applies the rules that act before the request is sent.
+
+        Asynchronous because a rule may wait here: a simulated timeout holds the request
+        rather than answering it. mitmproxy awaits an addon hook that returns a
+        coroutine, and waiting in one flow does not hold up the others.
+        """
         for rule in self.registry.snapshot(Phase.REQUEST):
             if not self._applies(rule, flow):
                 continue
             if not self.registry.consume(rule):
                 continue
             self._log(rule, flow)
-            if rule.action.apply(flow):
+            if await rule.action.apply_async(flow):
                 return
 
     async def response(self, flow: http.HTTPFlow) -> None:

MitmLibrary/rules.py

@@ -107,7 +107,7 @@ class BlockAction(Action):
 
     def apply(self, flow: http.HTTPFlow) -> bool:
         if self.mode is BlockMode.RESET:
-            _kill(flow)
+            kill_flow(flow)
             return True
         flow.response = http.Response.make(
             self.status_code, safe_str(self.body) if self.body is not None else b""
@@ -351,7 +351,7 @@ def _is_default_port(scheme: str, port: int) -> bool:
     return (scheme == "http" and port == 80) or (scheme == "https" and port == 443)
 
 
-def _kill(flow: http.HTTPFlow) -> None:
+def kill_flow(flow: http.HTTPFlow) -> None:
     """Drops the connection, if mitmproxy still lets us.
 
     `kill()` raises when the flow is no longer killable, which happens when something else

README.md

@@ -78,6 +78,24 @@ All matching rules are applied. A rule that blocks a request ends it and nothing
 runs; otherwise `Set Response` runs before rules that change part of a response, which run
 before delays, so combinations behave predictably rather than overwriting each other.
 
+### Simulating failures
+
+Most rules make a request succeed differently. These make it fail the way a network does:
+
+```robotframework
+Simulate Timeout                hang   /api/orders   hold=30s
+Simulate Truncated Response     cut    /api/orders   keep_bytes=10
+Block Requests                  drop   /api/orders   mode=RESET
+```
+
+How a client reports any of these depends on the HTTP library it uses, so assert that the
+request failed rather than on the particular error.
+
+Bandwidth throttling is not supported: mitmproxy hands a response body to a synchronous
+callback with no way to wait between chunks, so the only implementable version would delay
+the whole body and deliver it in one piece — which is what `Add Response Delay` already
+does, honestly named.
+
 ### Recording
 
 The proxy can also remember what went through it, so a suite can assert on what the

atest/testcases/Test.robot

@@ -231,6 +231,35 @@ Waiting For A Request That Never Comes Fails
 Recording Keywords Explain Themselves When Recording Is Off
     Run Keyword And Expect Error    *Start Recording*    Get Recorded Requests
 
+A Held Request Runs Into The Client's Own Timeout
+    [Documentation]    How a client reports this differs per HTTP library, so this asserts
+    ...    that the request failed rather than on a particular error. The hold is far
+    ...    longer than the client's timeout, so the client is the one that gives up and
+    ...    the test does not depend on the two being close together.
+    Simulate Timeout    hang    /test_post    hold=30s
+    ${start}    Get Time    epoch
+    Run Keyword And Expect Error    *
+    ...    POST On Session    alias=proxy    url=test_post/1    timeout=2
+    ${end}    Get Time    epoch
+    # It failed because it waited, not because it was refused outright.
+    Should Be True    ${end} - ${start} >= 2    The request failed without waiting
+
+Other Traffic Keeps Flowing While A Request Is Held
+    [Documentation]    Holding must not stall the proxy for everything else.
+    Simulate Timeout    hang    /test_get    hold=30s
+    Check POST Response    <number_size>smaller than 2</number_size>    ${200}
+
+A Truncated Response Fails The Client
+    [Documentation]    The response still claims its full length, so the client waits for
+    ...    a rest that never comes. Which error that produces is the client's business.
+    Simulate Truncated Response    cut    /test_post    keep_bytes=5
+    Run Keyword And Expect Error    *
+    ...    POST On Session    alias=proxy    url=test_post/1    timeout=5
+
+A Response Shorter Than The Cut Is Left Alone
+    Simulate Truncated Response    cut    /test_post    keep_bytes=1000
+    Check POST Response    <number_size>smaller than 2</number_size>    ${200}
+
 Delayed Response With Post
     Check POST Response    <number_size>smaller than 2</number_size>    ${200}
     Add Response Delay    alias=delay    url=test_post    delay=5s