- 2.1. Stage 1: Basic QoS Enforcement in Two Namespaces (eBPF Inspired)
- 2.2. Stage 2: Dockerised Two-Namespace Setup
- 2.3. Stage 3: Three-Container Setup with Middlebox (Packet Forwarder)
- 2.4. Stage 4: Multiple "UE" Containers and Middlebox
- 2.5. Stage 5: Integrating Open5GS UPF, UERANSIM, and UEs
To establish a testbed and methodology for validating Quality of Service (QoS) enforcement of a 5G User Plane Function (UPF). This validation aims to confirm that different traffic streams or User Equipments (UEs) with distinct QoS configurations receive varying levels of service quality as intended.
This plan adopts a phased approach, starting with a simplified environment and progressively increasing complexity to finally incorporate a 5G UPF and related components. This allows for incremental validation and easier debugging at each stage. The stages are inspired by and adapted from the eBPF Qdisc classifier example for foundational QoS principles.
Establish fundamental QoS enforcement based on port numbers within two network namespaces. This stage will verify basic traffic classification and prioritization using Linux Traffic Control (tc) and eBPF. The code is adapted from the eBPF Qdisc classifier example project.
- Objective: Establish a basic environment to test QoS enforcement based on port numbers.
- Components:
- Two network namespaces.
iperffor generating traffic.- eBPF Qdisc classifier for traffic control.
- Steps:
- Create two network namespaces.
- Configure eBPF Qdisc classifier to enforce QoS based on port numbers.
- Run
iperfin both namespaces to generate traffic. - Verify that QoS enforcement is correctly applied based on port numbers.
Testing setup
cd tc-basic-classifier
make
# Run the bpf setup. The script will create the namespaces, configure classifiers and start perd
./runner.sh bpfOutput
Starting setup
Warning: sch_htb: quantum of class 10001 is big. Consider r2q change.
Starting iperf3
Connecting to host 172.16.16.20, port 8080
[ 5] local 172.16.16.10 port 39376 connected to 172.16.16.20 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 10.3 MBytes 86.2 Mbits/sec 0 260 KBytes
[ 5] 1.00-2.00 sec 9.51 MBytes 79.8 Mbits/sec 0 260 KBytes
[ 5] 2.00-3.00 sec 8.95 MBytes 75.1 Mbits/sec 0 260 KBytes
[ 5] 3.00-4.00 sec 8.95 MBytes 75.1 Mbits/sec 0 260 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 37.7 MBytes 79.0 Mbits/sec 0 sender
[ 5] 0.00-4.06 sec 36.6 MBytes 75.8 Mbits/sec receiver
iperf Done.
Connecting to host 172.16.16.20, port 8081
[ 5] local 172.16.16.10 port 57776 connected to 172.16.16.20 port 8081
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 5.06 MBytes 42.4 Mbits/sec 0 80.6 KBytes
[ 5] 1.00-2.00 sec 4.47 MBytes 37.5 Mbits/sec 0 80.6 KBytes
[ 5] 2.00-3.00 sec 4.47 MBytes 37.5 Mbits/sec 0 80.6 KBytes
[ 5] 3.00-4.00 sec 4.72 MBytes 39.6 Mbits/sec 0 80.6 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 18.7 MBytes 39.3 Mbits/sec 0 sender
[ 5] 0.00-4.06 sec 18.3 MBytes 37.9 Mbits/sec receiver
iperf Done.
Connecting to host 172.16.16.20, port 8082
[ 5] local 172.16.16.10 port 49058 connected to 172.16.16.20 port 8082
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.88 MBytes 32.5 Mbits/sec 0 294 KBytes
[ 5] 1.00-2.00 sec 2.49 MBytes 20.9 Mbits/sec 0 294 KBytes
[ 5] 2.00-3.00 sec 1.86 MBytes 15.6 Mbits/sec 0 294 KBytes
[ 5] 3.00-4.00 sec 2.49 MBytes 20.9 Mbits/sec 0 294 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 10.7 MBytes 22.5 Mbits/sec 0 sender
[ 5] 0.00-4.11 sec 9.28 MBytes 18.9 Mbits/sec receiver
iperf Done.
We can see that the QoS was correctly enforced.
Adapt Stage 1 to run within Docker containers. This step ensures the setup is portable and easier to manage for subsequent stages.
- Objective: Transition the setup to a Docker-based environment.
- Components:
- Docker containers replacing network namespaces.
- Adapted eBPF Qdisc classifier for Docker.
- Steps:
- Compose a Dockerfiles for client and server
- Adapt the eBPF Qdisc classifier to function within Docker containers.
- Create docker compose for setup
- Run
iperfwithin Docker containers to test QoS enforcement.
Testing setup
cd tc-basic-classifier
docker compose upOutput
✔ Container tc-basic-classifier-server-1 Created 0.0s
✔ Container tc-basic-classifier-client-1 Created 0.0s
Attaching to client-1, server-1
client-1 | Warning: sch_htb: quantum of class 10001 is big. Consider r2q change.
client-1 | Starting iperf3
client-1 | Connecting to host 192.168.70.130, port 8080
client-1 | [ 5] local 192.168.70.131 port 47636 connected to 192.168.70.130 port 8080
client-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client-1 | [ 5] 0.00-1.00 sec 10.6 MBytes 88.8 Mbits/sec 0 260 KBytes
client-1 | [ 5] 1.00-2.00 sec 8.95 MBytes 75.1 Mbits/sec 0 260 KBytes
client-1 | [ 5] 2.00-3.00 sec 8.95 MBytes 75.1 Mbits/sec 0 260 KBytes
client-1 | [ 5] 3.00-4.00 sec 9.51 MBytes 79.8 Mbits/sec 0 260 KBytes
client-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client-1 | [ ID] Interval Transfer Bitrate Retr
client-1 | [ 5] 0.00-4.00 sec 38.0 MBytes 79.7 Mbits/sec 0 sender
client-1 | [ 5] 0.00-4.06 sec 36.6 MBytes 75.7 Mbits/sec receiver
client-1 |
client-1 | iperf Done.
client-1 | Connecting to host 192.168.70.130, port 8081
client-1 | [ 5] local 192.168.70.131 port 47930 connected to 192.168.70.130 port 8081
client-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client-1 | [ 5] 0.00-1.00 sec 5.80 MBytes 48.7 Mbits/sec 0 260 KBytes
client-1 | [ 5] 1.00-2.00 sec 4.47 MBytes 37.5 Mbits/sec 0 260 KBytes
client-1 | [ 5] 2.00-3.00 sec 4.47 MBytes 37.5 Mbits/sec 0 260 KBytes
client-1 | [ 5] 3.00-4.00 sec 5.03 MBytes 42.2 Mbits/sec 0 260 KBytes
client-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client-1 | [ ID] Interval Transfer Bitrate Retr
client-1 | [ 5] 0.00-4.00 sec 19.8 MBytes 41.5 Mbits/sec 0 sender
client-1 | [ 5] 0.00-4.08 sec 18.4 MBytes 37.9 Mbits/sec receiver
client-1 |
client-1 | iperf Done.
client-1 | Connecting to host 192.168.70.130, port 8082
client-1 | [ 5] local 192.168.70.131 port 42412 connected to 192.168.70.130 port 8082
client-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client-1 | [ 5] 0.00-1.00 sec 3.88 MBytes 32.5 Mbits/sec 0 294 KBytes
client-1 | [ 5] 1.00-2.00 sec 2.49 MBytes 20.9 Mbits/sec 0 294 KBytes
client-1 | [ 5] 2.00-3.00 sec 1.86 MBytes 15.6 Mbits/sec 0 294 KBytes
client-1 | [ 5] 3.00-4.00 sec 2.49 MBytes 20.9 Mbits/sec 0 294 KBytes
client-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client-1 | [ ID] Interval Transfer Bitrate Retr
client-1 | [ 5] 0.00-4.00 sec 10.7 MBytes 22.5 Mbits/sec 0 sender
client-1 | [ 5] 0.00-4.11 sec 9.28 MBytes 18.9 Mbits/sec receiver
client-1 |
client-1 | iperf Done.
client-1 exited with code 0
^CGracefully stopping... (press Ctrl+C again to force)
[+] Stopping 2/2
✔ Container tc-basic-classifier-server-1 Stopped 10.1s
✔ Container tc-basic-classifier-client-1 Stopped
The output confirms the enforcement of QoS inside the containers.
Introduce a middlebox container acting as a packet forwarder between two iperf containers. This mimics a more realistic network path and prepares for UPF integration.
- Objective: Introduce a middlebox to simulate the 5G UPF in the final deployment.
- Components:
- Three Docker containers: two for
iperfclients and one as the middlebox. - Middlebox configured as a packet forwarder.
- Three Docker containers: two for
- Steps:
- Create Dockerfile for middlebox
- Create docker compose for setup
- Configure the middlebox container to forward packets between the two
iperfcontainers. - Test the setup to ensure proper packet forwarding and QoS enforcement.
Testing setup
cd tc-basic-classifier
docker compose -f docker-compose-middlebox.yml upOutput
✔ Container tc-basic-classifier-server-1 Created 0.0s
✔ Container tc-basic-classifier-middlebox-1 Created 0.0s
✔ Container tc-basic-classifier-client-1 Created 0.0s
Attaching to client-1, middlebox-1, server-1
server-1 | Running iperf dummy script
middlebox-1 | Warning: sch_htb: quantum of class 10001 is big. Consider r2q change.
middlebox-1 | net.ipv4.ip_forward = 1
client-1 | Running iperf dummy script
client-1 | Connecting to host 192.168.71.130, port 8080
client-1 | [ 5] local 192.168.70.132 port 42456 connected to 192.168.71.130 port 8080
client-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client-1 | [ 5] 0.00-1.00 sec 12.1 MBytes 102 Mbits/sec 0 505 KBytes
client-1 | [ 5] 1.00-2.00 sec 9.80 MBytes 82.2 Mbits/sec 0 973 KBytes
client-1 | [ 5] 2.00-3.00 sec 8.75 MBytes 73.4 Mbits/sec 0 1.40 MBytes
client-1 | [ 5] 3.00-4.00 sec 8.75 MBytes 73.4 Mbits/sec 0 1.86 MBytes
client-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client-1 | [ ID] Interval Transfer Bitrate Retr
client-1 | [ 5] 0.00-4.00 sec 39.4 MBytes 82.7 Mbits/sec 0 sender
client-1 | [ 5] 0.00-4.24 sec 38.3 MBytes 75.8 Mbits/sec receiver
client-1 |
client-1 | iperf Done.
client-1 | Connecting to host 192.168.71.130, port 8081
client-1 | [ 5] local 192.168.70.132 port 58142 connected to 192.168.71.130 port 8081
client-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client-1 | [ 5] 0.00-1.00 sec 6.18 MBytes 51.8 Mbits/sec 0 293 KBytes
client-1 | [ 5] 1.00-2.00 sec 5.90 MBytes 49.5 Mbits/sec 0 526 KBytes
client-1 | [ 5] 2.00-3.00 sec 4.87 MBytes 40.8 Mbits/sec 0 758 KBytes
client-1 | [ 5] 3.00-4.00 sec 5.00 MBytes 41.9 Mbits/sec 0 993 KBytes
client-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client-1 | [ ID] Interval Transfer Bitrate Retr
client-1 | [ 5] 0.00-4.00 sec 21.9 MBytes 46.0 Mbits/sec 0 sender
client-1 | [ 5] 0.00-4.26 sec 19.2 MBytes 37.9 Mbits/sec receiver
client-1 |
client-1 | iperf Done.
client-1 | Connecting to host 192.168.71.130, port 8082
client-1 | [ 5] local 192.168.70.132 port 52734 connected to 192.168.71.130 port 8082
client-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client-1 | [ 5] 0.00-1.00 sec 3.13 MBytes 26.3 Mbits/sec 0 175 KBytes
client-1 | [ 5] 1.00-2.00 sec 2.98 MBytes 25.0 Mbits/sec 0 291 KBytes
client-1 | [ 5] 2.00-3.00 sec 3.04 MBytes 25.5 Mbits/sec 0 407 KBytes
client-1 | [ 5] 3.00-4.00 sec 2.92 MBytes 24.5 Mbits/sec 0 526 KBytes
client-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client-1 | [ ID] Interval Transfer Bitrate Retr
client-1 | [ 5] 0.00-4.00 sec 12.1 MBytes 25.3 Mbits/sec 0 sender
client-1 | [ 5] 0.00-4.27 sec 9.66 MBytes 19.0 Mbits/sec receiver
client-1 |
client-1 | iperf Done.
client-1 exited with code 0
^CGracefully stopping... (press Ctrl+C again to force)
[+] Stopping 3/3
✔ Container tc-basic-classifier-client-1 Stopped 0.0s
✔ Container tc-basic-classifier-middlebox-1 Stopped 10.2s
✔ Container tc-basic-classifier-server-1 Stopped
The output confirms the enforcement of QoS inside the middlebox container.
Simulate multiple UEs by deploying multiple iperf client containers, all communicating through the middlebox. This starts to resemble a multi-UE scenario in 5G.
- Objective: Simulate multiple UEs using separate Docker containers.
- Components:
- Multiple Docker containers, each representing a UE with an
iperfclient. - Middlebox container handling packet forwarding.
- Multiple Docker containers, each representing a UE with an
- Steps:
- Deploy multiple Docker containers, each running an
iperfclient to represent different UEs. - Configure the middlebox to handle traffic from multiple UEs.
- Verify that QoS enforcement is correctly applied to each UE based on their configurations.
- Deploy multiple Docker containers, each running an
Testing setup
cd tc-basic-classifier
docker compose -f docker-compose-middlebox-ues.yml upOutput
✔ Network tc-basic-classifier_phine5g_client_net Created 0.1s
[+] Running 7/7asic-classifier_phine5g_server_net Creating 0.1s
✔ Network tc-basic-classifier_phine5g_client_net Created 0.1s ✔ Network tc-basic-classifier_phine5g_server_net Created 0.1s
✔ Container tc-basic-classifier-server-1 Created 0.0s
✔ Container tc-basic-classifier-middlebox-1 Created 0.0s
✔ Container tc-basic-classifier-client2-1 Created 0.0s
✔ Container tc-basic-classifier-client1-1 Created 0.0s
✔ Container tc-basic-classifier-client3-1 Created 0.0s
Attaching to client1-1, client2-1, client3-1, middlebox-1, server-1
server-1 | Running iperf dummy script
middlebox-1 | Warning: sch_htb: quantum of class 10001 is big. Consider r2q change.
middlebox-1 | net.ipv4.ip_forward = 1
client3-1 | Running iperf dummy script
client2-1 | Running iperf dummy script
client1-1 | Running iperf dummy script
client1-1 | Connecting to host 192.168.71.130, port 8080
client1-1 | [ 5] local 192.168.70.132 port 55266 connected to 192.168.71.130 port 8080
client1-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client1-1 | [ 5] 0.00-1.00 sec 6.98 MBytes 58.6 Mbits/sec 0 335 KBytes
client1-1 | [ 5] 1.00-2.00 sec 6.71 MBytes 56.3 Mbits/sec 0 624 KBytes
client1-1 | [ 5] 2.00-3.00 sec 6.25 MBytes 52.4 Mbits/sec 0 926 KBytes
client1-1 | [ 5] 3.00-4.00 sec 6.25 MBytes 52.4 Mbits/sec 0 1.20 MBytes
client1-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client1-1 | [ ID] Interval Transfer Bitrate Retr
client1-1 | [ 5] 0.00-4.00 sec 26.2 MBytes 54.9 Mbits/sec 0 sender
client1-1 | [ 5] 0.00-4.25 sec 24.0 MBytes 47.4 Mbits/sec receiver
client1-1 |
client1-1 | iperf Done.
client1-1 exited with code 0
client2-1 | Connecting to host 192.168.71.130, port 8081
client2-1 | [ 5] local 192.168.70.133 port 49452 connected to 192.168.71.130 port 8081
client2-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client2-1 | [ 5] 0.00-1.00 sec 6.63 MBytes 55.6 Mbits/sec 0 1.23 MBytes
client2-1 | [ 5] 1.00-2.00 sec 3.75 MBytes 31.5 Mbits/sec 0 1.41 MBytes
client2-1 | [ 5] 2.00-3.00 sec 3.75 MBytes 31.5 Mbits/sec 0 1.58 MBytes
client2-1 | [ 5] 3.00-4.00 sec 3.75 MBytes 31.5 Mbits/sec 0 1.75 MBytes
client2-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client2-1 | [ ID] Interval Transfer Bitrate Retr
client2-1 | [ 5] 0.00-4.00 sec 17.9 MBytes 37.5 Mbits/sec 0 sender
client2-1 | [ 5] 0.00-4.53 sec 15.9 MBytes 29.4 Mbits/sec receiver
client2-1 |
client2-1 | iperf Done.
client2-1 exited with code 0
client3-1 | Connecting to host 192.168.71.130, port 8082
client3-1 | [ 5] local 192.168.70.134 port 44984 connected to 192.168.71.130 port 8082
client3-1 | [ ID] Interval Transfer Bitrate Retr Cwnd
client3-1 | [ 5] 0.00-1.00 sec 5.38 MBytes 45.1 Mbits/sec 0 1.32 MBytes
client3-1 | [ 5] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec 0 1.41 MBytes
client3-1 | [ 5] 2.00-3.00 sec 2.50 MBytes 21.0 Mbits/sec 0 1.51 MBytes
client3-1 | [ 5] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec 0 1.60 MBytes
client3-1 | - - - - - - - - - - - - - - - - - - - - - - - - -
client3-1 | [ ID] Interval Transfer Bitrate Retr
client3-1 | [ 5] 0.00-4.00 sec 10.4 MBytes 21.8 Mbits/sec 0 sender
client3-1 | [ 5] 0.00-4.81 sec 9.16 MBytes 16.0 Mbits/sec receiver
client3-1 |
client3-1 | iperf Done.
client3-1 exited with code 0
^CGracefully stopping... (press Ctrl+C again to force)
[+] Stopping 5/4
✔ Container tc-basic-classifier-client2-1 Stopped 0.0s
✔ Container tc-basic-classifier-client3-1 Stopped 0.0s
✔ Container tc-basic-classifier-client1-1 Stopped 0.0s
✔ Container tc-basic-classifier-middlebox-1 Stopped 0.3s
✔ Container tc-basic-classifier-server-1 Stopped
The output confirms the enforcement of QoS inside the middlebox container with multi-container iperf setup.
Replace the simple packet forwarder middlebox with a real 5G UPF (Open5GS). Integrate UERANSIM gNB and UEs to simulate a more complete 5G scenario and validate UPF's QoS enforcement capabilities.
- Objective: Integrate the 5G UPF into the middlebox.
- Components:
- Open5GS for the 5G core network.
- UERANSIM for gNB and UE simulation.
- Docker containers for Open5GS, UERANSIM gNB, and UERANSIM UEs.
- Steps:
- Deploy Open5GS in the middlebox container.
- Set up UERANSIM gNB in a separate container.
- Deploy UERANSIM UEs in separate containers, each running an
iperfclient. - Connect UEs to the
iperfserver on the data network. - Test and validate QoS enforcement with the integrated 5G UPF.
The setup will use OAI's core network for the rest of the 5GC network functions except for the UPF, which we change to different providers.
Testing setup
cd docker-compose
docker compose -f docker-compose-oai-nrf-eupf.yml up -d
# Wait 10 seconds
docker compose -f docker-compose-ueransim-eupf.yaml up -d
# Check the IP assigned to a ue
docker logs ue1
# Start sending traffic
docker exec -it ue1 \
iperf3 -t 4 -c 192.168.72.135 -p 8080 -B 12.1.1.4Output
$ docker exec -it ue1 iperf3 -t 4 -c 192.168.72.135 -p 8080 -B 12.1.1.4
Connecting to host 192.168.72.135, port 8080
[ 5] local 12.1.1.4 port 44981 connected to 192.168.72.135 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 629 KBytes 5.15 Mbits/sec 117 2.63 KBytes
[ 5] 1.00-2.00 sec 2.04 MBytes 17.1 Mbits/sec 219 2.63 KBytes
[ 5] 2.00-3.00 sec 1.85 MBytes 15.5 Mbits/sec 203 1.32 KBytes
[ 5] 3.00-4.00 sec 2.04 MBytes 17.1 Mbits/sec 218 2.63 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 6.54 MBytes 13.7 Mbits/sec 757 sender
[ 5] 0.00-4.04 sec 6.40 MBytes 13.3 Mbits/sec receiver
iperf Done.
$ docker exec -it ue2 iperf3 -t 4 -c 192.168.72.135 -p 8080 -B 12.1.1.3
Connecting to host 192.168.72.135, port 8080
[ 5] local 12.1.1.3 port 52859 connected to 192.168.72.135 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 463 KBytes 3.80 Mbits/sec 72 2.63 KBytes
[ 5] 1.00-2.00 sec 506 KBytes 4.14 Mbits/sec 96 5.27 KBytes
[ 5] 2.00-3.00 sec 253 KBytes 2.07 Mbits/sec 37 1.32 KBytes
[ 5] 3.00-4.00 sec 3.33 MBytes 28.0 Mbits/sec 244 2.63 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 4.53 MBytes 9.49 Mbits/sec 449 sender
[ 5] 0.00-4.04 sec 4.34 MBytes 9.00 Mbits/sec receiver
iperf Done
$ docker exec -it ue3 iperf3 -t 4 -c 192.168.72.135 -p 8080 -B 12.1.1.2
Connecting to host 192.168.72.135, port 8080
[ 5] local 12.1.1.2 port 52087 connected to 192.168.72.135 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 803 KBytes 6.58 Mbits/sec 123 1.32 KBytes
[ 5] 1.00-2.00 sec 3.15 MBytes 26.4 Mbits/sec 124 2.63 KBytes
[ 5] 2.00-3.00 sec 5.37 MBytes 45.0 Mbits/sec 104 2.63 KBytes
[ 5] 3.00-4.00 sec 8.15 MBytes 68.3 Mbits/sec 158 2.63 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 17.4 MBytes 36.6 Mbits/sec 509 sender
[ 5] 0.00-4.04 sec 17.1 MBytes 35.4 Mbits/sec receiver
iperf Done.
Testing setup
cd docker-compose
docker compose -f docker-compose-oai-nrf-open5gs.yaml up -d
# Wait 10 seconds
docker compose -f docker-compose-ueransim-open5gs.yaml up -d
# Check the IP assigned to a ue
docker logs ue1
# Start sending traffic
docker exec -it ue1 \
iperf3 -t 4 -c 192.168.70.135 -p 8080 -B 12.1.1.4Output
$ docker exec -it ue1 \
iperf3 -t 4 -c 192.168.70.135 -p 8080 -B 12.1.1.4
Connecting to host 192.168.70.135, port 8080
[ 5] local 12.1.1.4 port 37597 connected to 192.168.70.135 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 22.9 MBytes 192 Mbits/sec 232 32.9 KBytes
[ 5] 1.00-2.00 sec 11.5 MBytes 96.3 Mbits/sec 276 31.6 KBytes
[ 5] 2.00-3.00 sec 11.5 MBytes 96.8 Mbits/sec 303 11.8 KBytes
[ 5] 3.00-4.00 sec 11.3 MBytes 94.7 Mbits/sec 286 31.6 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 57.3 MBytes 120 Mbits/sec 1097 sender
[ 5] 0.00-4.04 sec 56.3 MBytes 117 Mbits/sec receiver
iperf Done.
$ docker exec -it ue2 \
iperf3 -t 4 -c 192.168.70.135 -p 8080 -B 12.1.1.3
Connecting to host 192.168.70.135, port 8080
[ 5] local 12.1.1.3 port 49781 connected to 192.168.70.135 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 22.9 MBytes 192 Mbits/sec 286 32.9 KBytes
[ 5] 1.00-2.00 sec 11.0 MBytes 92.1 Mbits/sec 302 34.2 KBytes
[ 5] 2.00-3.00 sec 11.4 MBytes 95.2 Mbits/sec 306 26.3 KBytes
[ 5] 3.00-4.00 sec 11.2 MBytes 94.2 Mbits/sec 353 10.5 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 56.5 MBytes 118 Mbits/sec 1247 sender
[ 5] 0.00-4.04 sec 55.8 MBytes 116 Mbits/sec receiver
iperf Done.
$ docker exec -it ue3 iperf3 -t 4 -c 192.168.70.135 -p 8080 -B 12.1.1.2
Connecting to host 192.168.70.135, port 8080
[ 5] local 12.1.1.2 port 51427 connected to 192.168.70.135 port 8080
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 23.3 MBytes 196 Mbits/sec 280 23.7 KBytes
[ 5] 1.00-2.00 sec 11.5 MBytes 96.8 Mbits/sec 245 46.1 KBytes
[ 5] 2.00-3.00 sec 11.5 MBytes 96.8 Mbits/sec 296 22.4 KBytes
[ 5] 3.00-4.00 sec 10.1 MBytes 84.9 Mbits/sec 237 75.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 56.5 MBytes 119 Mbits/sec 1058 sender
[ 5] 0.00-4.04 sec 55.8 MBytes 116 Mbits/sec receiver
iperf Done.
The output shows that the Open5GS UPF does not perform QoS enforcement.
The setup requires that on container x-dn, the IP 192.168.71.135 be on eth0. Docker compose seems to assign according to aphabetical order most of the time but not all of the time. If not, you will need to restart (trail and error). If there is a way to ensure that 192.168.71.135 is always assign to eth0, a Merge Request will be appreciated :-).
At the time of writing, OAI-UPF only applies QoS to DL, as will reflect in the results below.
$ docker exec -it docker-compose-x-dn-1 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if958: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether de:cc:57:e3:de:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.71.135/26 brd 192.168.71.191 scope global eth0
valid_lft forever preferred_lft forever
3: eth1@if961: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether d6:87:75:2f:4f:91 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.72.135/26 brd 192.168.72.191 scope global eth1
valid_lft forever preferred_lft foreverTesting setup
cd docker-compose
docker compose -f docker-compose-oai-nrf-ebpf.yaml up -d
# Wait 10 seconds
docker compose -f docker-compose-ueransim.yaml up -d
# Check the IP assigned to a ue
docker logs ue1
# Check if connection works. If it doesn't verify that eth0 on x-dn has IP 192.168.71.135, if not restart the deployment
docker exec -it ue1 \
ping -I uesimtun0 192.168.72.135 -c 2
# ====== DL QoS enforcement =========
# Check for DL QoS enforcement (start server on UE)
docker exec -it ue1 \
iperf3 -s -p 8080 -B 12.1.1.4
# Test DL QoS enforcement
docker exec -it x-dn \
iperf3 -t 4 -c 12.1.1.4 -p 8080 -B 192.168.72.135
# ====== UL QoS enforcement =========
# Run client on UE (already have server on x-dn port 8081)
docker exec -it ue2 \
iperf3 -t 4 -c 192.168.72.135 -p 8081 -B 12.1.1.3Output
DL QoS Enforcement
$ docker exec -it ue1 \
iperf3 -s -p 8080 -B 12.1.1.4
-----------------------------------------------------------
Server listening on 8080
-----------------------------------------------------------
Accepted connection from 192.168.72.135, port 34039
[ 5] local 12.1.1.4 port 8080 connected to 192.168.72.135 port 41835
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 5] 1.00-2.00 sec 2.20 MBytes 18.5 Mbits/sec
[ 5] 2.00-3.00 sec 2.20 MBytes 18.5 Mbits/sec
[ 5] 3.00-4.00 sec 2.20 MBytes 18.5 Mbits/sec
[ 5] 4.00-4.25 sec 559 KBytes 18.5 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-4.25 sec 9.28 MBytes 18.3 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 8080
UL QoS Enforcement (not supported yet)
$ docker exec -it ue2 iperf3 -t 4 -c 192.168.72.135 -p 8081 -B 12.1.1.3
Connecting to host 192.168.72.135, port 8081
[ 5] local 12.1.1.3 port 50817 connected to 192.168.72.135 port 8081
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 23.9 MBytes 201 Mbits/sec 1403 59.2 KBytes
[ 5] 1.00-2.00 sec 11.2 MBytes 94.4 Mbits/sec 731 31.6 KBytes
[ 5] 2.00-3.00 sec 13.8 MBytes 115 Mbits/sec 542 125 KBytes
[ 5] 3.00-4.00 sec 10.0 MBytes 83.9 Mbits/sec 684 94.8 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-4.00 sec 58.9 MBytes 124 Mbits/sec 3360 sender
[ 5] 0.00-4.04 sec 56.3 MBytes 117 Mbits/sec receiver
iperf Done.