Skip to content

s5uishida/simple_measurement_of_upf_performance_7

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 

Repository files navigation

Simple Measurement of UPF Performance 7

This describes simple performance measurements of several open source UPFs by using PacketRusher as the performance measurement tool. This was measured on the VMs on Proxmox VE. For other measurement results, please see Performance Measurement. PacketRusher is also featured on HPE Developer Blog.

Note. Performance measurement results are highly dependent on the measurement conditions. These results are only examples of results under certain measurement conditions. And this is a very simple measurement, and according to this comment, it doesn't seem to make much sense to measure between VMs. I hope it will serve as a reference for a simple configuration when measuring on real devices.



Table of Contents


Simple Overview of UPF Performance Measurements

Using Open5GS for 5GC, I will easily measure the performance of several open source UPFs with PacketRusher. Note that this configuration is implemented with Proxmox VE VMs.

The following minimum configuration was set as a condition.

  • Only one each for C-Plane, U-Plane(UPF) and RAN&UE(performance measurement tool).

The built simulation environment is as follows.

The 5GC / RAN&UE used are as follows.

The UPFs used are as follows.

Each VMs are as follows.

VM SW & Role IP address OS CPU Mem HDD
VM1 Open5GS 5GC C-Plane 192.168.0.111/24 Ubuntu 24.04 1 2GB 20GB
VM-UP each UPF U-Plane 192.168.0.151/24 Ubuntu 24.04
or 22.04
2 8GB 20GB
VM-DN Data Network Gateway 192.168.0.152/24 Ubuntu 24.04 2 2GB 10GB
VM2 PacketRusher RAN&UE 192.168.0.131/24 Ubuntu 24.04 2 2GB 10GB

Each VM-UP(UPFs) are as follows.

# SW Date Commit OS
a Open5GS UPF v2.7.2 2025.01.12 13585a34e3d5505dd86d2cdf78c91829f3936851 Ubuntu 24.04
b free5GC UPF
(go-upf) v1.2.4
2025.01.02 88fbd8bb7b86761cbbcf1367acf2df7d81392ce3 Ubuntu 24.04
c UPG-VPP v1.13.0 2024.03.25 dfdf64000566d35955d7c180720ff66086bd3572 Ubuntu 22.04
d eUPF v0.6.4 2024.12.23 307b80aec8c7bc8fa376e03927c2f5c55a1fd687 Ubuntu 24.04

The network interfaces of each VM except VM-UP are as follows.

VM Device Model Linux Bridge IP address Interface
VM1 ens18 VirtIO vmbr1 10.0.0.111/24 (NAPT NW)
ens19 VirtIO mgbr0 192.168.0.111/24 (Mgmt NW)
ens20 VirtIO vmbr4 192.168.14.111/24 N4
VM-DN ens18 VirtIO vmbr1 10.0.0.152/24 (NAPT NW)
ens19 VirtIO mgbr0 192.168.0.152/24 (Mgmt NW)
ens20 VirtIO vmbr6 192.168.16.152/24 N6 (default GW for VM-UP)
VM2 ens18 VirtIO vmbr1 10.0.0.131/24 (NAPT NW)
ens19 VirtIO mgbr0 192.168.0.131/24 (Mgmt NW)
ens20 VirtIO vmbr3 192.168.13.131/24 N3

The network interfaces of each VM-UP(UPFs) are as follows. Note that UPFs other than b connect to the Open5GS CN, but b free5GC UPF does not support FTUP flag in PFCP Association Setup Request/Respose, so it connects to free5GC CN.

# SW Device Model Linux Bridge IP address Interface
a Open5GS UPF ens18 VirtIO vmbr1 10.0.0.151/24 (NAPT NW) down
ens19 VirtIO mgbr0 192.168.0.151/24 (Mgmt NW)
ens20 VirtIO vmbr3 192.168.13.151/24 N3
ens21 VirtIO vmbr4 192.168.14.151/24 N4
ens22 VirtIO vmbr6 192.168.16.151/24 N6
b free5GC UPF
(go-upf)
ens18 VirtIO vmbr1 10.0.0.151/24 (NAPT NW) down
ens19 VirtIO mgbr0 192.168.0.151/24 (Mgmt NW)
ens20 VirtIO vmbr3 192.168.13.151/24 N3
ens21 VirtIO vmbr4 192.168.14.151/24 N4
ens22 VirtIO vmbr6 192.168.16.151/24 N6
c UPG-VPP ens18 VirtIO vmbr1 10.0.0.151/24 (NAPT NW)
ens19 VirtIO mgbr0 192.168.0.151/24 (Mgmt NW)
ens20 VirtIO vmbr3 192.168.13.151/24 N3 (Under DPDK by vfio-pci)
ens21 VirtIO vmbr4 192.168.14.151/24 N4 (Under DPDK by vfio-pci)
ens22 VirtIO vmbr6 192.168.16.151/24 N6 (Under DPDK by vfio-pci)
d eUPF ens18 VirtIO vmbr1 10.0.0.151/24 (NAPT NW) down
ens19 VirtIO mgbr0 192.168.0.151/24 (Mgmt NW)
ens20 VirtIO vmbr3 192.168.13.151/24 N3 (XDP)
ens21 VirtIO vmbr4 192.168.14.151/24 N4
ens22 VirtIO vmbr6 192.168.16.151/24 N6 (XDP)

Linux Bridges of Proxmox VE are as follows.

Linux Bridge Network CIDR Interface
vmbr1 10.0.0.0/24 NAPT NW
mgbr0 192.168.0.0/24 Mgmt NW
vmbr3 192.168.13.0/24 N3
vmbr4 192.168.14.0/24 N4
vmbr6 192.168.16.0/24 N6

The main subscriber Information is as follows. Please register the subscriber information on each WebConsole of Open5GS and free5GC.

IMSI DNN DN Key & OPc S-NSSAI
001010000001000 internet 10.45.0.0/16 Key:465B5CE8B199B49FAA5F0A2EE238A6BC
OPc:E8ED289DEBA952E4283B54E88E6183CA
SST:1
SD:010203

Changes in configuration files of Open5GS, free5GC, UPFs and PacketRusher

Please refer to the following for building Open5GS, free5GC, UPFs and PacketRusher respectively.

Changes in configuration files of Open5GS 5GC C-Plane

  • open5gs/install/etc/open5gs/amf.yaml
--- amf.yaml.orig       2025-01-15 04:12:06.000000000 +0900
+++ amf.yaml    2025-01-15 04:22:37.621726917 +0900
@@ -20,29 +20,32 @@
         - uri: http://127.0.0.200:7777
   ngap:
     server:
-      - address: 127.0.0.5
+      - address: 192.168.0.111
   metrics:
     server:
       - address: 127.0.0.5
         port: 9090
   guami:
     - plmn_id:
-        mcc: 999
-        mnc: 70
+        mcc: 001
+        mnc: 01
       amf_id:
         region: 2
         set: 1
   tai:
     - plmn_id:
-        mcc: 999
-        mnc: 70
+        mcc: 001
+        mnc: 01
       tac: 1
   plmn_support:
     - plmn_id:
-        mcc: 999
-        mnc: 70
+        mcc: 001
+        mnc: 01
       s_nssai:
         - sst: 1
+          sd: 1
+        - sst: 1
+          sd: 010203
   security:
     integrity_order : [ NIA2, NIA1, NIA0 ]
     ciphering_order : [ NEA0, NEA1, NEA2 ]
  • open5gs/install/etc/open5gs/nrf.yaml
--- nrf.yaml.orig       2025-01-15 04:12:06.000000000 +0900
+++ nrf.yaml    2025-01-15 04:22:49.084577534 +0900
@@ -11,8 +11,8 @@
 nrf:
   serving:  # 5G roaming requires PLMN in NRF
     - plmn_id:
-        mcc: 999
-        mnc: 70
+        mcc: 001
+        mnc: 01
   sbi:
     server:
       - address: 127.0.0.10
  • open5gs/install/etc/open5gs/smf.yaml
--- smf.yaml.orig       2025-01-15 04:12:06.000000000 +0900
+++ smf.yaml    2025-01-15 04:26:52.389445171 +0900
@@ -7,6 +7,8 @@
   max:
     ue: 1024  # The number of UE can be increased depending on memory size.
 #    peer: 64
+  parameter:
+    use_upg_vpp: true
 
 smf:
   sbi:
@@ -20,16 +22,14 @@
         - uri: http://127.0.0.200:7777
   pfcp:
     server:
-      - address: 127.0.0.4
+      - address: 192.168.14.111
     client:
       upf:
-        - address: 127.0.0.7
-  gtpc:
-    server:
-      - address: 127.0.0.4
+        - address: 192.168.14.151
+          dnn: internet
   gtpu:
     server:
-      - address: 127.0.0.4
+      - address: 192.168.14.111
   metrics:
     server:
       - address: 127.0.0.4
@@ -37,20 +37,17 @@
   session:
     - subnet: 10.45.0.0/16
       gateway: 10.45.0.1
-    - subnet: 2001:db8:cafe::/48
-      gateway: 2001:db8:cafe::1
+      dnn: internet
   dns:
     - 8.8.8.8
     - 8.8.4.4
-    - 2001:4860:4860::8888
-    - 2001:4860:4860::8844
   mtu: 1400
 #  p-cscf:
 #    - 127.0.0.1
 #    - ::1
 #  ctf:
 #    enabled: auto   # auto(default)|yes|no
-  freeDiameter: /root/open5gs/install/etc/freeDiameter/smf.conf
+#  freeDiameter: /root/open5gs/install/etc/freeDiameter/smf.conf
 
 ################################################################################
 # SMF Info

Changes in configuration files of free5GC 5GC C-Plane

  • free5gc/config/amfcfg.yaml
--- amfcfg.yaml.orig    2024-10-14 05:09:24.379203731 +0900
+++ amfcfg.yaml 2024-10-14 05:59:21.011914287 +0900
@@ -5,7 +5,7 @@
 configuration:
   amfName: AMF # the name of this AMF
   ngapIpList:  # the IP list of N2 interfaces on this AMF
-    - 127.0.0.18
+    - 192.168.0.111
   ngapPort: 38412 # the SCTP port listened by NGAP
 
   # Service-based Interface (SBI) Configuration
@@ -30,22 +30,22 @@
   servedGuamiList:
     # <GUAMI> = <MCC><MNC><AMF ID>
     - plmnId: # Public Land Mobile Network ID, <PLMN ID> = <MCC><MNC>
-        mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-        mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+        mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+        mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
       amfId: cafe00 # AMF identifier (3 bytes hex string, range: 000000~FFFFFF)
 
   # the TAI (Tracking Area Identifier) list supported by this AMF
   supportTaiList:
     - plmnId: # Public Land Mobile Network ID, <PLMN ID> = <MCC><MNC>
-        mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-        mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+        mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+        mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
       tac: 000001 # Tracking Area Code (3 bytes hex string, range: 000000~FFFFFF)
 
   # the PLMNs (Public land mobile network) list supported by this AMF
   plmnSupportList:
     - plmnId: # Public Land Mobile Network ID, <PLMN ID> = <MCC><MNC>
-        mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-        mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+        mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+        mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
       snssaiList: # the S-NSSAI (Single Network Slice Selection Assistance Information) list supported by this AMF
         - sst: 1 # Slice/Service Type (uinteger, range: 0~255)
           sd: 010203 # Slice Differentiator (3 bytes hex string, range: 000000~FFFFFF)
  • free5gc/config/ausfcfg.yaml
--- ausfcfg.yaml.orig   2024-09-01 09:47:28.519041774 +0900
+++ ausfcfg.yaml        2024-09-01 09:55:53.372615743 +0900
@@ -16,10 +16,8 @@
   nrfUri: http://127.0.0.10:8000 # a valid URI of NRF
   nrfCertPem: cert/nrf.pem # NRF Certificate
   plmnSupportList: # the PLMNs (Public Land Mobile Network) list supported by this AUSF
-    - mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-      mnc: 93  # Mobile Network Code (2 or 3 digits string, digit: 0~9)
-    - mcc: 123 # Mobile Country Code (3 digits string, digit: 0~9)
-      mnc: 45  # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+    - mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+      mnc: 01  # Mobile Network Code (2 or 3 digits string, digit: 0~9)
   groupId: ausfGroup001 # ID for the group of the AUSF
   eapAkaSupiImsiPrefix: false # including "imsi-" prefix or not when using the SUPI to do EAP-AKA' authentication
 
  • free5gc/config/nrfcfg.yaml
--- nrfcfg.yaml.orig    2024-09-01 09:47:28.520041774 +0900
+++ nrfcfg.yaml 2024-09-01 09:56:09.974381863 +0900
@@ -18,8 +18,8 @@
       key: cert/root.key
     oauth: true
   DefaultPlmnId:
-    mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-    mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+    mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+    mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
   serviceNameList: # the SBI services provided by this NRF, refer to TS 29.510
     - nnrf-nfm # Nnrf_NFManagement service
     - nnrf-disc # Nnrf_NFDiscovery service
  • free5gc/config/nssfcfg.yaml
--- nssfcfg.yaml.orig   2024-09-01 09:47:28.521041774 +0900
+++ nssfcfg.yaml        2024-09-01 09:56:46.775178233 +0900
@@ -18,12 +18,12 @@
   nrfUri: http://127.0.0.10:8000 # a valid URI of NRF
   nrfCertPem: cert/nrf.pem # NRF Certificate
   supportedPlmnList: # the PLMNs (Public land mobile network) list supported by this NSSF
-    - mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-      mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+    - mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+      mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
   supportedNssaiInPlmnList: # Supported S-NSSAI List for each PLMN
     - plmnId: # Public Land Mobile Network ID, <PLMN ID> = <MCC><MNC>
-        mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-        mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+        mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+        mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
       supportedSnssaiList: # Supported S-NSSAIs of the PLMN
         - sst: 1 # Slice/Service Type (uinteger, range: 0~255)
           sd: 010203 # Slice Differentiator (3 bytes hex string, range: 000000~FFFFFF)
  • free5gc/config/smfcfg.yaml
--- smfcfg.yaml.orig    2024-10-14 05:09:24.379203731 +0900
+++ smfcfg.yaml 2024-10-14 06:03:45.294139455 +0900
@@ -42,16 +42,16 @@
 
   # Optional: PLMN IDs configuration.
   plmnList:
-    - mcc: 208 # Mobile Country Code (3 digits string, digit: 0~9)
-      mnc: 93 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
+    - mcc: 001 # Mobile Country Code (3 digits string, digit: 0~9)
+      mnc: 01 # Mobile Network Code (2 or 3 digits string, digit: 0~9)
   locality: area1 # Name of the location where a set of AMF, SMF, PCF and UPFs are located
 
   # PFCP (Packet Forwarding Control Protocol) configuration for N4 interface.
   pfcp:
     # addr config is deprecated in smf config v1.0.3, please use the following config
-    nodeID: 127.0.0.1 # the Node ID of this SMF
-    listenAddr: 127.0.0.1 # the IP/FQDN of N4 interface on this SMF (PFCP)
-    externalAddr: 127.0.0.1 # the IP/FQDN of N4 interface on this SMF (PFCP)
+    nodeID: 192.168.14.111 # the Node ID of this SMF
+    listenAddr: 192.168.14.111 # the IP/FQDN of N4 interface on this SMF (PFCP)
+    externalAddr: 192.168.14.111 # the IP/FQDN of N4 interface on this SMF (PFCP)
     assocFailAlertInterval: 10s
     assocFailRetryInterval: 30s
     heartbeatInterval: 10s
@@ -63,8 +63,8 @@
         type: AN # the type of the node (AN or UPF)
       UPF: # the name of the node
         type: UPF # the type of the node (AN or UPF)
-        nodeID: 127.0.0.8 # the Node ID of this UPF
-        addr: 127.0.0.8 # the IP/FQDN of N4 interface on this UPF (PFCP)
+        nodeID: 192.168.14.151 # the Node ID of this UPF
+        addr: 192.168.14.151 # the IP/FQDN of N4 interface on this UPF (PFCP)
         sNssaiUpfInfos: # S-NSSAI information list for this UPF
           - sNssai: # S-NSSAI (Single Network Slice Selection Assistance Information)
               sst: 1 # Slice/Service Type (uinteger, range: 0~255)
@@ -72,9 +72,9 @@
             dnnUpfInfoList: # DNN information list for this S-NSSAI
               - dnn: internet
                 pools:
-                  - cidr: 10.60.0.0/16
+                  - cidr: 10.45.0.0/16
                 staticPools:
-                  - cidr: 10.60.100.0/24
+                  - cidr: 10.45.100.0/24
           - sNssai: # S-NSSAI (Single Network Slice Selection Assistance Information)
               sst: 1 # Slice/Service Type (uinteger, range: 0~255)
               sd: 112233 # Slice Differentiator (3 bytes hex string, range: 000000~FFFFFF)
@@ -91,7 +91,7 @@
         interfaces: # Interface list for this UPF
           - interfaceType: N3 # the type of the interface (N3 or N9)
             endpoints: # the IP address of this N3/N9 interface on this UPF
-              - 127.0.0.8
+              - 192.168.13.151
             networkInstances: # Data Network Name (DNN)
               - internet
 
@@ -99,7 +99,8 @@
     links:
       - A: gNB1
         B: UPF
-
+  ulcl: false
+  nwInstFqdnEncoding: true
   # retransmission timer for PDU session modification command
   t3591:
     enable: true # true or false

Changes in configuration files of UPFs

a-1. Changes in configuration files of Open5GS 5GC UPF (TUN)

  • open5gs/install/etc/open5gs/upf.yaml
--- upf.yaml.orig       2024-05-02 19:52:00.000000000 +0900
+++ upf.yaml        2024-05-19 12:38:00.000000000 +0900
@@ -11,18 +11,18 @@
 upf:
   pfcp:
     server:
-      - address: 127.0.0.7
+      - address: 192.168.14.151
     client:
 #      smf:     #  UPF PFCP Client try to associate SMF PFCP Server
 #        - address: 127.0.0.4
   gtpu:
     server:
-      - address: 127.0.0.7
+      - address: 192.168.13.151
   session:
     - subnet: 10.45.0.0/16
       gateway: 10.45.0.1
-    - subnet: 2001:db8:cafe::/48
-      gateway: 2001:db8:cafe::1
+      dnn: internet
+      dev: ogstun
   metrics:
     server:
       - address: 127.0.0.7

a-2. Changes in configuration files of Open5GS 5GC UPF (TAP)

  • open5gs/install/etc/open5gs/upf.yaml
--- upf.yaml.orig       2024-05-02 19:52:00.000000000 +0900
+++ upf.yaml        2024-09-23 14:00:20.724467385 +0900
@@ -11,18 +11,18 @@
 upf:
   pfcp:
     server:
-      - address: 127.0.0.7
+      - address: 192.168.14.151
     client:
 #      smf:     #  UPF PFCP Client try to associate SMF PFCP Server
 #        - address: 127.0.0.4
   gtpu:
     server:
-      - address: 127.0.0.7
+      - address: 192.168.13.151
   session:
     - subnet: 10.45.0.0/16
       gateway: 10.45.0.1
-    - subnet: 2001:db8:cafe::/48
-      gateway: 2001:db8:cafe::1
+      dnn: internet
+      dev: ogstap
   metrics:
     server:
       - address: 127.0.0.7

b. Changes in configuration files of free5GC 5GC UPF

  • go-upf/upfcfg.yaml
--- upfcfg.yaml.orig    2024-10-14 04:53:12.341028732 +0900
+++ upfcfg.yaml 2024-10-14 06:11:36.636303534 +0900
@@ -4,8 +4,8 @@
 # PFCP Configuration
 # The listen IP and nodeID of the N4 interface on this UPF (Can't set to 0.0.0.0)
 pfcp:
-  addr: 127.0.0.8   # IP addr for listening
-  nodeID: 127.0.0.8 # External IP or FQDN can be reached
+  addr: 192.168.14.151   # IP addr for listening
+  nodeID: 192.168.14.151 # External IP or FQDN can be reached
   retransTimeout: 1s # retransmission timeout
   maxRetrans: 3 # the max number of retransmission
 
@@ -18,7 +18,7 @@
   # If you bind to a specific IP, ensure SMF uses the same IP in its N3 configuration.
   # If you bind to all (0.0.0.0), SMF can use any of the available UPF IPs, but do not use 0.0.0.0 in SMF.
   ifList:
-    - addr: 127.0.0.8
+    - addr: 192.168.13.151
       type: N3
       # name: upf.5gc.nctu.me
       # ifname: gtpif
@@ -28,9 +28,7 @@
 # List of Data Network Names (DNN) supported by this UPF.
 dnnList:
   - dnn: internet # Data Network Name
-    cidr: 10.60.0.0/16 # Classless Inter-Domain Routing for assigned IPv4 pool of UE
-  - dnn: internet # Data Network Name
-    cidr: 10.61.0.0/16 # Classless Inter-Domain Routing for assigned IPv4 pool of UE
+    cidr: 10.45.0.0/16 # Classless Inter-Domain Routing for assigned IPv4 pool of UE
     # natifname: eth0
 
 # Logging Configuration

c. Changes in configuration files of UPG-VPP

See here for the original files.

  • openair-upf/startup.conf
    There is no change.

  • openair-upf/init.conf
    There is no change.

d. Changes in configuration files of eUPF

See here for the original file.

  • eupf/config.yml
    There is no change.

Changes in configuration files of PacketRusher

  • PacketRusher/config/config.yml
--- config.yml.orig     2024-08-31 21:55:09.517160735 +0900
+++ config.yml  2024-06-09 22:02:26.306291660 +0900
@@ -1,32 +1,32 @@
 gnodeb:
   controlif:
-    ip: "192.168.11.13"
+    ip: "192.168.0.131"
     port: 9487
   dataif:
-    ip: "192.168.11.13"
+    ip: "192.168.13.131"
     port: 2152
   plmnlist:
-    mcc: "999"
-    mnc: "70"
+    mcc: "001"
+    mnc: "01"
     tac: "000001"
     gnbid: "000008"
   slicesupportlist:
     sst: "01"
-    sd: "000001" # optional, can be removed if not used
+    sd: "010203" # optional, can be removed if not used
 ue:
-  msin: "0000000120"
-  key: "00112233445566778899AABBCCDDEEFF"
-  opc: "00112233445566778899AABBCCDDEEFF"
+  msin: "0000001000"
+  key: "465B5CE8B199B49FAA5F0A2EE238A6BC"
+  opc: "E8ED289DEBA952E4283B54E88E6183CA"
   amf: "8000"
   sqn: "00000000"
   dnn: "internet"
   routingindicator: "0000"
   hplmn:
-    mcc: "999"
-    mnc: "70"
+    mcc: "001"
+    mnc: "01"
   snssai:
     sst: 01
-    sd: "000001" # optional, can be removed if not used
+    sd: "010203" # optional, can be removed if not used
   integrity:
     nia0: false
     nia1: false
@@ -38,7 +38,7 @@
     nea2: true
     nea3: false
 amfif:
-  - ip: "192.168.11.30"
+  - ip: "192.168.0.111"
     port: 38412
 logs:
   level: 4

Network settings of UPFs and Data Network Gateway

a-1. Network settings of Open5GS 5GC UPF (TUN)

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1
# sysctl -p

Next, down the interface ens18 of the VM-UP and set the VM-DN IP address to default GW on the N6 interfaceens22.

# ip link set dev ens18 down
# ip route add default via 192.168.16.152 dev ens22

Then, configure the TUNnel interface.

# ip tuntap add name ogstun mode tun
# ip addr add 10.45.0.1/16 dev ogstun
# ip link set ogstun up

a-2. Network settings of Open5GS 5GC UPF (TAP)

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1
# sysctl -p

Next, down the interface ens18 of the VM-UP and set the VM-DN IP address to default GW on the N6 interfaceens22.

# ip link set dev ens18 down
# ip route add default via 192.168.16.152 dev ens22

Then, configure the TAP interface.

# ip tuntap add name ogstap mode tap
# ip addr add 10.45.0.1/16 dev ogstap
# ip link set ogstap up

b. Network settings of free5GC 5GC UPF

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1
# sysctl -p

Next, down the interface ens18 of the VM-UP and set the VM-DN IP address to default GW on the N6 interfaceens22.

# ip link set dev ens18 down
# ip route add default via 192.168.16.152 dev ens22

c. Network settings of UPG-VPP

See this.

d. Network settings of eUPF

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1
# sysctl -p

Next, down the interface ens18 of the VM-UP and set the VM-DN IP address to default GW on the N6 interfaceens22.

# ip link set dev ens18 down
# ip route add default via 192.168.16.152 dev ens22

Network settings of Data Network Gateway

First, uncomment the next line in the /etc/sysctl.conf file and reflect it in the OS.

net.ipv4.ip_forward=1
# sysctl -p

Next, configure NAPT and routing to N6 IP address of UPF.

# iptables -t nat -A POSTROUTING -s 10.45.0.0/16 -j MASQUERADE
# ip route add 10.45.0.0/16 via 192.168.16.151 dev ens20

According to the instructions in here [3], setup it as follows.

# ip link set ens20 mtu 1450

Build Open5GS, free5GC, UPFs and PacketRusher

Please refer to the following for building Open5GS, free5GC, UPFs and PacketRusher respectively.

Install MongoDB on Open5GS and free5GC C-Plane machines. MongoDB Compass is a convenient tool to look at the MongoDB database.

Run Open5GS, free5GC and UPFs

First run each UPF, then each 5GC. Each UPF uses the same IP address, so start only the UPF you want to measure.

Run UPFs

a-1. Run Open5GS 5GC UPF (TUN)

Please use the configuration files changed for TUN interface.

# cd open5gs
# ./install/bin/open5gs-upfd

a-2. Run Open5GS 5GC UPF (TAP)

Please use the configuration files changed for TAP interface.

# cd open5gs
# ./install/bin/open5gs-upfd

b. Run free5GC 5GC UPF

# cd go-upf
# ./upf -c upfcfg.yaml

c. Run UPG-VPP

See this.

d. Run eUPF

See this.

Run 5GC C-Plane

Run Open5GS 5GC C-Plane

./install/bin/open5gs-nrfd &
sleep 2
./install/bin/open5gs-scpd &
sleep 2
./install/bin/open5gs-amfd &
sleep 2
./install/bin/open5gs-smfd &
./install/bin/open5gs-ausfd &
./install/bin/open5gs-udmd &
./install/bin/open5gs-udrd &
./install/bin/open5gs-pcfd &
./install/bin/open5gs-nssfd &
./install/bin/open5gs-bsfd &

Run free5GC 5GC C-Plane

Create the following shell script and run it.

#!/usr/bin/env bash

PID_LIST=()

NF_LIST="nrf amf smf udr pcf udm nssf ausf chf nef"

export GIN_MODE=release

for NF in ${NF_LIST}; do
    ./bin/${NF} &
    PID_LIST+=($!)
    sleep 1
done

function terminate()
{
    sudo kill -SIGTERM ${PID_LIST[${#PID_LIST[@]}-2]} ${PID_LIST[${#PID_LIST[@]}-1]}
    sleep 2
}

trap terminate SIGINT
wait ${PID_LIST}

Measure using PacketRusher

This time, I will measure only one connection by one UE. First, run PacketRusher to establish a connection that will be used to measure performance. Then, start the iperf3 server on the Data Network Gateway.

Run PacketRusher on VM2

Establish a connection for one UE.

# cd PacketRusher
# ./packetrusher ue
INFO[0000] Selecting 192.168.0.111 for host 192.168.0.111 as AMF's IP address 
INFO[0000] Selecting 192.168.13.131 for host 192.168.13.131 as gNodeB's N3/Data IP address 
INFO[0000] Selecting 192.168.0.131 for host 192.168.0.131 as gNodeB's N2/Control IP address 
INFO[0000] Loaded config at: /root/PacketRusher/config/config.yml 
INFO[0000] PacketRusher version 1.0.1                   
INFO[0000] ---------------------------------------      
INFO[0000] [TESTER] Starting test function: Testing an ue attached with configuration 
INFO[0000] [TESTER][UE] Number of UEs: 1                
INFO[0000] [TESTER][UE] disableTunnel is false          
INFO[0000] [TESTER][GNB] Control interface IP/Port: 192.168.0.131/9487~ 
INFO[0000] [TESTER][GNB] Data interface IP/Port: 192.168.13.131/2152 
INFO[0000] [TESTER][AMF] AMF IP/Port: 192.168.0.111/38412 
INFO[0000] ---------------------------------------      
INFO[0000] [GNB] SCTP/NGAP service is running           
INFO[0000] [GNB] Initiating NG Setup Request            
INFO[0000] [GNB][SCTP] Receive message in 0 stream      
INFO[0000] [GNB][NGAP] Receive NG Setup Response        
INFO[0000] [GNB][AMF] AMF Name: open5gs-amf0            
INFO[0000] [GNB][AMF] State of AMF: Active              
INFO[0000] [GNB][AMF] Capacity of AMF: 255              
INFO[0000] [GNB][AMF] PLMNs Identities Supported by AMF -- mcc: 001 mnc:01 
INFO[0000] [GNB][AMF] List of AMF slices Supported by AMF -- sst:01 sd:000001 
INFO[0000] [GNB][AMF] List of AMF slices Supported by AMF -- sst:01 sd:010203 
INFO[0001] [TESTER] TESTING REGISTRATION USING IMSI 0000001000 UE 
INFO[0001] [GNB] Received incoming connection from new UE 
INFO[0001] [UE] Initiating Registration                 
INFO[0001] [UE] Switched from state 0 to state 1        
INFO[0001] [GNB][SCTP] Receive message in 1 stream      
INFO[0001] [GNB][NGAP] Receive Downlink NAS Transport   
INFO[0001] [UE][NAS] Message without security header    
INFO[0001] [UE][NAS] Receive Authentication Request     
INFO[0001] [UE][NAS][MAC] Authenticity of the authentication request message: OK 
INFO[0001] [UE][NAS][SQN] SQN of the authentication request message: VALID 
INFO[0001] [UE][NAS] Send authentication response       
INFO[0001] [UE] Switched from state 1 to state 2        
INFO[0001] [GNB][SCTP] Receive message in 1 stream      
INFO[0001] [GNB][NGAP] Receive Downlink NAS Transport   
INFO[0001] [UE][NAS] Message with security header       
INFO[0001] [UE][NAS] Message with integrity and with NEW 5G NAS SECURITY CONTEXT 
INFO[0001] [UE][NAS] successful NAS MAC verification    
INFO[0001] [UE][NAS] Receive Security Mode Command      
INFO[0001] [UE][NAS] Type of ciphering algorithm is 5G-EA0 
INFO[0001] [UE][NAS] Type of integrity protection algorithm is 128-5G-IA2 
INFO[0001] [GNB][SCTP] Receive message in 1 stream      
INFO[0001] [GNB][NGAP] Receive Initial Context Setup Request 
INFO[0001] [GNB][UE] UE Context was created with successful 
INFO[0001] [GNB][UE] UE RAN ID 1                        
INFO[0001] [GNB][UE] UE AMF ID 1                        
INFO[0001] [GNB][UE] UE Mobility Restrict --Plmn-- Mcc: not informed Mnc: not informed 
INFO[0001] [GNB][UE] UE Masked Imeisv: 1110000000ffff00 
INFO[0001] [GNB][UE] Allowed Nssai-- Sst: [01] Sd: [010203] 
INFO[0001] [GNB][NGAP][AMF] Send Initial Context Setup Response. 
INFO[0001] [GNB] Initiating Initial Context Setup Response 
INFO[0001] [GNB][NGAP] No PDU Session to set up in InitialContextSetupResponse. 
INFO[0001] [UE][NAS] Message with security header       
INFO[0001] [UE][NAS] Message with integrity and ciphered 
INFO[0001] [UE][NAS] successful NAS CIPHERING           
INFO[0001] [UE][NAS] successful NAS MAC verification    
INFO[0001] [UE][NAS] Receive Registration Accept        
INFO[0001] [UE][NAS] UE 5G GUTI: &{119 11 [242 0 241 16 2 0 64 192 0 3 134]} 
INFO[0001] [UE] Switched from state 2 to state 3        
INFO[0001] [UE] Initiating New PDU Session              
INFO[0001] [GNB][SCTP] Receive message in 1 stream      
INFO[0001] [GNB][NGAP] Receive Downlink NAS Transport   
INFO[0001] [UE][NAS] Message with security header       
INFO[0001] [UE][NAS] Message with integrity and ciphered 
INFO[0001] [UE][NAS] successful NAS CIPHERING           
INFO[0001] [UE][NAS] successful NAS MAC verification    
INFO[0001] [UE][NAS] Receive Configuration Update Command 
INFO[0001] [UE] Initiating Configuration Update Complete 
INFO[0001] [GNB][SCTP] Receive message in 1 stream      
INFO[0001] [GNB][NGAP] Receive PDU Session Resource Setup Request 
INFO[0001] [GNB][NGAP][UE] PDU Session was created with successful. 
INFO[0001] [GNB][NGAP][UE] PDU Session Id: 1            
INFO[0001] [GNB][NGAP][UE] NSSAI Selected --- sst: NSSAI was not selected sd: NSSAI was not selected 
INFO[0001] [GNB][NGAP][UE] PDU Session Type: ipv4       
INFO[0001] [GNB][NGAP][UE] QOS Flow Identifier: 1       
INFO[0001] [GNB][NGAP][UE] Uplink Teid: 1               
INFO[0001] [GNB][NGAP][UE] Downlink Teid: 1             
INFO[0001] [GNB][NGAP][UE] Non-Dynamic-5QI: 9           
INFO[0001] [GNB][NGAP][UE] Priority Level ARP: 8        
INFO[0001] [GNB][NGAP][UE] UPF Address: 192.168.13.151 :2152 
INFO[0001] [GNB] Initiating PDU Session Resource Setup Response 
INFO[0001] [UE][NAS] Message with security header       
INFO[0001] [UE][NAS] Message with integrity and ciphered 
INFO[0001] [UE][NAS] successful NAS CIPHERING           
INFO[0001] [UE][NAS] successful NAS MAC verification    
INFO[0001] [UE][NAS] Receive DL NAS Transport           
INFO[0001] [UE][NAS] Receiving PDU Session Establishment Accept 
INFO[0001] [UE][NAS] PDU session QoS RULES: [1 0 6 49 49 1 1 255 1] 
INFO[0001] [UE][NAS] PDU session DNN: internet          
INFO[0001] [UE][NAS] PDU session NSSAI -- sst: 1 sd: 123 
INFO[0001] [UE][NAS] PDU address received: 10.45.0.2    
INFO[0002] [UE][GTP] Interface val0000001000 has successfully been configured for UE 10.45.0.2 
INFO[0002] [UE][GTP] You can do traffic for this UE using VRF vrf0000001000, eg: 
INFO[0002] [UE][GTP] sudo ip vrf exec vrf0000001000 iperf3 -c IPERF_SERVER -p PORT -t 9000 

To avoid writing ip vrf exec vrf0000001000 command everytime you want to run a command in the context of the UE/VRF, run the command as follows.

# ip vrf exec vrf0000001000 bash

Then all the commands in the new shell will use the tunnel.

Run iPerf3 server on Data Network Gateway (VM-DN)

# iperf3 -s

Try ping and iPerf3 client on VM2

Try ping and iperf3 client to the address(192.168.16.152) of N6 interface on Data Network Gateway.

e.g.) The UPF used in the measurement below is eUPF v0.6.4.

# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=64 time=0.494 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=64 time=0.507 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=64 time=0.369 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=64 time=0.407 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=64 time=0.433 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=64 time=0.453 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=64 time=0.414 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=64 time=0.494 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=64 time=0.427 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=64 time=0.448 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9208ms
rtt min/avg/max/mdev = 0.369/0.444/0.507/0.041 ms
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 53640 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   492 MBytes  4.12 Gbits/sec  113    683 KBytes       
[  5]   1.00-2.00   sec   496 MBytes  4.16 Gbits/sec  122    531 KBytes       
[  5]   2.00-3.00   sec   470 MBytes  3.94 Gbits/sec  193    470 KBytes       
[  5]   3.00-4.00   sec   490 MBytes  4.11 Gbits/sec  124    547 KBytes       
[  5]   4.00-5.00   sec   507 MBytes  4.25 Gbits/sec    0    769 KBytes       
[  5]   5.00-6.00   sec   495 MBytes  4.16 Gbits/sec  133    558 KBytes       
[  5]   6.00-7.00   sec   496 MBytes  4.16 Gbits/sec   68    504 KBytes       
[  5]   7.00-8.00   sec   533 MBytes  4.47 Gbits/sec    0    788 KBytes       
[  5]   8.00-9.00   sec   527 MBytes  4.42 Gbits/sec    5    796 KBytes       
[  5]   9.00-10.00  sec   510 MBytes  4.27 Gbits/sec   89    780 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.93 GBytes  4.24 Gbits/sec  847             sender
[  5]   0.00-10.00  sec  4.93 GBytes  4.24 Gbits/sec                  receiver

iperf Done.

Results

These measurements are the values measured between the IP address 10.45.0.0/16 assigned by PacketRusher on VM2 and the IP address 192.168.16.152 of the Data Network Gateway N6 interface on VM-DN. The measurements were taken in the following pattern:

  1. iperf3 -c 192.168.16.152
  2. iperf3 -c 192.168.16.152 -u -b 5G
    UDP packet loss is a value measured under deliberate load (5 Gbps) in order to compare performance limits.
  3. ping 192.168.16.152 -c 10
  4. free5GC UPF connects to the free5GC CN, not the Open5GS CN.
# UPF Date TCP[1]
throughput
UDP[2]
throughput
UDP[2]
packet loss
RTT[3]
(msec)
a-1 Open5GS UPF v2.7.2 (TUN) 2025.01.12 S:790 Mbps
R:789 Mbps
S:2.43 Gbps
R:1.14 Gbps
53 % 0.585
a-2 Open5GS UPF v2.7.2 (TAP) 2025.01.12 S:741 Mbps
R:740 Mbps
S:2.46 Gbps
R:1.18 Gbps
52 % 0.602
b free5GC UPF v1.2.4[4] 2025.01.02 S:2.34 Gbps
R:2.34 Gbps
S:1.47 Gbps
R:1.00 Gbps
32 % 0.490
c UPG-VPP v1.13.0 2024.03.25 S:3.70 Gbps
R:3.70 Gbps
S:2.14 Gbps
R:1.79 Gbps
16 % 0.457
d eUPF v0.6.4 (native mode) 2024.12.23 S:4.24 Gbps
R:4.24 Gbps
S:2.39 Gbps
R:2.28 Gbps
4.5 % 0.444
a-1. Ping and iPerf3 logs for Open5GS UPF v2.7.2 (TUN)
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 57612 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  96.0 MBytes   804 Mbits/sec  227   84.6 KBytes       
[  5]   1.00-2.00   sec  95.5 MBytes   801 Mbits/sec  296   83.3 KBytes       
[  5]   2.00-3.00   sec  93.6 MBytes   785 Mbits/sec  272   76.5 KBytes       
[  5]   3.00-4.00   sec  94.5 MBytes   793 Mbits/sec  312   83.3 KBytes       
[  5]   4.00-5.00   sec  93.2 MBytes   782 Mbits/sec  234   86.0 KBytes       
[  5]   5.00-6.00   sec  94.1 MBytes   790 Mbits/sec  297   80.5 KBytes       
[  5]   6.00-7.00   sec  95.0 MBytes   797 Mbits/sec  275    104 KBytes       
[  5]   7.00-8.00   sec  94.2 MBytes   791 Mbits/sec  254   83.3 KBytes       
[  5]   8.00-9.00   sec  93.0 MBytes   780 Mbits/sec  213   79.2 KBytes       
[  5]   9.00-10.00  sec  92.4 MBytes   775 Mbits/sec  250   83.3 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   942 MBytes   790 Mbits/sec  2630             sender
[  5]   0.00-10.00  sec   941 MBytes   789 Mbits/sec                  receiver

iperf Done.
# iperf3 -c 192.168.16.152 -u -b 5G
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 58682 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   301 MBytes  2.53 Gbits/sec  226132  
[  5]   1.00-2.00   sec   281 MBytes  2.35 Gbits/sec  210490  
[  5]   2.00-3.00   sec   302 MBytes  2.53 Gbits/sec  226306  
[  5]   3.00-4.00   sec   321 MBytes  2.69 Gbits/sec  240805  
[  5]   4.00-5.00   sec   279 MBytes  2.34 Gbits/sec  208987  
[  5]   5.00-6.00   sec   272 MBytes  2.28 Gbits/sec  204138  
[  5]   6.00-7.00   sec   271 MBytes  2.27 Gbits/sec  203041  
[  5]   7.00-8.00   sec   285 MBytes  2.39 Gbits/sec  213416  
[  5]   8.00-9.00   sec   293 MBytes  2.46 Gbits/sec  219729  
[  5]   9.00-10.00  sec   288 MBytes  2.41 Gbits/sec  215741  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  2.82 GBytes  2.43 Gbits/sec  0.000 ms  0/2168785 (0%)  sender
[  5]   0.00-10.00  sec  1.32 GBytes  1.14 Gbits/sec  0.016 ms  1152024/2168785 (53%)  receiver

iperf Done.
# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=63 time=0.513 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=63 time=0.612 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=63 time=0.626 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=63 time=0.578 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=63 time=0.567 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=63 time=0.553 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=63 time=0.566 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=63 time=0.559 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=63 time=0.616 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=63 time=0.660 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9219ms
rtt min/avg/max/mdev = 0.513/0.585/0.660/0.040 ms
a-2. Ping and iPerf3 logs for Open5GS UPF v2.7.2 (TAP)
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 38564 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  90.4 MBytes   757 Mbits/sec  285   77.8 KBytes       
[  5]   1.00-2.00   sec  94.0 MBytes   789 Mbits/sec  350   87.4 KBytes       
[  5]   2.00-3.00   sec  86.8 MBytes   728 Mbits/sec  269   76.5 KBytes       
[  5]   3.00-4.00   sec  87.1 MBytes   731 Mbits/sec  278   79.2 KBytes       
[  5]   4.00-5.00   sec  86.5 MBytes   726 Mbits/sec  230   79.2 KBytes       
[  5]   5.00-6.00   sec  87.8 MBytes   736 Mbits/sec  310   80.5 KBytes       
[  5]   6.00-7.00   sec  88.0 MBytes   738 Mbits/sec  270   73.7 KBytes       
[  5]   7.00-8.00   sec  87.8 MBytes   736 Mbits/sec  249   87.4 KBytes       
[  5]   8.00-9.00   sec  86.9 MBytes   729 Mbits/sec  229   71.0 KBytes       
[  5]   9.00-10.00  sec  88.1 MBytes   739 Mbits/sec  284    124 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   883 MBytes   741 Mbits/sec  2754             sender
[  5]   0.00-10.00  sec   883 MBytes   740 Mbits/sec                  receiver

iperf Done.
# iperf3 -c 192.168.16.152 -u -b 5G
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 55888 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   322 MBytes  2.70 Gbits/sec  241435  
[  5]   1.00-2.00   sec   319 MBytes  2.68 Gbits/sec  239573  
[  5]   2.00-3.00   sec   293 MBytes  2.46 Gbits/sec  219932  
[  5]   3.00-4.00   sec   308 MBytes  2.59 Gbits/sec  231309  
[  5]   4.00-5.00   sec   276 MBytes  2.32 Gbits/sec  207046  
[  5]   5.00-6.00   sec   290 MBytes  2.44 Gbits/sec  217837  
[  5]   6.00-7.00   sec   278 MBytes  2.33 Gbits/sec  208278  
[  5]   7.00-8.00   sec   287 MBytes  2.41 Gbits/sec  215470  
[  5]   8.00-9.00   sec   297 MBytes  2.49 Gbits/sec  222829  
[  5]   9.00-10.00  sec   258 MBytes  2.16 Gbits/sec  193215  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  2.86 GBytes  2.46 Gbits/sec  0.000 ms  0/2196924 (0%)  sender
[  5]   0.00-10.00  sec  1.38 GBytes  1.18 Gbits/sec  0.019 ms  1137413/2196804 (52%)  receiver

iperf Done.
# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=63 time=0.541 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=63 time=0.653 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=63 time=0.606 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=63 time=0.640 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=63 time=0.584 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=63 time=0.637 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=63 time=0.596 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=63 time=0.485 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=63 time=0.672 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=63 time=0.610 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9246ms
rtt min/avg/max/mdev = 0.485/0.602/0.672/0.052 ms
b. Ping and iPerf3 logs for free5GC UPF v1.2.4
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.1 port 54104 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   282 MBytes  2.36 Gbits/sec  219   2.39 MBytes       
[  5]   1.00-2.00   sec   282 MBytes  2.37 Gbits/sec    0   2.57 MBytes       
[  5]   2.00-3.00   sec   284 MBytes  2.38 Gbits/sec    0   2.73 MBytes       
[  5]   3.00-4.00   sec   275 MBytes  2.31 Gbits/sec   14   2.02 MBytes       
[  5]   4.00-5.00   sec   281 MBytes  2.36 Gbits/sec    0   2.13 MBytes       
[  5]   5.00-6.00   sec   281 MBytes  2.35 Gbits/sec    0   2.21 MBytes       
[  5]   6.00-7.00   sec   284 MBytes  2.38 Gbits/sec    0   2.28 MBytes       
[  5]   7.00-8.00   sec   283 MBytes  2.37 Gbits/sec    0   2.37 MBytes       
[  5]   8.00-9.00   sec   252 MBytes  2.12 Gbits/sec  411   1.25 MBytes       
[  5]   9.00-10.00  sec   280 MBytes  2.34 Gbits/sec    0   1.40 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.72 GBytes  2.34 Gbits/sec  644             sender
[  5]   0.00-10.01  sec  2.72 GBytes  2.34 Gbits/sec                  receiver

iperf Done.
# iperf3 -c 192.168.16.152 -u -b 5G
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.1 port 41194 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   166 MBytes  1.39 Gbits/sec  124675  
[  5]   1.00-2.00   sec   163 MBytes  1.37 Gbits/sec  122202  
[  5]   2.00-3.00   sec   162 MBytes  1.36 Gbits/sec  121383  
[  5]   3.00-4.00   sec   137 MBytes  1.15 Gbits/sec  103062  
[  5]   4.00-5.00   sec   217 MBytes  1.82 Gbits/sec  163031  
[  5]   5.00-6.00   sec   225 MBytes  1.89 Gbits/sec  169021  
[  5]   6.00-7.00   sec   181 MBytes  1.52 Gbits/sec  136118  
[  5]   7.00-8.00   sec   168 MBytes  1.41 Gbits/sec  125940  
[  5]   8.00-9.00   sec   166 MBytes  1.39 Gbits/sec  124305  
[  5]   9.00-10.00  sec   163 MBytes  1.37 Gbits/sec  122083  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  1.71 GBytes  1.47 Gbits/sec  0.000 ms  0/1311820 (0%)  sender
[  5]   0.00-10.00  sec  1.17 GBytes  1.00 Gbits/sec  0.019 ms  415152/1311809 (32%)  receiver

iperf Done.
# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=63 time=0.453 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=63 time=0.542 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=63 time=0.543 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=63 time=0.498 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=63 time=0.496 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=63 time=0.507 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=63 time=0.425 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=63 time=0.504 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=63 time=0.466 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=63 time=0.466 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9216ms
rtt min/avg/max/mdev = 0.425/0.490/0.543/0.035 ms
c. Ping and iPerf3 logs for UPG-VPP v1.13.0
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 40034 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   439 MBytes  3.68 Gbits/sec  202    388 KBytes       
[  5]   1.00-2.00   sec   410 MBytes  3.44 Gbits/sec  135    427 KBytes       
[  5]   2.00-3.00   sec   420 MBytes  3.52 Gbits/sec  124    395 KBytes       
[  5]   3.00-4.00   sec   440 MBytes  3.70 Gbits/sec   39    591 KBytes       
[  5]   4.00-5.00   sec   450 MBytes  3.77 Gbits/sec  152    355 KBytes       
[  5]   5.00-6.00   sec   449 MBytes  3.76 Gbits/sec    9    527 KBytes       
[  5]   6.00-7.00   sec   516 MBytes  4.33 Gbits/sec   97    344 KBytes       
[  5]   7.00-8.00   sec   412 MBytes  3.46 Gbits/sec   36    460 KBytes       
[  5]   8.00-9.00   sec   432 MBytes  3.62 Gbits/sec   36    404 KBytes       
[  5]   9.00-10.00  sec   402 MBytes  3.36 Gbits/sec   80    324 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.31 GBytes  3.70 Gbits/sec  910             sender
[  5]   0.00-10.00  sec  4.31 GBytes  3.70 Gbits/sec                  receiver

iperf Done.
# iperf3 -c 192.168.16.152 -u -b 5G
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 35302 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   277 MBytes  2.32 Gbits/sec  207568  
[  5]   1.00-2.00   sec   247 MBytes  2.07 Gbits/sec  185442  
[  5]   2.00-3.00   sec   224 MBytes  1.88 Gbits/sec  168171  
[  5]   3.00-4.00   sec   244 MBytes  2.04 Gbits/sec  182659  
[  5]   4.00-5.00   sec   268 MBytes  2.25 Gbits/sec  201241  
[  5]   5.00-6.00   sec   262 MBytes  2.20 Gbits/sec  196486  
[  5]   6.00-7.00   sec   278 MBytes  2.33 Gbits/sec  208483  
[  5]   7.00-8.00   sec   227 MBytes  1.91 Gbits/sec  170573  
[  5]   8.00-9.00   sec   262 MBytes  2.20 Gbits/sec  196404  
[  5]   9.00-10.00  sec   263 MBytes  2.21 Gbits/sec  197386  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  2.49 GBytes  2.14 Gbits/sec  0.000 ms  0/1914414 (0%)  sender
[  5]   0.00-10.00  sec  2.08 GBytes  1.79 Gbits/sec  0.004 ms  313978/1914370 (16%)  receiver

iperf Done.
# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=63 time=0.438 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=63 time=0.440 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=63 time=0.444 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=63 time=0.502 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=63 time=0.467 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=63 time=0.474 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=63 time=0.450 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=63 time=0.455 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=63 time=0.455 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=63 time=0.449 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9227ms
rtt min/avg/max/mdev = 0.438/0.457/0.502/0.018 ms
d. Ping and iPerf3 logs for eUPF v0.6.4
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 53640 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   492 MBytes  4.12 Gbits/sec  113    683 KBytes       
[  5]   1.00-2.00   sec   496 MBytes  4.16 Gbits/sec  122    531 KBytes       
[  5]   2.00-3.00   sec   470 MBytes  3.94 Gbits/sec  193    470 KBytes       
[  5]   3.00-4.00   sec   490 MBytes  4.11 Gbits/sec  124    547 KBytes       
[  5]   4.00-5.00   sec   507 MBytes  4.25 Gbits/sec    0    769 KBytes       
[  5]   5.00-6.00   sec   495 MBytes  4.16 Gbits/sec  133    558 KBytes       
[  5]   6.00-7.00   sec   496 MBytes  4.16 Gbits/sec   68    504 KBytes       
[  5]   7.00-8.00   sec   533 MBytes  4.47 Gbits/sec    0    788 KBytes       
[  5]   8.00-9.00   sec   527 MBytes  4.42 Gbits/sec    5    796 KBytes       
[  5]   9.00-10.00  sec   510 MBytes  4.27 Gbits/sec   89    780 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.93 GBytes  4.24 Gbits/sec  847             sender
[  5]   0.00-10.00  sec  4.93 GBytes  4.24 Gbits/sec                  receiver

iperf Done.
# iperf3 -c 192.168.16.152 -u -b 5G
Connecting to host 192.168.16.152, port 5201
[  5] local 10.45.0.2 port 38040 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   291 MBytes  2.44 Gbits/sec  218534  
[  5]   1.00-2.00   sec   319 MBytes  2.67 Gbits/sec  238928  
[  5]   2.00-3.00   sec   286 MBytes  2.40 Gbits/sec  214328  
[  5]   3.00-4.00   sec   274 MBytes  2.30 Gbits/sec  205619  
[  5]   4.00-5.00   sec   271 MBytes  2.27 Gbits/sec  203268  
[  5]   5.00-6.00   sec   294 MBytes  2.47 Gbits/sec  220757  
[  5]   6.00-7.00   sec   298 MBytes  2.50 Gbits/sec  223527  
[  5]   7.00-8.00   sec   313 MBytes  2.63 Gbits/sec  234790  
[  5]   8.00-9.00   sec   255 MBytes  2.14 Gbits/sec  191590  
[  5]   9.00-10.00  sec   250 MBytes  2.10 Gbits/sec  187617  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  2.78 GBytes  2.39 Gbits/sec  0.000 ms  0/2138958 (0%)  sender
[  5]   0.00-10.00  sec  2.66 GBytes  2.28 Gbits/sec  0.006 ms  97057/2138958 (4.5%)  receiver

iperf Done.
# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=64 time=0.494 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=64 time=0.507 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=64 time=0.369 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=64 time=0.407 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=64 time=0.433 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=64 time=0.453 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=64 time=0.414 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=64 time=0.494 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=64 time=0.427 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=64 time=0.448 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9208ms
rtt min/avg/max/mdev = 0.369/0.444/0.507/0.041 ms

Summary

These measurement results show that eUPF has relatively outstanding performance even on Proxmox VE VM. If measuring using virtual machines, it would be better to measure on VMs on a hypervisor such as Proxmox VE. Also, it is good to select VirtIO as the network interface to ensure that the network does not become a bottleneck in the measurement.

It is very simple mesurement and may not be very meaningful when measuring between virtual machines, but it may be a little helpful when comparing the relative performance of UPF. I would appreciate it if you could use this as a reference as a configuration example when measuring with real devices.

Performance of N6 interface only

I simply measured the raw communication performance between VM-UP and VM-DN. This is a measurement of the N6 interface and therefore does not include communication over GTP-U.

A--B TCP[1]
throughput
UDP[2]
throughput
UDP[2]
packet loss
RTT[3]
(msec)
VM-UP --(N6)-- VM-DN S:25.6 Gbps
R:25.5 Gbps
S:2.99 Gbps
R:2.83 Gbps
6.9 % 0.260
1. iperf3 -c 192.168.16.152
# iperf3 -c 192.168.16.152
Connecting to host 192.168.16.152, port 5201
[  5] local 192.168.16.151 port 49214 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.97 GBytes  25.5 Gbits/sec    0   2.63 MBytes       
[  5]   1.00-2.00   sec  2.96 GBytes  25.4 Gbits/sec    0   2.91 MBytes       
[  5]   2.00-3.00   sec  2.94 GBytes  25.2 Gbits/sec    0   3.52 MBytes       
[  5]   3.00-4.00   sec  2.98 GBytes  25.6 Gbits/sec    0   3.52 MBytes       
[  5]   4.00-5.00   sec  2.98 GBytes  25.6 Gbits/sec    0   3.52 MBytes       
[  5]   5.00-6.00   sec  2.97 GBytes  25.5 Gbits/sec    0   3.52 MBytes       
[  5]   6.00-7.00   sec  2.97 GBytes  25.5 Gbits/sec    0   3.52 MBytes       
[  5]   7.00-8.00   sec  3.00 GBytes  25.8 Gbits/sec    0   3.72 MBytes       
[  5]   8.00-9.00   sec  2.99 GBytes  25.7 Gbits/sec    0   3.72 MBytes       
[  5]   9.00-10.00  sec  2.99 GBytes  25.7 Gbits/sec    0   3.72 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  29.8 GBytes  25.6 Gbits/sec    0             sender
[  5]   0.00-10.01  sec  29.8 GBytes  25.5 Gbits/sec                  receiver

iperf Done.
2. iperf3 -c 192.168.16.152 -u -b 5G
# iperf3 -c 192.168.16.152 -u -b 5G
Connecting to host 192.168.16.152, port 5201
[  5] local 192.168.16.151 port 58877 connected to 192.168.16.152 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   316 MBytes  2.65 Gbits/sec  228619  
[  5]   1.00-2.00   sec   317 MBytes  2.66 Gbits/sec  229287  
[  5]   2.00-3.00   sec   315 MBytes  2.65 Gbits/sec  228422  
[  5]   3.00-4.00   sec   364 MBytes  3.05 Gbits/sec  263482  
[  5]   4.00-5.00   sec   380 MBytes  3.19 Gbits/sec  275106  
[  5]   5.00-6.00   sec   374 MBytes  3.14 Gbits/sec  271097  
[  5]   6.00-7.00   sec   375 MBytes  3.14 Gbits/sec  271209  
[  5]   7.00-8.00   sec   378 MBytes  3.17 Gbits/sec  274073  
[  5]   8.00-9.00   sec   372 MBytes  3.12 Gbits/sec  269176  
[  5]   9.00-10.00  sec   372 MBytes  3.12 Gbits/sec  269731  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  3.48 GBytes  2.99 Gbits/sec  0.000 ms  0/2580202 (0%)  sender
[  5]   0.00-9.84   sec  3.24 GBytes  2.83 Gbits/sec  0.002 ms  176852/2580202 (6.9%)  receiver

iperf Done.
3. ping 192.168.16.152 -c 10
# ping 192.168.16.152 -c 10
PING 192.168.16.152 (192.168.16.152) 56(84) bytes of data.
64 bytes from 192.168.16.152: icmp_seq=1 ttl=64 time=0.263 ms
64 bytes from 192.168.16.152: icmp_seq=2 ttl=64 time=0.266 ms
64 bytes from 192.168.16.152: icmp_seq=3 ttl=64 time=0.222 ms
64 bytes from 192.168.16.152: icmp_seq=4 ttl=64 time=0.235 ms
64 bytes from 192.168.16.152: icmp_seq=5 ttl=64 time=0.295 ms
64 bytes from 192.168.16.152: icmp_seq=6 ttl=64 time=0.293 ms
64 bytes from 192.168.16.152: icmp_seq=7 ttl=64 time=0.238 ms
64 bytes from 192.168.16.152: icmp_seq=8 ttl=64 time=0.275 ms
64 bytes from 192.168.16.152: icmp_seq=9 ttl=64 time=0.246 ms
64 bytes from 192.168.16.152: icmp_seq=10 ttl=64 time=0.274 ms

--- 192.168.16.152 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9193ms
rtt min/avg/max/mdev = 0.222/0.260/0.295/0.023 ms

I would like to thank all the excellent developers and contributors who created PacketRusher and other great systems and tools.

Changelog (summary)

  • [2025.01.21] Initial release.