The Internet has shifted from an end-to-end paradigm to an end-to-ends paradigm, i.e. to serve a provider’s content (e.g. a web page, an app feed, …) the client must connect to many ends to fetch various types of resources like web documents, pictures, scripts, news feed, videos, etc… However, even the recent QUIC transport protocol failed to catch up with this change of paradigm, and the user is forced to waste many round-trip times to fetch the entire content from multiple servers. In this paper, we propose Multi-End QUIC, an extension of Multipath QUIC that enables the establishment of sub-streams directly to backend servers or third-party servers. Multi-End QUIC alleviates the need for proxies in the edge that re-encode and delay those sub-streams, that is able to bypass the relays entirely. This leads to a ~50% latency improvement in a preliminary experiment.
Our paper “OpenDesc: From Static NIC Descriptors to Evolvable Metadata Interfaces” will be presented today at HotNets’25.
In OpenDesc, we propose to use P4 as an interface to define packet descriptors. OpenDesc enables to expose NIC capabilities and match them with application intents. We propose a prototype compiler to generate accessors that can directly extract metadata from a negotiated NIC descriptor without the need for intermediate data structures like sk_buff, xdp_sock, rte_mbuf & cie.
Our latest paper with Louis Navarre, Quentin de Coninck, Tom Barbette and Olivier Bonaventure has recently appeared in SIGCOMM CCR!
In short: Flexicast QUIC brings multicast back to the Internet by blending it with unicast, all within QUIC. It offers scalability where multicast works, and robustness where it doesn’t — making it a practical transport for the next generation of large-scale applications.
When distributing live video, software updates, or cloud gaming streams, today’s Internet almost exclusively relies on unicast: each receiver gets its own copy of the data. This is simple and robust but highly inefficient, especially when thousands of receivers consume the same content. The cost is felt both at the sender — which must generate and encrypt per-receiver packets — and in the network, which carries redundant traffic.
Multicast was designed decades ago to solve exactly this problem: a source transmits once, and routers replicate packets along a multicast tree. But despite its promise, multicast never became mainstream on the global Internet. It is difficult to deploy across ISPs, hard to monetize, and fragile — applications always need to fall back to unicast anyway. Most content providers gave up and built massive unicast infrastructures instead.
With the wide deployment of QUIC, a modern transport protocol running above UDP, there is a chance to revisit multicast — not at the IP layer, but directly at the transport layer. This is where Flexicast QUIC comes in.
FlexiCast uses a multicast stream shared between users, and individual unicast streams to send feedback or act as a fallback. The feedback is used to send more FEC frame to compensate loss, and if the client can’t keep up, use unicast as a fallback.
The Idea of Flexicast QUIC
Flexicast QUIC, presented in our SIGCOMM CCR 2025 paper, extends Multipath QUIC to combine the efficiency of multicast with the reliability of unicast. The idea is to make multicast flexible:
Each receiver gets a unicast QUIC path for control and fallback.
The sender also establishes a flexicast flow: a shared, unidirectional path encrypted with a common key and intended for all receivers.
If multicast routing is available, this flow is carried efficiently through the network. If not, the sender can still replicate the packets itself and deliver them over unicast.
Receivers can join or leave the flexicast flow at any time. If multicast fails for one receiver, it automatically falls back to unicast — without affecting others. All this happens within the same QUIC connection, so applications don’t need to juggle two protocols.
Key Design Points
Flexicast QUIC builds on QUIC’s extensible design:
Per-path keys: unlike regular Multipath QUIC, each unicast path and the flexicast flow use distinct encryption keys.
Reliability: acknowledgments are sent over unicast paths, aggregated at the sender to avoid the classic ack implosion problem.
Congestion control: the sender maintains per-receiver congestion states. If one receiver drags the group down, it can be removed from the flexicast flow and served over unicast.
Implementation and Results
We implemented Flexicast QUIC in Cloudflare’s quiche library (Rust), adding ~10,000 lines of code and 5,000 lines of tests. The evaluation was run on CloudLab and emulated networks.
FlexiCast can scale much better than pure QUIC delivery
Scalability
Unicast QUIC: saturates at ~200 receivers (~20 Gbps). CPU is the bottleneck, as every packet must be encrypted per receiver.
Flexicast QUIC: supports 1000 receivers and delivers >80 Gbps, over 4× higher throughput than unicast QUIC, with acceptable CPU usage.
With a small acknowledgment delay (5 ms), Flexicast QUIC perfectly matches the ideal UDP baseline.
Even without multicast in the network, Flexicast QUIC still helps: the sender can replicate encrypted packets using sendmmsg, which is far cheaper than generating per-receiver packets.
Robustness
We tested Flexicast QUIC with a 5 Mbps live video stream under failing multicast trees. Some receivers randomly lost multicast connectivity, forcing fallback to unicast. Results:
Other receivers were unaffected.
Video quality stayed excellent (SSIM > 0.99 for 99.4% of frames).
Latency remained low, with only small tail increases during recovery.
This shows Flexicast QUIC provides seamless continuity even when multicast is unreliable.
Why It Matters
Flexicast QUIC makes multicast practical again:
Efficient: one packet can serve thousands of receivers.
Robust: unicast fallback is built-in, so failures don’t break the stream.
Practical: works today, even without multicast routers.
Deployable: it’s just QUIC — already used by major Internet services.
This makes it promising for content delivery networks, software updates, and live streaming at scale.
What’s Next?
Our future work will explore:
Forward Erasure Correction to improve reliability.
Smarter flow control for heterogeneous receivers.
Source authentication to defend against spoofed multicast traffic.
Multiple flexicast flows (e.g., different video bitrates).
The P4 switch has a “normal switch” mode that is using SONIC and comes with a P4 dataplane compiled and flashed, nothing to compile using Tofino or whatsoever. This is useful when one of the switch is not used for an experiment and you want to simply wire machines virtually.
Creating a L2 bridge or a VLAN
By default, the switch boots with all interface in Router mode. They have (random 10.0…) IPs. To use those interface as a switch, you must remove the IPs before adding them to a vlan or you will get the error “Ethernet0 is a router interface”
$ show ip interfaces
Interface Master IPv4 address/mask Admin/Oper BGP Neighbor Neighbor IP
You can use mine as you wish, I tried to find the original authors and the appropriate license whenever I could. Don’t hesitate to send me your own.
NAND SSD (inspired from https://commons.wikimedia.org/wiki/File:NAND-ssd.svg, CC )RAM Module ( inspired from https://fr.m.wikibooks.org/wiki/Fichier:Ram-module.svg CC)CPU (absed on https://commons.wikimedia.org/wiki/File:Abstract_i7_CPU_icon.svg, CC)DPI (unsure but I think it’s my own. Anyway it’s standard)Fast (own)GPU (own)IPSEC (unsure)Load Balancer (unsure)Monitoring, monitor, measurements (unsure)Mellanox NIC (not SVG, Mellanox)100G NIC (inspired from the above, consider my own I guess)Router (unsure, but this is quite sandard…)VLAN (own)
I’m pleased to announce Retina has been accepted to appear at SIGCOMM at the end of the month ! It is the result of a pleasant collaboration with Gerry Wan, Fengchen Gong and Zakir Durumeric from Stanford.
Retina enables high-speed network forensics by building a binary tailored to a specific experiment written in Rust. It provides convenient filtering capabilities to easily answer questions such as “Is the TLS SNI really random?” or “How many TLS handshake are destined to Netflix?”. Tested at up to 160Gbps with a commodity server on a Stanford traffic TAP, it supports 5-100x higher traffic rates than standard “bloatware” IDSes.
paper ; github ; the video will follow after SIGCOMM
It is possible to use MPTCP, but WSL2 uses a virtual interface that prevents advertising multiple paths. There might be a solution using multiple forwarded ports but I haven’t been able to use it yet.
Prerequisite
Install Ubuntu in WSL2 (simply look for Ubuntu in the Microsoft Store)
Optional: Allow Windows to keep both Wifi and Ethernet open
Windows will automatically turn off wifi when Ethernet is plugged in. If you want to try MPTCP over Wifi + Ethernet (or 4G through USB, all the same) you must disable this behavior :
1. Open Registry Editor.
2. Go to HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Windows\WcmSvc\Local.
3. Create/change the fMinimizeConnections registry DWORD to 0.
4. Close Registry Editor and reboot.
Step 1 : Install an MPTCP-compatible Kernel (easier than it sounds!)
Despite old threads that can be seen on VOO’s forum, VOO do not seem to use SLAAC in bridge mode (anymore?), but DHCPv6. Also VOO only gives a /64 prefix so you can’t do internal subnets 🙁
Important: my outgoing (WAN) interface directly connected to the VOO modem in bridge mode is enx000ec6ec03b3 . My internal LAN interface is br0 (it’s a bridge between my actual eth0 LAN interface and a WiFi access point using hostapd, but that’s for another day).
This tutorial assumes Ubuntu 18.04:
sudo apt install wide-dhcpv6-client
sudo vi /etc/wide-dhcpv6/dhcp6c.conf
interface enx000ec6ec03b3 {
send ia-na 1;
send ia-pd 1;
request domain-name-servers;
request domain-name;
script "/etc/wide-dhcpv6/dhcp6c-script";
};
# Only for prefix delegation
id-assoc pd 1 {
prefix-interface br0 { #internal facing interface (LAN)
sla-id 0; # subnet. Combined with ia-pd to configure the subnet for this interface.
ifid 1; #IP address "postfix". if not set it will use EUI-64 address of the interface. Combined with SLA-ID'd prefix to create full IP address of interface.
sla-len 0; # Number of prefix bits assigned. Sadly this is 0 with voo...
};
};
id-assoc na 1 {
# id-assoc for eth1
};
interface br0 # LAN interface
{
AdvManagedFlag off; # no DHCPv6 server here.
AdvOtherConfigFlag off; # not even for options.
AdvSendAdvert on;
AdvDefaultPreference high;
AdvLinkMTU 1280;
prefix ::/64 #pick one non-link-local prefix assigned to the interface and start advertising it
{
AdvOnLink on;
AdvAutonomous on;
};
};
sudo service radvd restart
Some configuration is taken and adapted from https://www.ipcalypse.ca/?p=204
In this journal version, we extended our conference paper with additional, peer-reviewed material:
We implemented our system on QUIC using P4 and Picoquic. This demonstrates that our approach does not depend solely on TCP timestamps. The code in ‘bmv2’ and ‘p4-tofino’ has been made publicly available. All of our code is available at https://github.com/cheetahlb/.
We added an experiment using the Tofino implementation and the QUIC implementation of Cheetah for an HTTP webserver.
We added an experiment to verify whether today’s OSes support TCP timestamp, have them enabled by default, and correctly echo the TCP timestamp set by a server.
We added an experiment to verify the granularity of the TCP timestamp units used by some of the largest Alexa top 100 websites.
We added a proof sketch on the size of the cookies given a number of servers.
We added an implementation in bmv2 of the “TCP timestamp”-based system. We have also rewritten and published the P4- tofino code of the system. The implementation of the stateful LB is non-trivial as it requires the insertions/lookups/deletions operations to be applied in constant time (and more restrictions apply). We describe our implementation of a stack-based data structure for the Tofino in Section 4.3.
We added a micro-benchmark of the performance of the Cheetah LB, e.g., compared SYN insertions with cuckoo, normal packets,
We broke down the benefits of SSE parsing of TCP options instructions.
We evaluated the packet processing latency overheads of realizing Cheetah on a Tofino for both the TCP timestamp and QUIC implementation.
We clarified the design challenges in the introduction.
Georgios P. Katsika, Tom Barbette, Dejan Kostić, JR. Gerald Q. Maguire, Rebecca Steinert
The NSDI version of Metron supported the integration of blackbox network functions (NFs) using ring buffers. This choice limited Metron’s applicability, as real networks might contain hardware blackboxes (also known as middleboxes) or closed-source blackbox binaries running inside virtual machines (VMs) or containers. In this extended journal version published in ACM Transaction on Computer Systems, we put special effort on integrating these important blackbox types into Metron, while maintaining Metron’s hardware-level performance.
Metron achieves 100G for a chain of VNF, up to 8* better efficiency than SoTA. Check the paper for more details.
This integration was not trivial as it involved tedious low-level system aspects related to (i) efficiently dispatching packets without introducing unnecessary inter-core communication and (ii) techniques to allow high-speed service chaining. These were key principles of Metron that we wanted to maintain. Moreover, we incorporated the latest functionalities of modern 100 GbE NICs, such as single root I/O virtualization (SR-IOV) that enables physical to virtual NIC dispatching, avoiding the need for software switching. Metron instructs the physical NIC to tag the packets according to the core associated with a traffic class by the controller. The tag can then be used to dispatch packets to queues just as a Metron agent does.
As appeared in USENIX NSDI 2018, the original Metron system demonstrated an experiment on dynamic scaling at 10 Gbps. 100 GbE deployments are becoming the new commodity. Therefore, we put substantial effort on refining Metron’s scaling algorithm. Part of this algorithm uses our new method for deriving the load of a CPU core even when this core performs NIC polling (e.g., using DPDK poll mode drivers).
Metron rapidly reacts to change in the input load, see Fig 16 for more details
The 100 GbE testbed used in the NSDI version of Metron exhibited hardware limitations that prevented Metron from reaching line-rate performance. In this journal, we repeated the same experiment on two additional testbeds: First we upgraded the 100 GbE NICs of the original testbed (i.e., replacing the Mellanox ConnectX-4 with newer Mellanox ConnectX-5 NICs) and managed to increase the maximum throughput at 85 Gbps (76 Gbps was the previous limit). Then, we also upgraded the servers of the testbed using new workstations with Intel’s Skylake hardware architecture (the old servers used Intel’s Haswell hardware architecture) and managed to achieve line-rate 100 Gbps packet processing.
The paper also presents a dozen other novelties compared to the NSDI version, so check it out!
