A while ago, an inexperienced TAC engineer was claiming double fragmentation was not possible! Worse, he was as assertive as a hallucinating LLM! So let’s go back to basics…
For the sake of the discussion, I’ll ignore the header size so the following might not be completely accurate, but I don’t want to overcomplicate it either…
So let’s imagine we have a large datagram of 3000 bytes, suddenly it has to be transferred over a wire with a MTU of 1400 bytes… We’ll have to chop it into 3 pieces:
- The first IP packet will have an offset of 0, this is where the data starts. There is more data following in another fragment so we’ll set the MF bit to 1 (true).
- The next fragment includes the following data but since we’ve already sent 1400 bytes in the first fragment, the offset will now be set to 1400. We are still not done so the MF bit is also set to 1.
- The last fragment will have an offset of 2800 since we have already sent 2 x 1400 bytes. This time, since there is no more data to send after this one, the MF bit will be set to 0.
All good, we now have our three fragments flowing through the network. Suddenly they reach another choke point where the MTU is set to 1200. This is actually where double fragmentation will happen.
Our first fragment (offset 0, MF bit 1) is being processed, it will be chopped in two:
- We will include 1200 bytes in the first part. It will obviously keep an offset of 0 and MF bit set to 1.
- Where it gets interesting is for our second sub-fragment: it is only 200 bytes large and since we have managed to squeeze 1200 bytes earlier, its offset will now point to 1200.
- The critical part here is that the MF bit was set to 1 in the initial fragment and because of that, we will leave it at that value.
The second fragment will go through the same process and the third fragment is already small enough, no need to fragment it a second time.
Why this matters #
So far, this all looks logical… the issue arises because we end up with small packets with the MF bit set.
Technically, this shouldn’t be a problem… except it has been exploited by malicious actors to generate DoS attacks. Small packets mean you can squeeze a lot of them into a limited bandwidth, and an MF bit set to 1 means the receiving end needs to allocate memory and queues to store the other fragments in order to reassemble the whole frame. The trick is that these other fragments will never arrive, exhausting resources on the target.
If you want to learn more, Wikipedia provides more details: https://en.wikipedia.org/wiki/IP_fragmentation_attack
To protect against this, some security devices would simply drop fragments that look suspicious. For example this F5: https://my.f5.com/manage/s/article/K52103592
On Cisco IOS XE, the feature responsible for this is called Virtual Fragmentation Reassembly (VFR). It detects and prevents the following types of fragment attacks:
- Tiny fragment attack: the attacker makes the fragment size small enough to force Layer 4 (TCP and UDP) header fields into the second fragment. The ACL rules configured for those fields do not match → VFR-3-TINY_FRAGMENTS
- Overlapping fragment attack → VFR-3-OVERLAP_FRAGMENT
- Buffer Overflow attack → VFR-4_FRAG_TABLE_OVERFLOW
VFR is automatically enabled by other features such as Cisco IOS XE Firewall… and here is the sneaky part: “If VFR is enabled by features, the ip virtual-reassembly [-out] command is not displayed in the show running-config.” So you won’t even know it is there.
Do you see where this is going?
ISE as a canary #
ISE can actually be used as a canary to detect this kind of network issue.
Who hasn’t seen authentication failures such as:
- 5440 Endpoint abandoned EAP session and started new
- 12935 Supplicant stopped responding to ISE during EAP-TLS certificate exchange
Too often we would quickly blame it on some misbehaving wireless endpoint, but it is definitely worth giving it a closer look. This is especially true with EAP-TLS and large certificates (and this is definitely not getting better when we’ll have ML-DSA based certs to protect against the quantum threat).
In a real case I investigated, after initial troubleshooting we could see the client was answering back with its certificate but the answer was never making it back to ISE.
Looking at the ASR firewall, the show platform hardware qfp active statistics drop output told a different story:
The certificate exchange response was actually pretty large, large enough to require fragmentation. A second fragmentation was clearly happening, but what exactly was dropping the packets was still unclear.
It is only when we put all the pieces of the puzzle together that we got the full picture:
The IOS-based firewall was indeed the culprit. Disabling VFR was the workaround in this case!