MTU mismatches are the primary reason an OSPF adjacency becomes stuck in the EXSTART state.  After hellos are exchanged and the routers become neighbors, each OSPF speaker advertises the IP MTU of it’s local interface in a Data Base Description (DBD) LSA.  If there is a mismatch you’ll probably just adjust the configuration to be identical on both ends of the link and be done. However, when you try to peer two OSPF routers with different network operating systems, things start to fall apart fairly quickly.

What do you mean by MTU?

One of the biggest issues I have seen with MTU is that it means different things to different vendors and engineers.  From an OSPF perspective the IP MTU is the maximum size of an IP packet including the headers. OSPF exchanges this information in the initial DBD LSAs so that it can effeciently pack LSAs or LSA headers into IP packets, hoping to avoid IP packet fragmentation or likely loss of the packet.   However every vendor OS seems to have a different way to set the IP MTU, and misconfiguration is the root cause of most OSPF issues.  Let’s explore some examples.

MTU on Cisco IOS

If you’re familiar with Cisco IOS on routers, you’ll know that there are two interface level commands available: ‘mtu’ and ‘ip mtu‘.  Ivan Pepelnjak calls this first mtu the ‘hardware mtu’, and I’ll adopt that term here. All MTU’s are measured in bytes.
The aim of the ‘hardware mtu‘ is to ensure that physical interace can handle the full datagram when layer-2 framing overhead is added to the maximum IP packet size.
The ‘ip mtu’ is the maximum sized IP packet that can be transmitted out an interface. The router will need to fragment an IP packet before transmission if it exceeds the ‘ip mtu’.
On Cisco IOS  the ‘mtu’ and ‘ip mtu’ are set to both set to 1500 bytes by default, which really confused me. How can you have an IP MTU of 1500 bytes and then add an 18-byte ethernet header, but still keep your total frame size under the 1500 byte ‘hardware mtu’?
Well it turns out that the interface controller is programmed with a value of the configured ‘hardware mtu’ + 24 bytes.  (well on my test dynamips 7200VXR anyway)

7200-r1#sh interfaces fa1/0 | i MTU
 MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec
7200-r1#sh ip interface f1/0 | i MTU
 MTU is 1500 bytes
7200-r1#show controllers fa1/0 | i mtu
 max_mtu=1524

Note that there are some odd rules for how and when the ‘ip mtu’ and ‘mtu’ will appear in the IOS running config. Best to use the interface commands above and read Ivan’s post to learn more about this. 
It is frustrating that you have to set the hardware MTU indirectly, knowing that IOS will add 24 bytes to the ‘hardware MTU’ you configure.  Life is still simple for ‘ip mtu’,  what you configure (e.g. 1500) will be what OSPF advertises in it’s DBD.  You can verify the IP MTU using the ip-specific show interface command ‘sh ip interface <if_name>’.

NXOS

NXOS has a single MTU command, but it’s meaning is bound to the interface role. For brevity I only consider the Nexus 7000.  The behaviour is different again for Nexus 5000/2000 series switches, which we’ll leave for now.
If you have a layer 2 port (switchport), then you have two choices for the layer-2 frame size. 1500 or 9216 (if system jumbo mtu is enabled).
If you have a default layer 3 port (no switchport), then you can set any value from 576 to 9216 as the advertised ip mtu, still using the ‘mtu’ command.
So for NXOS there is only a single command ‘mtu’.  This keyword signifies ip mtu if it’s a layer-3 interface.  For the layer-2 interface, ‘mtu’ means hardware mtu.  Similar to IOS,  you’re still not configuring the hardware mtu explicitly as there’s a hidden overhead.

IOS-XE

IOS-XE is the OS you’ll see on the lower end ASR routers like the ASR1004.   The MTU configuration on IOS-XE is very similar to IOS. The IOS-XE configuration guide clearly calls out the difference between the configured ‘interface mtu’ and the ‘ip mtu’. This time there’s a 22 byte difference between the configured interface MTU and the actual hardware MTU.  You set the interface mtu using the ‘mtu’ command.  The IOS-XE configuration guide does a good job of breaking out these 22-bytes:
The default interface MTU size accommodates a 1500-byte packet, plus 22 additional bytes to cover  the following additional overhead:

• – Layer 2 header—14 bytes
• – Dot1q header—4 bytes
• – CRC—4 bytes

The IP MTU again is actually pretty straightforward.  You can set it explicitly on the interface directly using an ‘ip mtu’ command.

How big is the ethernet frame overhead?

I know it seems like a softball question.  It’s 18 bytes right?  Well, not so fast.   The ether ‘header’, properly defined as the bytes preceding the ethernet payload is 14 bytes long.   There is also a 4 byte CRC trailer.
Again precision in terminology is important. The ethernet header is 14 bytes, the ethernet trailer is 4 bytes and the total ethernet overhead is 18-bytes.

IOS-XR

This header size issue, becomes really important when you deal with IOS-XR, as seen on ASR9000 and XR-series (formerly GSR) routers.  In IOS-XR there are a few changes in the approach to MTU.

• You must configure the true hardware MTU explicitly. e.g 1514 for ethernet. There are no hidden overheads added by the router.
• You do not get to configure the ip ‘mtu’ explicitly, it is derived from the hardware/interface MTU.
• Ethernet overhead is seen as 14 bytes.
• XR does not count the CRC trailer when calculating IP mtu.  E.g. a 1514 interface mtu provides a 1500 byte IP MTU.

RP/0/RP0/CPU0:ios#sh ipv4 int gi0/0/0/2 | i MTU
 MTU is 1514 (1500 is available to IP)

I love the fact that you’re in full control of the MTU here, but the 14-byte difference can throw you for a loop the first time you see it. Also not having direct control over the IP MTU is a little odd.

Junos

Junos also takes the ‘you know what you’re doing’ approach to MTU, but provides you full direct access to configure the IP and hardware MTU directly.

• You can explicitly configure both the hardware and IP MTU. Again, no hidden overheads like IOS/IOS-XE/NXOS.
• Junos like IOS-XR, does not count the 4-byte CRC trailer as part of the L2 frame overhead.
• E.g. If the interface mtu is 1514 bytes, junos will not allow you to configure an IP  mtu greater than 1500 bytes.
• You can examine the hardware mtu and IP mtu directly, using ‘show interface xe-0/0/0 detail | match mtu’

lab@junos-r1> show interfaces xe-0/0/5 detail | match MTU
 Link-level type: Ethernet, MTU: 1514, Speed: 10Gbps,  <--snip-->
 Protocol inet, MTU: 1500, Generation: 161, Route table: 0

What about Vlan tags?

All the examples above have considered default ethernet frame sizes without VLAN tagging.

• For IOS-like OSs you get the first vlan tag without needing any additional configuration.  The hidden 22-24 bytes difference between configured interface MTU and actual hardware MTU helps you here.
• If you want to do Q-in-Q tunnelling, then you’ll need to hack the interface MTU to a higher value e.g. 1504, so that the hardware MTU is sufficiently high to handle the extra tag.
• For junos and IOS-XE, you configure the overhead explicitly. See below.

 

Sherpa Summary

OSPF is only concerned with the ‘IP MTU’ of an interface. In general configuring IP MTU is much simpler than configuring the interface or hardware MTU. However it can be hard to determine which MTU is configured on an interface and what value it takes.

• You cannot assume that two platforms calculate IP MTU in the same way. 
• You need to clarify exactly what is meant by MTU per platform. Use the configuration guides for your router to
• Once again, awareness of the potential confusion is the key.

[amazon_link asins=’0201634724,B008K9ADRI,1587144239,0596002750,B00RN14G7W’ template=’ProductCarousel’ store=’seamlnetwo-20′ marketplace=’US’ link_id=’c3a556d7-f3f1-11e7-aa67-1d78eab3d1da’]