Data center cooling and large chassis

As a network engineer you need to be aware of the data-center environment where your chosen device will be deployed.  A huge wedge of the cost of running a datacenter are spent trying to keep it cool.  So preserving hot-aisle and cold-aisle airflow containment is a big deal for your data-center manager.  But it’s pretty easy to order hardware that messes with the datacenter airflow.  You need to watch for context and read the fine print to avoid unneeded data-center headaches.

Take the Nexus 7018,  which has both ‘front-to-back’ and ‘side-to-side’ airflow.  This side-to-side component is common in a large chassis, but causes problems.  If you deploy these suckers densely in a datacenter they will each feed their hot exhaust into it’s neighbors supposedly-cold air-intake.  The result would be self-shutdowns or component damage.  So the mitigation here is to deploy a specific cabinet (thanks FryGuy) that converts side to side to front-to-back airflow.  It’s frustrating to have to buy a special cabinet and burn an extra 50% floor space though.  The main point is is that you must be on top of your airflow management before you purchase.

Airflow in high-density 1RU switches

Thanks to the merchant silicon vendors such as Broadcom and Intel, it’s now standard to pack 64 x 10G ports into a a single rack-unit (RU) network device.  These devices throw off a lot less heat than a chassis, but that heat still needs to be managed.

Let’s have a look at three switches based on the same Trident+ chipset;  Cisco Nexus 3064,  Juniper QFX3500 and Arista 7050S.
These switches are targeted for top-of-rack (TOR) deployments, in racks with 10Gbps servers. Servers generally present their NICs at the ‘rear’ of the rack to hide the messy cabling and stuff.   It makes sense to deploy TOR switch in a way that ensures it’s ports are also at the rear of the rack.
However, these switches could also be deployed as aggregation layer switches in smaller datacenter environments.  In this case they would be deployed in a network-only rack, where it would be preferable to have the cabling at the front to stay in line with other networking gear.
Thankfully all the vendors have thought of this and will ship a ‘front-to-back’ and ‘back-to-front’ airflow master-SKU for their devices.

What does front-to-back mean?

Using terminology like ‘front-to-back’ assumes a common understanding on where the front of the device is.  I always thought the ‘front’ of a networking device was the bit with all the ports and blinkly-lights and stuff.   Emm..  bad assumption.
For the Cisco Nexus 3064 the ‘front’ of the device is the side with the fans, power supplies and management interfaces.  The ‘back’ of the device as the side with the ports.   Arrrggghh!!  However, it does allow you to order the ‘default part code’ which has front to back airflow’ designed to be deployed in a TOR environment with it’s ports nearest the servers’ NICs.

Port-side and FRU-side

To tackle the front/rear ambiguity, Juniper uses the terms ‘port’ or ‘FRU’ side in the datasheet for the QFX 3500.  For example a TOR would use ‘port-side exhaust’.  Kudos to Juniper for ensuring there is no ‘default’ airflow assumed.  It would be sweet if they would provide a master-SKU for each airflow option, rather than choosing it for every FRU type though.

What about Arista?

I have to give credit where it’s due.  Arista tackled this issue elegantly for it’s 7050 switch by explaining the options clearly. Like Juniper, they don’t assume any default airflow.
You are forced to make a choice between two master-SKU’s;  DCS-7050S-64-F (front-to-back) or DCS-7050S-64-R (rear-to-front).  See the 7050 datasheet
Arista also preserves the definition of ‘Front’ being port-side and ‘Rear’ being PSU-side.
Lastly,  Arista uses color coded PSUs and fans using classic design principles. Red for hot (exhaust), blue for cold (intake).  Then they show you a picture on their datasheet.  Thank you Arista.

 
 
 
 
 

Take-aways

• Airflow is important, get it wrong and you’ll burn cash or melt hardware
• Always ensure you know the airflow of your device before you order hardware.
• Double check your BoM to ensure all parts and FRU’s have consistent airflow.
• Use terms like port-side and FRU-side, as the terms front and back are open to interpretation.