Switchable PDUs

In my day job I work as a network test engineer and support a remote lab.  We have  a few devices connected to network switchable PDUs that allow us to remotely reset the power outlet of test devices.  Test switches and routers get ‘borked’ pretty often so these switchable PDUs can come in really handy.

The ruby/expect automation solution

Sadly, the CLI on the APC PDUs are menu-driven and is a real pain in the butt.  Aha… enter ‘captain complicated’ with an overly elaborate and exotic solution.  I’ve been trying to learn Ruby and was happy to get the opportunity to practice and hack up a solution.
The final script takes the PDU IP address and power outlet number as inputs, telnets to the PDU,  walks through the menu and resets the power for the device connected to that outlet:

#!/usr/bin/ruby
require 'pty'
require 'expect'
ip = ARGV[0]
outlet_id = ARGV[1]
# Spawn new PTY session
PTY.spawn( 'telnet ' + ip ) do |read,write,pid|
write.sync = true
$expect_verbose = true
read.expect(/User Name : $/,3)
write.print "#######\r"
read.expect(/Password : $/)
write.print "#######\r"
read.expect(/^>.*/,2)
write.print "1\r" #Type 1 for "Device Manager"
read.expect(/^>.*/,2)
write.print "2\r" #Type 2 for "Outlet Management"
read.expect(/^>.*/,2)
write.print "1\r\r" #Type 2 for "Outlet Control/Configuration"
read.expect(/^>.*/,2)
write.print "#{outlet_id}\r\r" #Enter the required outlet number"
read.expect(/^>.*/,2)
write.print "1\r" #Select 1 to control the outlet"
read.expect(/^>.*/)
write.print "3\r" #Select 3 for immediate reboot"
read.expect(/cancel.*:/) #Select 1 to control the outlet"
write.print "YES\r"  # outlet has now been power cycled
read.expect(/Press <ENTER> to continue.*/) #press enter to return to menu
write.print "\r"
read.expect(/^>.*/)
end # end pty spawn

It’s not pretty (or robust, or very secure for that matter), but it works.  I was pretty proud of myself and I was telling a co-worker about this the following day.  My co-worker casually mentioned that she had used SNMP writes in her last job to do the same thing.

The SNMP write solution

Sure enough, a little research and MIB browsing revealed that you could use a one liner command from an bash to achieve the same outcome.  The integer 3 value means ‘immediate reboot’ and corresponds to menu option 3 above.

snmpset -c $snmp_write_community $pdu_ip_address 1.3.6.1.4.1.318.1.1.12.3.3.1.1.4.$outlet integer 3

Cool!  I’m going to keep expanding my scripting skills, but hopefully this will serve as a useful reminder not to reinvent the wheel and that SNMP is still a very useful tool.  Ivan Pepelnjak has has been discussing ‘SNMP as a config tool’ quite a bit recently.  Check out his recent post about snmp over at ipspace.net

Disclaimer

Please remember although this is a great solution for my lab, you’ll need to carefully consider the availability risk vs. reward if you deploy switchable PDU’s in a live environment.
It’s all too easy to have a situation where the PDU’s ethernet connected to a switch which is powered off the PDU.  Sounds daft, but it’s possible to saw off the branch you’re sitting on.   I use the ‘reset’ command rather than ‘power-off’, so at least most mistakes are recoverable.