Learning to Love Test-Driven Development

I'm a pretty decent programmer (OK, maybe I'm a bit rusty since my primary job responsibility isn't development). One facet that I've ignored though is the value of rigorously testing my code. Oh, sure, I run my code and try various things to make sure it does what I want under normal circumstances. But that's not the same as writing formal tests that cover a wide variety of inputs and conditions that your code needs to deal with. Taken to the extreme, test code should be written first, and then the "real" code gets written.

I haven't done much with writing test cases because the code that I write these days for the most part are short scripts for automating day-to-day tasks. Also, writing test code is something of a mindset that you have to practice.

I'm going to work a lot harder at writing test code, partly because it'll improve the code that I write for my own use, but also because many open source projects that I contribute to require tests for any new code (or for any old code that gets changed).

Monitoring BGP with Reconnoiter

Here's another quick Reconnoiter tip for monitoring BGP on Cisco routers. You'll need to make sure that the IETF standard BGP4-MIB and the Cisco CISCO-BGP4-MIB are installed where net-snmp can find them. Then add the following XML to your noit.conf file.

    <bgp4 module="snmp" period="60000" timeout="15000">
      <config>
        <community>public</community>
        <oid_peer_identifier>BGP4-MIB::bgpPeerIdentifier.%[:ccns:name]</oid_peer_identifier>
        <oid_peer_state>BGP4-MIB::bgpPeerState.%[:ccns:name]</oid_peer_state>
        <oid_peer_admin_status>BGP4-MIB::bgpPeerAdminStatus.%[:ccns:name]</oid_peer_admin_status>
        <oid_peer_negotiated_version>BGP4-MIB::bgpPeerNegotiatedVersion.%[:ccns:name]</oid_peer_negotiated_version>
        <oid_peer_local_addr>BGP4-MIB::bgpPeerLocalAddr.%[:ccns:name]</oid_peer_local_addr>
        <oid_peer_remote_addr>BGP4-MIB::bgpPeerRemoteAddr.%[:ccns:name]</oid_peer_remote_addr>
        <oid_peer_remote_as>BGP4-MIB::bgpPeerRemoteAs.%[:ccns:name]</oid_peer_remote_as>
        <oid_peer_in_updates>BGP4-MIB::bgpPeerInUpdates.%[:ccns:name]</oid_peer_in_updates>
        <oid_peer_out_updates>BGP4-MIB::bgpPeerOutUpdates.%[:ccns:name]</oid_peer_out_updates>
        <oid_peer_in_total_messages>BGP4-MIB::bgpPeerInTotalMessages.%[:ccns:name]</oid_peer_in_total_messages>
        <oid_peer_out_total_messages>BGP4-MIB::bgpPeerOutTotalMessages.%[:ccns:name]</oid_peer_out_total_messages>
        <oid_peer_last_error>BGP4-MIB::bgpPeerLastError.%[:ccns:name]</oid_peer_last_error>
        <oid_peer_fsm_established_transitions>BGP4-MIB::bgpPeerFsmEstablishedTransitions.%[:ccns:name]</oid_peer_fsm_established_transitions>
        <oid_peer_fsm_established_time>BGP4-MIB::bgpPeerFsmEstablishedTime.%[:ccns:name]</oid_peer_fsm_established_time>
        <oid_peer_connect_retry_interval>BGP4-MIB::bgpPeerConnectRetryInterval.%[:ccns:name]</oid_peer_connect_retry_interval>
        <oid_peer_hold_time>BGP4-MIB::bgpPeerHoldTime.%[:ccns:name]</oid_peer_hold_time>
        <oid_peer_keep_alive>BGP4-MIB::bgpPeerKeepAlive.%[:ccns:name]</oid_peer_keep_alive>
        <oid_peer_hold_time_configured>BGP4-MIB::bgpPeerHoldTimeConfigured.%[:ccns:name]</oid_peer_hold_time_configured>
        <oid_peer_keep_alive_configured>BGP4-MIB::bgpPeerKeepAliveConfigured.%[:ccns:name]</oid_peer_keep_alive_configured>
        <oid_peer_accepted_prefixes_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerAcceptedPrefixes.%[:ccns:name].ipv4.unicast</oid_peer_accepted_prefixes_ipv4_unicast>
        <oid_peer_accepted_prefixes_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerAcceptedPrefixes.%[:ccns:name].ipv6.unicast</oid_peer_accepted_prefixes_ipv6_unicast>
        <oid_peer_denied_prefixes_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerDeniedPrefixes.%[:ccns:name].ipv4.unicast</oid_peer_denied_prefixes_ipv4_unicast>
        <oid_peer_denied_prefixes_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerDeniedPrefixes.%[:ccns:name].ipv6.unicast</oid_peer_denied_prefixes_ipv6_unicast>
        <oid_peer_prefix_admin_limit_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerPrefixAdminLimit.%[:ccns:name].ipv4.unicast</oid_peer_prefix_admin_limit_ipv4_unicast>
        <oid_peer_prefix_admin_limit_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerPrefixAdminLimit.%[:ccns:name].ipv6.unicast</oid_peer_prefix_admin_limit_ipv6_unicast>
        <oid_peer_prefix_threshold_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerPrefixThreshold.%[:ccns:name].ipv4.unicast</oid_peer_prefix_threshold_ipv4_unicast>
        <oid_peer_prefix_threshold_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerPrefixThreshold.%[:ccns:name].ipv6.unicast</oid_peer_prefix_threshold_ipv6_unicast>
        <oid_peer_prefix_clear_threshold_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerPrefixClearThreshold.%[:ccns:name].ipv4.unicast</oid_peer_prefix_clear_threshold_ipv4_unicast>
        <oid_peer_prefix_clear_threshold_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerPrefixClearThreshold.%[:ccns:name].ipv6.unicast</oid_peer_prefix_clear_threshold_ipv6_unicast>
        <oid_peer_advertised_prefixes_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerAdvertisedPrefixes.%[:ccns:name].ipv4.unicast</oid_peer_advertised_prefixes_ipv4_unicast>
        <oid_peer_advertised_prefixes_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerAdvertisedPrefixes.%[:ccns:name].ipv6.unicast</oid_peer_advertised_prefixes_ipv6_unicast>
        <oid_peer_suppressed_prefixes_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerSuppressedPrefixes.%[:ccns:name].ipv4.unicast</oid_peer_suppressed_prefixes_ipv4_unicast>
        <oid_peer_suppressed_prefixes_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerSuppressedPrefixes.%[:ccns:name].ipv6.unicast</oid_peer_suppressed_prefixes_ipv6_unicast>
        <oid_peer_withdrawn_prefixes_ipv4_unicast>CISCO-BGP4-MIB::cbgpPeerWithdrawnPrefixes.%[:ccns:name].ipv4.unicast</oid_peer_withdrawn_prefixes_ipv4_unicast>
        <oid_peer_withdrawn_prefixes_ipv6_unicast>CISCO-BGP4-MIB::cbgpPeerWithdrawnPrefixes.%[:ccns:name].ipv6.unicast</oid_peer_withdrawn_prefixes_ipv6_unicast>
      </config>
      <rtr1 target="172.16.0.1">
        <check uuid="a75d0260-0631-461d-ac0f-21a3ebd0ae02" name="rtr1:bgp:peername::192.168.0.1"/>
      </rtr1>
    </bgp4>

Above, 192.168.0.1 is the IP address of the peer router and 172.16.0.1 is the IP address of your router.

Monitoring a F5 BIG-IP LTM with Reconnoiter

Recently we purchased an F5 BIG-IP LTM - basically a very fancy web load balancer - to provide some redundancy and scalability to some of our web services at work. I've also been looking at Reconnoiter for monitoring our network. I'm not going to provide a complete review of either product here, or even a complete how-to on setting this up, but here are some tips on monitoring your LTM with Reconnoiter:

First, download the F5 mibs from your LTM. To do that log into your LTM and navigate the menus to Overview->Welcome. Also, configure SNMP on your LTM if you haven't already. There will be a section called "Downloads" that has the SNMP MIBs. Download those MIBs and install them on the server that runs your Reconnoiter noitd service. Read through the MIB for documentation on what the various metrics mean.

Then, add the following to the checks section of your noitd configuation file.

    <f5 module="snmp" period="60000" timeout="15000">
      <config>
        <community>community</community>
        <oid_description>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatName."%[:ccns:name]"</oid_description>
        <oid_csminconndur>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatCsMinConnDur."%[:ccns:name]"</oid_csminconndur>
        <oid_csmaxconndur>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatCsMaxConnDur."%[:ccns:name]"</oid_csmaxconndur>
        <oid_csmeanconndur>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatCsMeanConnDur."%[:ccns:name]"</oid_csmeanconndur>
        <oid_nonodeserrors>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatNoNodesErrors."%[:ccns:name]"</oid_nonodeserrors>
        <oid_clientpktsin>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientPktsIn."%[:ccns:name]"</oid_clientpktsin>
        <oid_clientbytesin>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientBytesIn."%[:ccns:name]"</oid_clientbytesin>
        <oid_clientpktsout>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientPktsIn."%[:ccns:name]"</oid_clientpktsout>
        <oid_clientbytesout>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientBytesOut."%[:ccns:name]"</oid_clientbytesout>
        <oid_clientmaxconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientMaxConns."%[:ccns:name]"</oid_clientmaxconns>
        <oid_clienttotconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientTotConns."%[:ccns:name]"</oid_clienttotconns>
        <oid_clientcurconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatClientCurConns."%[:ccns:name]"</oid_clientcurconns>
        <oid_ephemeralpktsin>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralPktsIn."%[:ccns:name]"</oid_ephemeralpktsin>
        <oid_ephemeralbytesin>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralBytesIn."%[:ccns:name]"</oid_ephemeralbytesin>
        <oid_ephemeralpktsout>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralPktsOut."%[:ccns:name]"</oid_ephemeralpktsout>
        <oid_ephemeralbytesout>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralBytesOut."%[:ccns:name]"</oid_ephemeralbytesout>
        <oid_ephemeralmaxconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralMaxConns."%[:ccns:name]"</oid_ephemeralmaxconns>
        <oid_ephemeraltotconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralTotConns."%[:ccns:name]"</oid_ephemeraltotconns>
        <oid_ephemeralcurconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatEphemeralCurConns."%[:ccns:name]"</oid_ephemeralcurconns>
        <oid_pvapktsin>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaPktsIn."%[:ccns:name]"</oid_pvapktsin>
        <oid_pvabytesin>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaBytesIn."%[:ccns:name]"</oid_pvabytesin>
        <oid_pvapktsout>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaPktsOut."%[:ccns:name]"</oid_pvapktsout>
        <oid_pvabytesout>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaBytesOut."%[:ccns:name]"</oid_pvabytesout>
        <oid_pvamaxconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaMaxConns."%[:ccns:name]"</oid_pvamaxconns>
        <oid_pvatotconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaTotConns."%[:ccns:name]"</oid_pvatotconns>
        <oid_pvacurconns>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatPvaCurConns."%[:ccns:name]"</oid_pvacurconns>
        <oid_totrequests>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatTotRequests."%[:ccns:name]"</oid_totrequests>
        <oid_totpvaassistconn>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatTotPvaAssistConn."%[:ccns:name]"</oid_totpvaassistconn>
        <oid_curpvaassistconn>F5-BIGIP-LOCAL-MIB::ltmVirtualServStatCurrPvaAssistConn."%[:ccns:name]"</oid_curpvaassistconn>
      </config>
      <bigip-1 target="10.1.1.1">
        <check uuid="c3b664fc-2bed-433a-bf59-6dfc65d7830c" name="bigip-1::owa_v4_https_virtual_server"/>
        <check uuid="5359942e-77c2-4930-bcdb-482d8ec5c338" name="bigip-1::owa_v4_https_virtual_server"/>
      </bigip-1>
    </f5>

The emphasized items above need to be customized for your site, of course. Once you've restarted the noitd service you can create graphs of the various statistics that are gathered. Here's one that I created for our Exchange 2010 server:

It's working!

With the first real snow of the season threatening, I figured I should gas up the snow blower and make sure it starts up. After a few unsuccessful tries, I let the snow blower sit for a while. After I came back after thirty minutes or so it started up on the 2nd pull. Maybe I flooded it the first time or maybe it just took a while for the gas to work it's way though the lines. In either case it looks like I'll avoid the heavy shovelling for another season.

The Asterisk SLN16 Codec

Google failed me, so I figured that I'd memorialize it here... Here's how to convert to/from Asterisk's SLN16 format using GStreamer:

From SLN16:

gst-launch filesrc location=<filename>.sln16 ! audio/x-raw-int,signed=true,rate=16000,channels=1,width=16,depth=16,endianness=1234 ! audioconvert ! vorbisenc ! oggmux ! filesink location=<filename>.ogg

To SLN16:

gst-launch filesrc location=<filename>.ogg ! decodebin ! audioconvert ! audiorate ! audio/x-raw-int,signed=true,rate=16000,channels=1,width=16,depth=16,endianness=1234 ! filesink location=<filename>.sln16

Generating Random Passwords With Python

Every now and then, I need to generate a password for use with something or other that I won't be typing in very often. For maximum security in these situations I like generating a long randomly generated password. Here's a quick Python snippet for generating them:

$ python
Python 2.6.2 (r262:71600, Aug 21 2009, 12:23:57) 
[GCC 4.4.1 20090818 (Red Hat 4.4.1-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> file('/dev/random', 'rb').read(12).encode('base64')
'XIUhRiMM/kcmsQl8\n'
>>> file('/dev/random', 'rb').read(12).encode('base64')
'BkgURdSXKORS8In/\n'
>>>

It's such a short snippet that I haven't ever bothered putting it in a script file, I just fire up a Python interpreter and type it from memory.

ADSL Bummer (and Closed Source Object Lesson)

Vyatta announced an alpha release of version 6 of their Linux-based router software. I've been using Vyatta version 5 since it was in beta test because they have integrated support for my Sangoma S518 ADSL Modem Card. Before Vyatta version 5 I was using Fedora and manually installing Sangoma's drivers. However, Sangoma's drivers contain binary blobs and I was never able to keep my router from locking up every few days. Vyatta's engineers were able to figure out how to keep my router stable in spite of the binary blobs, which made me very happy. I had Linux on my router/firewall and I didn't have to switch from ADSL to cable modem. My wife was happy because the router didn't crash every couple of days. About the only downside was the confusion my non-standard setup caused my ISP's tech support the few times that I had to call them.

Unfortunately Vyatta will be removing ADSL support from the free version of their software in version 6. I could get a software subscription from Vyatta and keep the ADSL support, but at $750 even the cheapest subscription is way more than I can justify. For now I'll stick with version 5 and keep my ADSL going, but eventually I'll have to look into a different solution.

As far as I know, there are no solutions for ADSL interfaces that have open-source drivers available in the US. If there is a solution I haven't found it yet. I'd rather not go back to installing the Sangoma drivers on top of a standard Fedora install, although that would mean that my router would be based on Fedora again rather than the Debian base that Vyatta uses. Nothing against Debian, but I'm much more comfortable with Fedora since I use it daily on my laptop and other systems I manage. I'd really rather not go back to a cable modem, as I left our cable company for ADSL because I didn't care for how the local cable company did business.

I suppose it's too much to hope that someone will deliver fiber to my door in the next few months...