A lot of bit of nothing

As it sometimes happens personal stuff has taken hold of my life and stopped me from doing anything major with anything technology related. I decided that I should pick a little project to pick up some new skills, so I’ll be setting up Cisco’s AIR-CTVM Wireless controller along with a couple LAP-1142Ns 802.11n (draft) access points that I picked up off of eBay to get rid of the DD-WRT APs which haven’t been entirely cooperative. For example, the Netgear WNR834B v2 will only use the base channel assigned with the second channel being two channels above it (currently channels 6 and 8) which is clearly not optimal for throughput.

I’m going to be rearranging my home network to segment it a bit more and do some more with routing. I want to keep the LAPs running off the 3560-24PS with PoE power instead of powering them with external bricks, so unfortunately each AP will be limited to 100mbit of throughput — that’s actually still better than what I get now over the 2.4GHz N AP, so it’ll still be a usable throughput improvement.

I’ll also be able to actually do some L3 segmenting instead of needing to share a VLAN across physical boundaries for the ‘dumb’ AP bridges currently in place.

I’ve been doing some work on IP management software, and while a lot of the back-end functionality is currently there for calculation, I’d like to rewrite some of it for speed. There are parts that are written strictly for readability using strings instead of bit compares, and they’re much slower than I’d like them to be for large address spaces. I should have something interesting to show if I can manage to put a little more time into it.

PHP, Screen Scraping & SevOne Deferred Data — or a Network Operator/Engineer That Can Do More Than Networks! (Possible Rant)

This post is semi work related, but I do have SevOne running at home as my NMS for graphing, trending and alerting. There are some statistics that I insert via their ‘deferred data,’ which is done through a (fairly horrible) SOAP API. Before Cablevision I hadn’t written a line of PHP, or used SOAP in any language — but all of the SevOne examples were written in PHP. I picked it up to write some scripts that insert data from CMTS (Cable Modem Termination Systems) that were not available  through SNMP which was really helpful in our monitoring and alerting.  As I do have some kind of programming background it wasn’t difficult to pickup the ‘beginners,’ scripting language. Eventually I wrote a class to encapsulate a bunch of stuff that I do to interface with SevOne as we could really spike the CPU usage if we did things by way of their example scripts.

The entire CMTS script involved me writing a telnet wrapper that uses socket calls and is entirely written in PHP. This was probably a naive approach and a stream to telnet would’ve been much better (as we will be moving to SSH anyway), although it did allow me to flex some critical thought muscles I forgot I had and haven’t been used in some time. After some tuning it’s actually pretty fast and doesn’t present a bottleneck or resource drain. Eventually even the telnet functions got expanded on considerably into a couple of classes for accessing data on IOS and Arris C4. The IOS functions got expanded on to parse data for some critical things for CMTS-specific stuff, then for other pieces of equipment (ie 7600/6500). The base telnet class was later extended to IOS-XR for ASR9K devices. Then NX-OS. A flexible SNMP wrapper was created primarily for polling STB (Set Top Boxes). I got pretty good at writing PHP in a time-efficient manner to parse all kinds of stuff. As a “network operator,” this was great! I could get all kinds of statistics that would normally be “impossible,” or really “time-intensive. The SevOne stuff expanded and got refined as I dug more into my programming background.  Even as I was using a fairly clumsy language without formal training I had a good idea of how to write EFFECTIVE PHP (in my mind anyway). Parts of it might be ugly, but it gets the job done. I became adept at writing regular expressions, I even dynamically generated it based on the output so it would be extremely easy to update in the future and could even deal with a good deal of changes without being updated. I wrote SevOne wrappers to generate (or update) hundreds of threshold alarms at a time that would otherwise take a LOT of man hours to manually input. I wrote scripts to poll SevOne for information so we could figure out how to most effectively utilize our servers, or just to look at devices for certain criteria.. things that we wouldn’t be able to do an effective manner otherwise.

That’s a lot of background, but the end result: I created a great SevOne Deferred Data wrapper that caches a lot of information to cut back on CPU hits to the SevOne servers. It connects to peers dynamically as required to cut back on CPU. It takes a lot of work away from SevOne that might otherwise be (naively) processed on SevOne’s side. I know because I did it. And we pegged CPU cores on our SevOne boxes. We ran into so many issues. Because of that we were unable to do what we needed to do without some of these (admittedly somewhat minor) innovations. For Cablevision it’s amazing. For me, it allows a great amount of work in small amounts of time. All of the SevOne nitty-gritty is abstracted away and done automagically. Things are created on the fly, SOAP calls that might generate an exception (and this happens often) are automatically retried depending on the error. There’s little need for error checking in a script that screen scrapes to collect data. In my mind that’s an example of a great piece of code, it doesn’t make you think about what you’re doing with something. It does what you expect of it (for the most part). When upper management asks for something I look good because I can get it to them extremely quickly. When SevOne says it will take months for their Professional Services division to create something or cost some obscene amount of money, I’m there offering a quick solution which is cheap for them. Maybe that’s not great, but I like doing my job well.

I’m not the best programmer ever. I’m not a scripting pro. I don’t know that many scripting languages, but I can pick them up fairly quickly. I’ve never touched Python. I can whip things up in Ruby (ON RAILS EVEN!) but I’m not a Ruby guru. I can write shell scripts but I’m not extremely proficient in BASH, nevermind TCSH, KSH or any of the others. I’m not really a network engineer. But combining my general purpose background has allowed me to do things that might not otherwise be possible. Certainly your regular CCNP or CCIE carrying network guy is not going to have quite the scripting background to get the data required into an NMS — and your regular programmer is not going to have enough of an idea about network devices or the network in general to have a good understanding of what might need to be collected into the NMS.  At home I want to monitor some things on my VM server that aren’t available through SNMP. I could write scripts and custom OIDs to access them through SNMP but that requires configuration on the SevOne side. It’s much faster for me to use my wrapper to insert the data I need (a good example — CPU wattage).

I realize this post is more of a rant than usual, but feel free to check the wrappers out at SevOne PHP Wrappers



© 2017 Musings

Theme by Anders NorenUp ↑