Part 3: Splunk LED Meter – Making it all work

Posted on by
Reply

Part 1 covered hardware assembly, part 2 covered accessing the Splunk API using PowerShell, so now it’s time to complete this project.  To do so, we need a script to send commands to the USB controller and a “driver” script to loop and call the PowerShell script that communicates with the Spunk API and the one that sends commands to the USB controller.

There’s a great example available on the Teensy website that demonstrates using PowerShell to communicate with the Teensy.  My SendSerialCommand.ps1 is partly based on this example.

Script: SendSerialCommand.ps1

Since we want the LED meter to continually update, I created another PowerShell script that’s in charge of calling SplunkSearchGauge.ps1 and SendSerialCommand.ps1; this script is considered the “driver” and will be the one you launch from the PowerShell terminal.

Script: Main.ps1

Finally, if you’re interested in testing the LED light strip, this script will light up each light from 1-32 and then turn them back off from 32-1.

Script: TestGauge.ps1

    Part 2: Splunk LED Meter – Hardware Assembly

    Posted on by
    Reply

    Part 1 covered using PowerShell to issue Splunk REST API search commands.  Now it’s time to attach the Teensy USB controller to the LED light strip.

    The text in the picture below corresponds to the text on the light strip.  Using some wire solder the connections on the light strip to the associated Teensy outputs (ensure you are making the connections to the DI and CI, and not the DO and CO solder pads on the LED light strip).  Honestly, I’m not sure why I selected the Teensy outputs I did, but it works.

    OK, now that the light strip is connected to the Teensy it’s time to program the Teensy.  As mentioned in Part 1 I’m using Arduino, Teensyduino and the libraries supplied by Adafruit.

    Once you have everything installed, you should be able to run the strandtest Sketch available in Arduino under File > Examples > LPD8806 > strandtest.  This Sketch really shows off the power of the light strip.  If this works, great.  If not, not great.  Once working it’s time to show off how little the light strip can do with my splunkmeter code. :)

    The splunkmeter code accepts a value 0-100 (and 999, explained below) and lights up the LED strip in green (first 12 lights), yellow (next 10 lights) and finally read (last 10 lights).

    The code is not pretty since it was hacked together one day before a demonstration, so there are some known issues (e.g. sending 0 doesn’t turn off the LEDs, you have to send 999; the distribution of lights to a particular percentage is slightly uneven; finally, the huge else if statement is nasty.)

    Here’s the code: splunkmeter.ino

    Part 3 will cover the PowerShell script that will send commands to the Tennsy, another PowerShell script to tie everything together and finally a test PowerShell script to test the meter (used in the video above).

      Part 1: Splunk LED Meter – PowerShell and Splunk

      Posted on by
      Reply

      Sure it’s been done before, but I figured why not build a USB controlled LED light strip to represent some arbitrary Splunk metric.  In my case — simply for demonstration purposes — I wanted to gauge the number of unique visits to a particular website in the past 4 hours as a percentage of what I would consider high (1000 unique IP addresses).

      Here’s what I used/purchased:

      Currently I’m waiting for the hardware to show up, so I figured I would start with the PowerShell component of this project.  Splunk offers a set of REST APIs capable of doing just about anything Splunk related.  However, all I’m interested in is issuing a search and retrieving the results, so I’ll use the search calls.

      The logic of the PowerShell script is really simple: Issue a search, pause, get the SID (search ID), check the status of the search, pause, check again, and eventually retrieve the results once complete.  For this project my Splunk search is returning one numeric value named “gauge” which gets set as a global variable.  The Splunk search actually calculates the percentage (c_ip/1000*100)  Later this value will be sent to the Teensy board by another PowerShell script.

      Here’s the script.  It’s pretty basic and doesn’t do much (any!) error handling, but hey what’s the fun in error handling.  You’ll need to change the server, port, search, splunkuser and splunkpass variables.  Keep in mind my search always returns one value, in one column named gauge.

      Script: SplunkSearchGauge.ps1

      Part 2 will cover hardware assembly and the code for the USB controller.

        Google Fusion Tables and KML Polygons

        Posted on by
        Reply

        Google Fusion Tables (Fusion) makes it easy to visualize relatively simple data on a Google Map, assuming some portion of your data is or can be geocoded.

        Recently, I needed to visualize California county level data on a Google Map.  My requirements were pretty simple:

        1. Outline the counties
        2. Fill county polygons with a color representative of the data
        3. Provide an informational pop-up when a county is clicked

        Since the requirements were pretty simple I decided to give Fusion a shot.  Fusion allows you to import a Google Docs Spreadsheet as a datasource, so that’s the route I took.

        In order to outline the California counties I turned to Fusion’s ability to process KML.  Using the KML polygon I was able to provide standard polygon coordinates (derived from Census data), which ended up being my “location” source for my data.

        Fusion offers several options for formatting polygons on the map.  In my case I used a number in my import file to color the counties/polygons creating a semi choropleth map.

        Finally, Fusion allows you to format your informational popups using a combination of data from your datasource and HTML.

        If interested, I’ve created a sample Google Docs spreadsheet that contains polygons for California’s 58 counties.  Note: The county_name in the sample_data sheet is being used to lookup the kml_polygon in the ca_counties sheet.

          Visualizing County Level Data

          Posted on by
          Reply

          A few weeks ago I picked up a copy of Visual This at the bookstore.  Overall I thought the book was great and was particularly interested in an example that used FIPS codes to create a choropleth map using a public domain map from WikiMedia.

          The hardest part was creating an appropriate dataset.  The map itself is an SVG image, which is nothing more than an XML formatted text file.  Each county has an associated id which just happens to be its FIPS code.  So if you’re able to match up a piece of your data, in my case the zipcode, to a FIPS code you’re almost there.

          I’m pretty happy with the results.  I ended up using Inkscape to tweak the SVG file before sharing it — not represented here.

          Update: Here’s a link to a tutorial from the book’s author.  Wish I would have found it first!

            Parsing Files with Python

            Posted on by
            Reply

            Recently, I’ve had to work with a lot of data stored in various text files: comma and tab delimited, fixed width, etc… So I decided, based on a little research, Python would be my tool of choice to parse and manipulate these files prior to inserting them into an Oracle database.

            My most recent task: extract several columns from a fixed width file with over 22 million lines, remove leading zeros from two of the columns (making them a proper number) and write the results to a new CSV file.

            Here’s what I came up with — it’s quick and dirty, but works.

            Python code:

            import re
i = open("zipcty10")
o = open("10.csv","w")
line = i.readline()
while line:
    print >>o,line[0:5] + "," + re.sub(r"^[0]*","",line[15:19].strip()) + "," + re.sub(r"^[0]*","",line[19:23].strip()) + "," + line[23:25] + "," + line[25:28] + "," + line[28:53].strip()
    line = i.readline()
i.close()
o.close()

              2011 Chicago Marathon

              Posted on by
              Reply

              I’ve been contemplating running the Chicago Marathon for a while, and today I finally decided to do it!  The marathon is Sunday, October 9, 2011.

              I’ve decided to register/run under a charity/non-profit entry for the United Parent Support for Down Syndrome (UPS for DownS for short).  This mean I need to raise $950 that will go to the Katie MacDonald Literacy Project.

              This give me about 5 months to train and raise the needed $950.  So if you are feeling charitable, please make a donation using my Active.com fundraising page.  100% of your donation goes to the Katie MacDonald Literacy Project.

              My goal is to keep this site up to date with my training and fundraising progress, so please sign up for email updates.

              Thanks for your support.

              Mike