Sunday, March 1, 2015

Multichannel Professional Data Logger on Raspberry Pi - Part 3

See the Part 1 for the overall description and Part 2 for the hardware implementation.


The software for Data Logger based on a super handy WebIOPi Internet of Things framework. I would suggest to check their website for more details. We will use version 0.7 of the framework.
The WebIOPi basically consists from a python-based web server, a set of native libraries to communicate with Raspberry Pi and REST interface. Additionally, the WebIOPi supports a variety of analog and digital converters, port expanders and sensors. Installed "from the box", it would allow you to communicate with Raspberry Pi from web, update and retrieve data from your devices, and code a basic UI using the javascript extensions and python macros. Also it has a friendly Apache 2.0 license. Our Data Logger custom UI will rely on WebIOPi REST interface to periodically retrieve data from our 16 ports over two ADC devices. We will have a set of python macros to supply the proper REST endpoints.

I will omit all WebIOPi installation details (see their website). From the WebIOPi config standpoint, we need to configure two GPIO ports to automatic pull-up in OUT mode. This two ports will be used in low-power detection logic. The state of this two ports will be periodically analyzed, and if port 18 is "0" - the Data Logger will start playing an annoying alert beeps (means the battery is low), and right after the port 23 will get to "0" (means the battery is almost dead) - the Data Logger will loudly announce the "initiate of shutdown sequence" and power off.
Also we need to configure and enable two ADC devices. The reference voltage is 2.5 V.

# Initialize following GPIOs with given function and optional value
# This is used during WebIOPi start process
18 = OUT 2
23 = OUT 2

adc0 = MCP3008 chip:0 vref:2.5
adc1 = MCP3008 chip:1 vref:2.5

We will supply our own custom python script which runs by WebIOPi framework and register it in the WebIOPi config in "[SCRIPTS]" section. The script implements several mandatory methods: setup(), destroy() and loop() which are called by WebIOPi at script loading, at server shutdown, or periodically. Also we have to add in this script a several REST endpoints as a python macros (annotated as @webiopi.macro) in code:
  • getAvailableSpace() - return a free space in MB on Raspberry Pi;
  • toC() - serialize a user configuration settings on file system;
  • getDate() - return a local date and time on Raspberry Pi;
  • getTempSimple() - return a current set of values from Temperature ADC in JSON format;
  • getVISimple() - return a current set of values from Voltage/Current ADC in JSON format;
  • startRead() - enable reading and serialization of values from both ADCs;
  • stopRead() - disable reading and serialization;
The REST endpoints were called by jQuery handlers on UI side. Below is a screenshot of Configuration screen. Notice the familiar Bootstrap look-and-feel.

The UI includes two base screens - Configuration and Status (below). On the Configuration screen you can select the types and enter the names of your input ports (two sections: Temperature Channels and V & I Channels), specify "Recording Mode". It means mostly how long you would like the Data Logger to run - until data storage fills up or until trigger signal received on GPIO port (currently disabled). The data store overfill is very hard to achieve, we had Data Logger with common 4GB SD card running successfully for several days with the most aggressive settings. Another option is "Sampling Interval" - the number of seconds between data collection events (1...60). The last section is "Start Mode" - you can start the Data Logger immediately after saving the settings, by specifying a date and time, or manually by selecting a menu option on Status screen.

The Status screen is what you will be looking on the most of your time. It displays a mesmerizing live feed of data from sensors with the intervals previously defined on Configuration screen. The graph is based on d3 framework , with the Rickshaw JavaScript toolkit on top - to make a graph coding simpler (for example note the roller on the graph horizontal axis, this is just three lines of code). The table grid is also represent a live data, channels can be filtered out by Search field - thanks to the DataTables JQuery plugin.

The menu system on Status screen is very simple. The Export option allows you to download the CSV file with the collected data. Note that when a configuration gets updated on Configuration screen, the CSV files are rewritten, so you always get a latest data. If you need to keep the recording results for future - make sure that you export the data before updating configuration.
If on Configuration screen for Starting Mode you selected "Start Menu on Status Screen" (like on the screenshots), you will get one additional menu option named "Action". There you can pick "Start" or "Stop" option to control the logging process manually.

The last moment worth to mention is that the Data Logger also works as a Wireless Router and Access Point, since the easiest way to get the data from it is to use Web UI over Wi-Fi. For the detailed instructions I'll redirect to the nice article which I followed with a small variations. The Raspberry Pi is a regular Linux box, so basically it should have a DHCP server, Wireless Access Point, Routing and IP-Forwarding services configured. With a proper (Realtek-based) Wi-Fi dongle it work flawlessly in the field, and can provide internet to your tablet, laptop or phone in lab if you plug Ethernet cable from your cable or modem in the Raspberry Pi jack.


  1. Hello,

    Very nice job !

    I need to build a logger like yours, but with I2C, SPI and serial devices.
    Is it possible to get your software sources in order to have a look around and test it ?


    1. the logger software basedon WebIOPi, here is the download link:
      Feel free to download and play with it, AFAIK the WebIOPi supports I2C, SPI and serial protocols, as well as many devices.

      I've only added a javascript forms to configure WebIOPi-basedbackend and included some graphs based on D3.js framework. Nothing really complicated.
      Here is the screen capture of Web Interface I made recently:

  2. Thanks a lot, I will test it soon !


Note: Only a member of this blog may post a comment.