Raspberry PI, sensors and DIoTY MQTT cloud

In a previous post I showed how to connect your Arduino to DIoTY's MQTT cloud infrastructure and make your sensor data instantaneously available to any MQTT client.

This time round, I'm going to work with a Raspberry PI and a DHT22 Humidity Temperature Sensor Module.  I will use MQTT to publish the values received from the sensor and use the new and improved DIoTY website to view the how warm and humid it actually is in my office!

We use following hardware for our example:

• Raspberry PI B+ (but any model should do)
• micro SD card (or the one matching your Raspberry PI model)
• USB wifi nano
• Type-T GPIO Expansion Board Accessory for Raspberry Pi B+
• DHT22 Humidity Temperature Sensor Module
• breadboard and some wires

I assume most readers already have their Raspberry PI up and running, but for those who are new to Raspberry PI, here's very brief how you get that far:

1. Download the OS image from http://www.raspberrypi.org/downloads/ 
• I used the Raspbian Debian Wheezy
2. Copy the image to your SD card
• check the volume of your SD card using diskutil list
• unmount the volume (eg if the volume found is called disk1 then  execute diskutil unmountDisk /dev/disk1)
• copy image with sudo dd bs=1m if=2014-09-09-wheezy-raspbian.img of=/dev/disk1 (adapt image name and volume name to match yours)
3. Configure your Rasberry PI
• startup your Raspberry PI with ethernet plugged in and log in with user pi and password raspberry
• start configuration with sudo raspi-config
• expand filesystem
• change pi password to your own
• change timezone to yours under internationalisation
• finish with reboot and log in with your new password
4. Update the OS
• execute sudo apt-get update to get the latest (security) updates
5. Configure the wifi (I'm using a wpa with  a wep key on my router)
• execute sudo vi /etc/wpa_supplicant/wpa_supplicant.conf and add the following network section (filling in your values for ssid and hex web_key0):
  network={
     ssid="MyNetworkName"
     wep_key0=abcdef0123456789
     key_mgmt=NONE
    }
• install wicd-curses
• execute sudo apt-get install wicd-curses
• plug in the wifi stick and configure by running sudo wicd-curses
• configure network --> connect automatically using WEP key

Unlike with Arduino, with a Raspberry PI we can choose what programming language we'd like to use.  As I've been working a lot with Node.js lately and started liking it I'll do this exercise with Node.  

Node is not the ideal language when it comes to real time programming.  In fact, you won't find any pure Node modules that interact with the GPIO.  You do find libraries for this in C however which you can wrap in your Node module.  Even better, as Node.js is so well adopted in the open source community you'll often find those wrapper modules readily available for your sensor... and the DHT22 is no exception (see Node Package Manager website for node-dht-sensor).

So let's get started!

First we need Node and NPM installed.  There are several ways of doing this, but I prefer to do it the safe (or long, whichever you prefer) way.  Installing Node.js is done using the following set of commands:

     sudo apt-get install python g++ wget libssl-dev

     mkdir /tmp/nodejs && cd /tmp/nodejs

     wget http://nodejs.org/dist/node-latest.tar.gz

     tar xzvf node-latest.tar.gz && cd node-v*

     ./configure

     make

     sudo make install

Note that those make commands will take time!

Next install the Node Package Manager:

     curl https://www.npmjs.org/install.sh | sudo sh

The link above on the NPM node-dht-sensor package shows us how to install the module and gives also some sample code on how to use it.  All we have to do is adapting the sample program to start talking to our MQTT cloud service.  And yes, there is also an MQTT package available which does all the hard work on the MQTT side for us.  So lets create a new directory for our project in our home drive and install both packages required:

     cd 

     mkdir dht22 && cd dht22

The node-dht-sensor package requires the bcm2835, so let install that first:

     wget http://www.airspayce.com/mikem/bcm2835/bcm2835-1.38.tar.gz

     tar zxvf bcm2835-1.38.tar.gz

     cd bcm2835-1.38

     ./configure

     make

     sudo make check

     sudo make install

Now we can install both node packages:

     cd ~/dht22

     sudo npm install node-dht-sensor

     sudo npm install mqtt

The node-dht-sensor module comes with a test program located under ~/dht22/node_modules/node-dht-sensor.  Copy test.js to your root project folder ~/dht22 and make the changes as indicated below:

// Module node-dht-sensor demo

// Reads relative air humidity from DHT sensor

var fs = require('fs');

var mqtt = require('mqtt')

  , host = 'mqtt.dioty.co'

  , myUserId = 'xxxxxx@gmail.com'  // your DIoTY userId

  , myPwd = 'xxxxxx'               // your DIoTY password

  , client = mqtt.createClient(1883, host, {username: myUserId, password: myPwd});

var sensorLib = require('./node_modules/node-dht-sensor/build/Release/node-dht-sensor');

var sensor = {

  initialize: function() {

    this.totalReads = 0;

    return sensorLib.initialize(22, 4);

  },

  read: function() {

    var readout = sensorLib.read();

    this.totalReads++;

    console.log('Temperature: '+readout.temperature.toFixed(1)+'C, humidity: '+readout.humidity.toFixed(1)+'%'+

                ', valid: '+readout.isValid+

                ', errors: '+readout.errors);

    fs.appendFile('log.csv',

      new Date().getTime()+','+readout.temperature+','+readout.humidity+',"'+(readout.checksum ? 'Ok' : 'Failed')+'",'+

readout.errors+'\n',

      function (err) { });

    if (readout.isValid && readout.checksum) {

      client.publish('/'+myUserId+'/office/temp', readout.temperature.toFixed(1).toString(), {retain: true});

      client.publish('/'+myUserId+'/office/humidity', readout.humidity.toFixed(1).toString(), {retain: true});

    }

    if (this.totalReads < 300) {

      setTimeout(function() {

        sensor.read();

      }, 500);

    }

  }

};

if (sensor.initialize()) {

  sensor.read();

} else {

  console.warn('Failed to initialize sensor');

}

We will run this with the command sudo node test,js but let's first look at the hardware side.  This as well is pretty easy.  The dht22 sensor module I have comes with a 4.7kOhm resistor build in, but if that's not the case for you then you should put one in between the VCC and Data line of your sensor.  Apart from that we have just 3 connections to make:

• Connect the VCC (+) of your sensor to the 3.3V (pin 1) or your Raspberry PI
• Connect the GND (-) of your sensor to the GND (pin 39) of your Raspberry PI
• Connect the Data (S) of your sensor to the GPIO4 (pin 7) of your Raspberry PI

So now that we are all set up and as specified below, we will run the test script with the command sudo node test.js

This start writing temperature and humidity levels to:

• the console
• a log file
• our DIoTY MQTT Cloud infrastructure

This last we can see on www.dioty.co by following the "My DIoTY" link you will see after signing in.

Connecting Arduino with DIoTY's Cloud MQTT Broker

In my previous post I showed how to get started with DIoTY's Cloud MQTT Broker.  Now that you are all set up, proved to yourself all is working, it's time to do something a little more interesting.  So today we will use an Arduino Uno and a temperature sensor to publish the current temperature to the MQTT Broker.

We use following hardware for our example:

• Arduino Uno with ethernet shield
• One-wire temperature sensor (DS18B20)
• 4.7k resistor
• breadboard and some wires

Make sure you verify the the pin layout before wiring as incorrect wiring can damage the sensor and the arduino + ethernet shield (and we accept no liability at all).

Wiring is done as in the picture below:

Next we need to load the software to the Arduino.  For this, first import the following two libraries into your Arduino Development Environment: OneWire and PubSubClient.

The latest OneWire library can be downloaded from Paul Stoffregen's site.
The latest PubSubClient library can be downloaded from GitHub.  Full documentation is found on Nick O'Leary's site.

If you haven't imported libraries before take a look at following guide.

Once the libraries are in place it's just a matter of copying and pasting the following code into the sketch, replacing the bits in red with values suitable for yourself.  Note that yourUserID is the email address with which you registered at DIoTY.  You need to use that same full email address at the start of your topic as you will only have authorisation under that subtree.

#include <SPI.h>
#include <PubSubClient.h>
#include <Ethernet.h>
#include <OneWire.h>

#define DEBUG
#ifdef DEBUG
  #define DEBUG_PRINT(x)  Serial.println (x)
#else
  #define DEBUG_PRINT(x)
#endif

byte mac[]= {0x90, 0xA2, 0xDA, 0x0D, 0x88, 0x5E} ; // change by your arduino mac address
char userId[] = "yourUserID"; // use your DIoTY user id (=email address)
char passwd[] = "yourPassword";  // use your DIoTY password
char server[] = "mqtt.dioty.co";
unsigned int port = 1883;
char topic[] = "/yourUserID/temp"; // topic where to publish; must be under your root topic
EthernetClient client; 
PubSubClient arduinoClient(server, port, 0, client) ; //no callback function is specified as we only publish


void setup() {                
  Serial.begin(9600); 
  DEBUG_PRINT(F("Initialisation"));
  beginConnection();
}

void beginConnection(){
  DEBUG_PRINT(F("Entering beginConnection"));
  if (Ethernet.begin(mac) == 0) {
    Serial.println(F("Failed to configure Ethernet using DHCP"));
    exit(-1);
  };
  DEBUG_PRINT(F("Obtained IP Address:"));
  DEBUG_PRINT(Ethernet.localIP());
  if (arduinoClient.connect(NULL,userId,passwd)) {
    DEBUG_PRINT(F("Connected to MQTT Server..."));
  } else {
    Serial.println(F("Failed to connect to the MQTT Server"));
    exit(-1);
  }
}

// reconnect after network hiccup      
void reConnect(){
  DEBUG_PRINT(F("Entering reConnect"));
  
  if (arduinoClient.connected()){
    DEBUG_PRINT(F("arduinoClient is connected"));
  } else {
    DEBUG_PRINT(F("arduinoClient is not connected"));
      if (Ethernet.begin(mac) == 0) {
        Serial.println(F("Failed to configure Ethernet using DHCP"));
      };
      DEBUG_PRINT(Ethernet.localIP());
      if (arduinoClient.connect(NULL,userId,passwd)) {
        DEBUG_PRINT(F("Reconnected to MQTT Server..."));
      } else {
        Serial.println(F("Failed to connect to the MQTT Server"));
      }
  };
}
  

// DS18S20 Temperature chip i/o
OneWire ds(2);  // on pin 10
byte i;
byte present = 0;
byte data[12];
byte addr[8];
int HighByte, LowByte, SignBit, Whole, Fract, TReading, Tc_100;
char buf[10];
char lastPublished[10];

void loop(void) {

  if (arduinoClient.loop()){
    DEBUG_PRINT(F("Arduino Client loop ok"));
    
    if ( !ds.search(addr)) {
        DEBUG_PRINT(F("No more addresses."));
        ds.reset_search();
        return;
    };
    if ( OneWire::crc8( addr, 7) != addr[7]) {
        DEBUG_PRINT(F("CRC is not valid!"));
        return;
    };

    getTemp();
    dtostrf(((float)(Whole*100+Fract)/100),1,2, buf);
    
    if (strcmp (buf,lastPublished) == 0) {
      DEBUG_PRINT(F("temp not changed"));
      DEBUG_PRINT(buf);
    } else {
      arduinoClient.publish(topic, (uint8_t*) buf, strlen(buf), 1); 
      memcpy(lastPublished, buf, 10);
      DEBUG_PRINT(buf);
    };
    
  } else {
    DEBUG_PRINT(F("Arduino Client loop nok"));
    reConnect();
  }
  delay(5000);     
}

void getTemp() {
  int foo, bar;
  
  ds.reset();
  ds.select(addr);
  ds.write(0x44,1);
  
  present = ds.reset();
  ds.select(addr);    
  ds.write(0xBE);

  for ( i = 0; i < 9; i++) {
    data[i] = ds.read();
  }
  
  LowByte = data[0];
  HighByte = data[1];
  TReading = (HighByte << 8) + LowByte;
  SignBit = TReading & 0x8000;  // test most sig bit
  
  if (SignBit) {
    TReading = -TReading;
  }
  Tc_100 = (6 * TReading) + TReading / 4;    // multiply by (100 * 0.0625) or 6.25
  Whole = Tc_100 / 100;          // separate off the whole and fractional portions
  Fract = Tc_100 % 100;
  if (Fract > 49) {
    if (SignBit) {
      --Whole;
    } else {
      ++Whole;
    }
  }

  if (SignBit) {
    bar = -1;
  } else {
    bar = 1;
  }
}

After uploading the sketch to your Arduino, it will measure the temperature every 5 seconds.  When the measured temperature is different to the previous measuring, the new temperature will be published to the topic /temp topic underneath your root topic (/<yourUserID>/temp).  As the publication is retained, all clients subscribing to the topic will immediately receive the latest temperature measured and receive any changes for as long as they stay subscribed...

Getting started with DIoTY's Cloud MQTT Broker...

In my first post I briefly explained the possibilities provided by DIoTY.co.  In this post, I'll help you getting started with the basics.  I will show you how to use the MQTT cloud broker with existing software you can freely download.  It's a quick and easy way to check you're correctly set up and to understand how MQTT works.

If you haven't done so yet, first go to DIoTY.co and sign in using a google email address.  You will receive your password to connect to the MQTT broker with your welcome email.  Note that this will take some time so do this before continuing reading this post...

When working with MQTT you will always have a client publishing messages to a topic and one or more clients receiving those messages by subscribing to that topic.

A sample publish and subscribe program comes with the free Mosquitto broker which you can download here.  It is available for most operating systems including Windows, Linux and Mac OS X and comes even with the full source if you which to compile it yourself.  I suggest you follow the installation instructions on the download page.  As you will connect to DIoTY's MQTT broker there is no need to start the mosquitto broker locally and all you are really interested in are the two programs mosquitto_pub and mosquitto_sub.

Assuming you have received your welcome email from DIoTY.co and you managed to download and install Mosquitto you're now ready to test.  For this open a couple of command line windows.  After installation, the mosquitto_pub and mosquitto_sub programs should be in your path but if not, it's probably a good idea to add them.

Publishing your first message:

You can now publish a message with the following command (just replace <your email address>, twice, with the email address you used to sign up and <your password> with the password you receive in your welcome email):

mosquitto_pub -h mqtt.dioty.co -p 1883 -u <your email address> -P <your password> -t /<your email address>/hello -m "Hello DIoTY"

DIoTY.co allows you only to publish in your own topic tree.  This means that the topic where you publish to must start with /<your email address>/.  Messages published outside of your topic tree will just be ignored.

Of course, publishing a message only makes sense when there is also at least one subscriber.

Subscribing to your topic tree:

You can subscribe to all topics in your topic tree with the following command (just replace <your email address> and <your password> like you did when publishing):

mosquitto_sub -h mqtt.dioty.co -p 1883 -u <your email address> -P <your password> -t /<your email address/#

     # is a wildcard and means that /any/subtree/will/match...
     mosquitto_sub keeps listening to the topics you subscribe to until you brake (ctr-c ; cmd-c; kill).

When you have followed this blog, the first thing you learn from this example is that messages published before the subscriber has been started will just be lost.  So whilst your subscriber is still subscribed, open another window and run again the mosquitto_pub command.  Now you will see "Hello DIoTY" appearing in your subscriber window.  Yes, it does work after all ;-)

I can hear people thinking already, when I connect with my mobile device to see the temperature at home, I don't want to wait till the temperature changes and the sensor publishes the new value.  I want to see immediately the last know temperature.  To obtain this, use so called retained publications.  Publish again the same messages as before, but this time add a "-r" to the mosquitto_pub command.  When you afterwards start a new subscriber client (either in a new window or by stopping and starting the one you have running already) you will see the retained message immediately.

Have a look at the documentation mosquitto_pub and mosquitto_sub for other options available.

Since our MQTT broker is in the cloud, you can also download a mobile application like mymqtt on your android device.  Just connect to mqtt.dioty.co, add your username, password and subscription topic and you not only see the published messages on your computer but also on your phone or tablet.

The beginning...

I'm writing this blog as founder of DIoTY.co, a cloud based platform to help you experiment faster with your IoT projects.  
DIoTY currently provides you with two services, an MQTT broker and a Node-RED development tool.  I'll start with explaining what they are and what you can do with them.


MQ Telemetry Transport (MQTT) is a lightweight broker-based publish/subscribe messaging protocol.  MQTT was originally developed by IBM but is currently standardised by OASIS and is both free and royalty free.  It is rapidly becoming one of the standards for IoT/M2M.  

MQTT is a lightweight broker-based publish/subscribe messaging protocol:

• Clients (sensors, mobile apps,...) connect to a broker.
• Clients communicate by sending and receiving messages to/from the broker.  For the broker, a messages is just a chuck of data.
• A client publishes a message to a topic (eg: /home/livingroom/temperature).
• A client can subscribe to many topics.  It will then start receiving all messages send to those topic(s).

As you can easily see, an MQTT broker as provided by DIoTY can remove the need to run your own web server at home.  Your sensors publish to the MQTT broker, your mobile app subscribes to the topics of interest and you're done...

Node-RED is a visual tool for wiring the internet of things.  Node-RED is a creation of IBM emerging technologies, open source licensed under Apache 2.0.

With the Node-RED tool provided by DIoTY you can subscribe and publish to the MQTT broker.  You can alter the messages by applying functions to it (eg: subscribe to /home/livingroom/temperature/c ; convert the temperature from Celsius to Fahrenheit and then publish again to the topic /home/livingroom/temperature/f ).

You can also interact over other protocols like http, websockets,... to retreive for example weather information from the bbc website and push it to your mobile application.

Finally, with the twitter node, building your own twittering house becomes as easy as pie.