Skynet

You’ve probably heard of the “Internet of Things”—the cool kids call it IoT—but if you’re like most people, all you know is that it has something to do with kitchen appliances that can order groceries for you.

Like so many of us, jaded by decades of technological miracles, for years I mostly ignored IoT as just another TLA in the never-ending cacophony of buzzwords, but now that I’ve put some time into learning what it’s all about, the scales have fallen from my eyes.  It’s not just smart coffeepots and burglar alarms the call your iPhone. I’m coming around to the view that it’s ultimately going to rival the Internet itself in scale, and may well result in epochal changes to the human condition.

However, all that’s a subject for a different blog piece.  This one’s about how you can help Skynet go self-aware right on your own kitchen table.

It’s not so much that the IoT world is technically new or wonderous. Huge slices of the technology aren’t new at all. What’s new is the way that devices that once required serious electrical engineering chops have been systematically reduced to electronic Lego that you can put together out of a cookbook.  You can go amazingly far just aping what you see on a YouTube video.

So, without further ado…

Cool Thing Number One

Actually, one more bit of ado. A disclaimer. To be clear, I’m not holding up what you see in the picture below as a masterpiece of electrical engineering, nor, despite the fact that I’m a programmer, is the small amount of computer code behind any more polished than the wiring.  It’s not an incredible piece of tech—it’s barely even a creditable piece of tech—but what it points to is breathtaking.

The point of this is to show you what pretty much any 21st Century monkey can now put together using dirt-cheap hardware, open-source libraries and an Internet connection. If this required expertise, I’d be writing a different piece.

What you’re looking at is a home-brew contraption for gathering sensor data, responding to it.

The little green board is a Raspberry Pi-3 single-board computer running Linux A brand-name Pi costs $30, but you can get cheaper Chinese knockoffs for less.

The Pi is connected to an Arduino embedded controller prototyping board which is behind it, under the rat’s nest of wires. The Arduino has a small processor on it and a lot of pin-sockets into which you can plug sensors, motor drivers, lights, etc. The Arduino runs a program that you write on a real computer and download to it via the USB port.  Once you’ve loaded the program, you can disconnect the Arduino and it will run that program forever, doing whatever it says. The memory is permanent, and each time you turn it on, it will initialize and then run your program until you turn it off. A typical program will read some sensors and then something with the data it read, e.g. turn on a motor, send the data to some other system, etc.  It’s the same general kind of device that runs smart coffee pots or keeps Segways balanced upright. A brand-name Arduino costs about $20, but the equally-good Chinese knock-offs are half of that.

The thing behind the Arduino is a “breadboard,” which is just a sort of peg-board for plugging wires into to connect things. Once you get everything working you’d typically replace it with a circuit board that you solder wires to.

In this case, every five seconds the Arduino reads temperature and humidity sensors that are mounted on the breadboard, notes the time of time of day, then formats that information into a text string and sends the information to the Pi over the serial port.  It also has some LED’s that it can light up, but more on that later.

This is all very basic Arduino beginner stuff. Once you set up the Arduino development environment on your PC, and sort through the parts box, you can be building things like that in an hour or two.

Cool Thing Number Two

The real point of this exercise is that the Pi is forwarding the data generated by the Arduino to the AWS cloud, a message every five seconds.

Every time the sensors are read, the data is transmitted to The Cloud via something called MQTT over HTTPS. The HTTPS part gets it to the Cloud, but the more significant part is the MQTT, which is a simple protocol designed for machine-to-machine communication, especially between low-power devices like you’d find in IoT applications.

MQTT is a “pub/sub” protocol, which means that the messages the Pi sends arrive in the sky tagged with a string called a “topic.”  A topic isn’t an IP address; it’s just an identifier, i.e, an arbitrary name.  You just tag your messages and send them, and the generic receiver in the cloud catches them and passes them on to any process that has properly ‘subscribed’ to that particular topic. Sending a message is like putting a note on a bulletin board.  Unlike a direct connection, the receiver doesn’t have to be connected when the message is sent, and the sender doesn’t have to know the addresses for the receivers. There can be zero, one, or many listeners and the messages will stay on the board for a while.

I used “sdkTest/sub” as the sending topic because some demo code I cribbed from used that topic, but I could have used ‘barney_rubble.’ It’s just a string.

In the Cloud, I set up AWS’s Kinesis service to listen to the sdkTest/sub topic and collect all the messages that arrive for either ten minutes or until 50MB of data have accumulated, whichever comes first (it’s always the 10 minutes in this case.) When ten minutes worth of messages have accumulated, the service writes them all to a file that is stored in another service called S3. The S3 file is stored by year, month, day, and time, so you can always find the data for a given interval.

I also defined what AWS calls a “lambda” function.  A  lambda is a free-floating chunk of code that executes on each record as it arrives. The lambda can re-format and resend the message, respond to it, store it in a database, etc. In this case, the lambda tests whether the temperature and humidity are outside of certain ranges, and if so, sends a message back on a different topic to which the Pi is subscribed, and the Pi, in turn, tells the Arduino what to do.  You can breathe on the humidity sensor on my desk, and down in Northern Virginia, a decision is made to tell the Arduino to blink a light.

There’s a second Lambda function that massages the data into another format and inserts it into DynamoDB, which is AWS’s proprietary big-data key-value store database.

This is a pretty Rube Goldberg way to light up an LED, but look what we’ve done.  The data is sitting in the ready to be combined with any other data accessible from the cloud in any quantity.  And we’re also responding to it dynamically, second by second and reflecting that response back down to earth where an action is taken. In this case, it’s just turning on an LED, but we could just as easily be activating more complex functionality down on earth—reading more sensors, turning on a video camera or an air conditioner—anything.

Cool Thing Number Three

I mentioned that the data is sent to AWS via MQTT over HTTPS. The alert reader may have wondered why bother with MQTT? Why not just sent it via HTTPS, like your browser does when you type in a URL?

That question gets us to the most interesting part. The Pi speaks TCP/IP to the internet, but the Pi and the Arduino don’t need TCP/IP or any other network protocol to talk to each other because they are physically wired together. Actually, the Arduino isn’t even necessary for our device, because the sensors could have been wired directly to the Pi. The Arduino is mainly there because it’s easier and cheaper to replace an Arduino than a Pi when you “let the smoke out of it” by wiring the wrong thing to the wrong pin (to use AvE’s memorable phrase.)

If you wanted to wire up a thousand sensors, say for a factory or commercial building, or to study a patch of forest, you could repeat what I did a thousand times, but it would be a huge job. For one thing, TCP/IP networks are inherently complicated to administer because they need routers that know how to get to specific device addresses. There would also be the hassle of setting up thousands of sensors as separate AWS “things.” And finally, a general purpose Linux computer like the Pi or even the Arduino is more hardware than you need for a sensor—low-power sensors can run for months or years on a battery charge that an Internet-connected Pi would exhaust in a few hours.

The third cool thing is that the Pi and in this case, the Arduino, have a capability called Bluetooth Low Energy V5 (BLE.) It’s built into the Pi and the Arduino has it as an add-on. It’s an easy and cheap feature to add to almost any IoT device.

Devices with BLE use radio to talk directly to each other at a distance of up to several tens of yards. This isn’t a TCP/IP connection—there’s no IP address, no routing, etc. It only works for maybe 30 feet or so, but here’s the magic: BLE devices have the ability to automatically link up into what is called a “mesh” network.  The devices in the mesh still can only communicate directly with nearby devices but the mesh can pass data from device to device like a bucket-brigade to reach far away devices. As long as there is an unbroken chain of connected devices to the destination, the data will get there.

No router or central authority has to know that there is such a chain. The mesh network doesn’t use routers or routing tables–it just finds the routes dynamically. You can Google up how it works, but the name of the routing strategy is a hint. It’s called “managed flooding.”

Eventually, the data has to get to some device that has a TCP/IP connection to the Internet, but for low bandwidth devices like this, even a tiny computer like a Pi could act as a gateway to the Cloud for hundreds of sensors located throughout a building or area of land.

So yes, if all the data going up to the AWS IoT service originated on TCP/IP-capable devices, the IoT service probably wouldn’t bother with MQTT, but it makes sense to use it because the TCP/IP capable devices are the exception in the IoT world, just passing along the data from more numerous low power devices that communicate via MQTT.  The next step in this project, in fact, is to get a chain of Arduinos passing data to the Pi via BLE.

What Does it Cost To Run Skynet From Your Kitchen Table?

So even a hobbyist could cobble together an entire network of low-power sensors and run it out of the AWS Cloud but what would it cost?  It’s pretty inexpensive.

The cost of the Pi, Arduino, breadboard, sensors, etc. that you see here adds up to around $75, but simple boards with BLE cost as little as $5, plus a couple of dollars for sensors and a few more for a battery.  If you set them up as a mesh, you only need the one Pi online.  So the hardware cost is $75 for the Pi and two sensors, plus about $20 for each additional sensor board (each of which might handle multiple sensors.)

S3 storage is $0.023/GB/month, or $0.27/GB/year.  This demo writes about 250B every five seconds, so that’s about 230 days of data per GB, or a little over a penny a day. The DynamoDB data is more, but also negligible at this data volume.

Notice that I never said anything about the cost of deploying computers in the cloud. That’s because I’m not using any.  The Lambdas and the Kinesis run as services.  If you want to do some computing on the data or expose it via HTTP you’d probably run virtual machines, at a cost of anywhere from free to several dollars an hour.  Typical machines would cost about $0.35/hour.

That’s Where It’s Going

So that’s where the world is going right now, not in some flying-car future that never gets here. Swarms of sensors and smart devices pumping everything up to a giant distributed brain in the sky. And that’s just in MY basement.