Aquaponics: Arduino Email & Text Messaging

Background
With all of the lauded benefits of aquaponics, there are a number of drawbacks and they primarily lie in the need for electricity.  Typically there is at least one pump, but if you also use grow lights, water heaters and other electronics, you'll inevitably increase possible points of failure.  Wouldn't it be nice if you knew right away when pumps fail, grow lights burn out or don't come on, water heaters die, water temp plummets, a grow bed or fish tank springs a leak or overflows, humidity gets too high/low or the summer sun roasts your greenhouse?

Automation can immediately act to correct some of these conditions but sometimes you don't have a backup pump, grow light or stock tank heater.  Leaks, especially in basement systems, can ruin property if it's not addressed quickly.

What you want is a way to know what's going on as it happens and a very good way of doing that is through text message or email.  That's what this guide is all about and it's not unique to aquaponics or hydroponics - it can be used any time the Arduino needs to alert you of a sensor reading.

The Arduino Data Acquisition and Control System (ADACS) projects in Automating Aquaponics with Arduino use the mail API to notify owners of failed/blocked pumps, when environmental conditions move outside of user-defined ranges, and when grow lights fail or don't turn on.  Additionally, you'll be notified when the Arduino has failed to connect to the webapp for a set period of time to indicate a potential power loss or loss of an internet connection.  Knowledge of what is happening in your system is the most powerful tool of all and when you combine the Arduino in your aquaponics or hydroponics system, you get an inexpensive means of acquiring that knowledge.

How It Works
Like all of our web-based projects we'll use Google's cloud infrastructure, App Engine.  We'll set up an Arduino Uno with two push buttons, which, when pressed, will send a web request to App Engine and prompt either an email or a text message.  The client consists of a very basic form to collect the designated email address and text address.

Hardware
1 x Arduino Uno R3
1 x Arduino Ethernet Shield R3
2 x Momentary button or switch
2 x 10k ohm resistor
1 x Small breadboard
Breadboard jumper wires

Software
Arduino-1.0.3 IDE
Google App Engine Python SDK: 1.7.4
Python 2.7
Ubuntu 12.04*

*These instructions are probably very easy to translate for Windows or Mac, but as we have neither I can't help you there.

Step 1:  Create a New App Engine Application
The first step will be to create a new application on App Engine.  As all of our web application use App Engine, we've created a standalone tutorial to cover it: Creating a New Application.

Step 2:  App Engine - Configure app.yaml
  • The email alerts reside in a file called alerts.py.  We need to add that file to our app.yaml file, Fig 1.
Fig 1.  Add alerts to app.yaml
Step 3:  App Engine - models.py
  • In the typical MVC (Model, View, Controller) pattern, we need to create a model of our data, which resides in models.py and is imported to the files that need to access it.
  • We have one class to create, the UserPrefs class, which has three properties
    1. Time zone offset - not used in this tutorial
    2. Email Address - string property
    3. Text Address - string property
Fig 2.  models.py
 Step 4:  App Engine - main.py and Templates
  • Open main.py and add the import statements, Fig 3.
Fig 3.  main.py import statements
  • The first three imports are used to display the templates and run the application.  The users import is used to get the email address of you (the email address you used to create the application).
  • models is used is the file containing the UserPrefs data, which we covered in step 3.
  • main.py consists of just two classes: MainPage and SaveAddress. You can see MainPage in Fig 4.  
Fig 4.  MainPage class.
  • The title of the page is sent to the header template, which you can see in Fig 5.  This is how data can be passed into generic templates, reducing redundant programming.
Fig 5.  The first part of header.html
  •  In order to send emails or text messages you'll need to provide an address, so we add a very simple form to header.html.  You can see the complete file in Fig 6.  The form consists of two text fields and two buttons.  Each button is given a JavaScript onclick handler discussed in the next step.
Fig 6.  Full header.html file
  •  footer.html is a truly uninspiring file and only consists of closing tags
</body>
</html>
Step 5:  Email & Text Messages
A word on email addresses.  The email address you put as the sender must be an email address associated with the account you used to make the webapp.  You can also invite other Gmail addresses to be admins.  The email address you are sending to can be any valid email address.
Text messages can be sent through an email client such as Gmail given you format the address correctly.  Here is a link on the different formats carriers use for SMS and MMS.  We have only confirmed the Sprint format so I will use that as the example.

Sprint's SMS email format looks like this:
<your phone number>@messaging.sprintpcs.com
Example:
 
1234567890@messaging.sprintpcs.com
A Sprint user would substitute their phone number in and use this for the text message box.

Step 6: JavaScript and Form Processing
  • With our template and form in place, we will use JavaScript to send an asynchronous request to the server to process the form.  I've separated the JavaScript into two separate files.  The first is utils.js, Fig 7, and contains the function to create a request.  Looking back at Fig 6, you'll see utils.js is loaded before our second JavaScript file, main.js
Fig 7 - utils.js
  • Next, main.js contains the two onclickhandlers we referenced in the form.  I created them separately for readability in this tutorial, we could have easily created a single button to do this work. When a save button is clicked, a series of events takes place and you can read that in the comments.
Fig 8.  main.js
  • The webapp code to process the JavaScript requests is the second class in main.py, SaveAddress.
Fig 9.  SaveAddress in main.py
  • The JavaScript request is parsed and the UserPrefs entity with the email address you used to create the app used as the key_name.  If the entity exists, you will update the emailAddress or textAddress properties, otherwise a new entity is created.  Finally, the entity is put into the datastore and a response is made.
Step 7:  App Engine - alerts.py
  • alerts.py has one class, Alert, used to handle the Arduino request.  There is also one function, SendAlert used to send an email or text message.  First we list our imports
Fig 10.  alerts.py import statements
  • The Alert request handler parses the request from the Arduino to determine the type of alert you want, email or text, Fig 11.
Fig 11.  Alert request handler
  • The request handler for the Arduino parses the request to find out which type of alert you want, text or email.  Then it calls the SendAlert function passing in the type of alert and a message.  This makes the alert function generic and reusable.
Fig 12.  SendAlert
Step 8: Arduino - Fritzing Diagram
  • The circuit for this tutorial is very straight forward and consists of two push buttons.  One push button is used to request an email, the other a text alert.
Fig 13.  Fritzing Diagram
  • In Fig 13 you see a tiny breadboad sitting on an Arduino Uno R3.  In reality, the breadboard resides on an Ethernet Shield, or better yet, a proto shield on an ethernet shield on an Arduino R3.
Step 9: Arduino - Code
  • Start by setting up the Arduino with your assignments
Fig 14.  Arduino assignments and imports
  • The setup loop begins the serial output and the Ethernet.  It sets the pin mode for the pushbuttons to INPUT and finally notifies the user that setup is complete.
Fig 15.  Arduino setup loop.
  • The operating loop monitors digital pins two and three to see if the push buttons have been pressed.  If they have, we call the sendAlert function with the type of alert we want.
Fig 16.  Operating loop
  • sendAlert outputs via serial the type of alert being requested.  Then it creates a GET request to the Arduino and finally outputs the response to the screen.
Fig 17.  sendAlert Arduino function
  • The final two functions in the Arduino program are our normal response parsing and httpRequest functions.
Fig 18.  HTTP response parsing and request functions
And there you have it - two buttons to send either a text message or an email.  While the button format may seem useless think about this - the Arduino read the state of a pin before making the request and the GET request was generalized to allow either button to request the same function.

By extension, the Arduino can read the state of any input pin, analyze the reading via conditionals and send an alert if it needs to.  The Arduino Aquaponics code in the ADACS is rather large, so rather than apply conditionals on the Arduino, the values were automatically sent to App Engine and App Engine applied the conditionals to decide if an alert was needed.  This freed up valuable programming space and extended the number of conditionals that could be applied.

Online Relay Control


Intro
The most exciting thing for someone new to monitoring, control and automation over the internet, is being able to manually turn something on or off using your computer, tablet or phone, from anywhere. 

Inevitably, the next step is to automate the control.  Some automation examples include toggling the relay based on environmental conditions, sensor reports, time, GPS coordinates, motion detection or some combination of these.

We break down the uses of the Arduino in aquaponics to three main categories
  1. Monitoring.  Using sensors to monitor the environment be it air or water temperature, relative humidity, light levels, etc.
  2. Control.  Manually prompting an action, turning a pump on/off, grow light on/off, opening/closing windows, etc.
  3. Automation.  Combines steps one and two to use sensor readings to implement controls, without your input.
What kind of automation projects can the Arduino solve for aquaponics? Actually, there are many, but here are a few.
  • Pump timer
  • Backup pump triggered when primary pump died (true story).
  • Automatically increase/decrease pump cycles on consecutively cloudy/sunny days
  • Grow lights based on time of day, cloudy/sunny conditions
  • Light shades on bright days
  • Open/close greenhouse windows based on climate forecast
  • Toggle Portable heaters or air conditioners
  • Toggle light on when you enter greenhouse/basement/garage
No matter what intelligence you want to implement, it all starts with getting the relay talking to the internet and that is what this project is all about.  For an advanced version, see the Pump Controller project.


How it works
The technique we are going to use is polling and is analogous to a child sitting in the backseat on a long road trip asking "Are we there yet?" every twenty seconds.  An Arduino will make a GET request to App Engine, which will query the datastore for the relay entity and return the relay's state property.  The Arduino will parse the response and trigger the relay pin HIGH/LOW.

The webapp is a simple image, whose class changes based on the the current state.  Click the power button image and it will toggle the class, create an AJAX request to the server which in turn will toggle the state property of the relay entity in the datastore.

Fig 1.  On/Off Button.
Parts List
1 x Arduino Uno R3
1 x Arduino Ethernet Shield, R3
1 x Powerswitch Tail II
2 x breadboard male/male jumper wires
1 x Arduino wall wart (optional, for better power)

Software Versions
Arduino IDE 1.0.3
Google App Engine, Python SDK 1.7.4
Ubuntu 12.04
Python 2.7


Step 0 - Create a New Webapp on App Engine
This step has it's own short tutorial.

Step 1 - Download Project Code
  • Download the project source code here.
  • Extract the tar file to the directory of your choosing.  The following instructions assume it's in your home directory, so amend the instructions with whatever directory you put the code in.
  • The extracted folder is called IAquaponics_Relay and inside are two folders, myapsystem containing the GAE webapp and arduino.
  • The first thing you will need to do is open app.yaml inside of myapsystem.  The first line is the application name and needs to be changed to reflect the new application identifier you created in Step 0.
Fig 2.  app.yaml.  Replace the highlighted name with your project name.
  • Take note of the bottom.  The main page of our webapp is listed last because it will catch all url requests not listed above it.  In fact, this webapp only has two pages: adacs and main.  The former is used by the Arduino and the latter is the visible interface you will see with the web browser (client).
  • Finally, under main.app, you will see login: admin.  This app uses Google Accounts as you saw in Step 2, but we are restricting application to the webapp to just you.

Step 2 - Launch the AppEngine SDK with Your Project
  • Open a terminal and launch the AppEngine SDK and your project.
python2.7 AppEngine/dev_appserver.py IAquaponics_Relay/myapsystem
  • If all went well, the terminal will tell you the project is located at
http://localhost:8080/
  • So fire up a web browser and point it to that url.  Because we have restricted access to admin, you will need to login as admin.  The login name is irrelevant, but beneath the email input box you need to check the box that says "Sign in as Administrator".  Then login.
Fig 3.  Login as Administrator.
  • At this point, you should see Fig 4.
Fig 4.  On/Off Button.
  • With your terminal in view, click the button.  It will change to green and in your terminal you will see the request made to the server, Fig 5. 
Fig 5.  Terminal output on class change.
  • The button's class has now changed and the request to the server makes the Relay entity's state property change to reflect the class.  Open a new tab in your browser and go to your admin page.
http://localhost:8080/_ah/admin
  • This should load the datastore viewer pane.  The only entity is Relay.  Click List Entities and it will load the properties for Relay, Fig 6.  You can see the state property is set to On now.
Fig 6.  Relay entity with state property set to 'on'.

Step 3 - Upload the Webapp to App Engine
  • If everything has worked for you so far, it's time to upload the webapp to a live server.  Open a terminal and type
cd AppEngine/
./appcfg.py update ~/IAquaponics_Relay/myapsystem
  • You will likely be prompted for the Google credentials you used to create the application in Step 0.
  • When the update is done, open a browser and go to your live webapp.
http://myapsystem.appspot.com
  • If you are not already logged into your Google account, you will be prompted to.
  • The main screen is a repeat of Fig 4.  Click the power button to change the state to on and go into your Admin Console to confirm the state has changed.  The admin console for live webapps is
http://appengine.google.com

Step 4 - Arduino
  • Connect the Ethernet shield to the Powerswitch Tail II as shown in the Fritzing diagram in Fig 7.
Fig 7.  Fritzing diagram of Ethernet shield and Powerswitch Tail II.
  • To test the relay, plug an appliance such as a lamp into the relay and plug the other end of the relay into the wall.
  • Now load the Arduino code from the project directory.
  • You will need to amend the Arduino code, pointing it to your live webapp.  There are two locations where the webapp link is listed.  Change ONLY the highlighted code in Fig 8 and Fig 9.
Fig 8.  First location.

Fig 9.  Second Location.
  • Save and upload to your Arduino.
  • Open the serial monitor.
  • In your webapp, toggle the button and wait to see the relay toggle.
  • And that's it.
Notes
  1. Since working with AppEngine and Arduino Ethernet we have encountered one consistent error.  The Arduino will fail to make the third request.  The first two are fine.  Every request after the third connects and works, but for some reason the third one will fail, every time.
  2. The Arduino creates a GET request and then pauses for ten seconds before parsing the response.  Toggling the power button in the webapp will not immediately toggle the light.
  3. The rate of the GET requests is quite high, for demonstration purposes.  If you are going to implement this, you may want to double the delayed time interval in order to keep extra instances from spawning and reducing the read/write operations.
The Next Steps
The point of this tutorial was to show the polling technique the Arduino uses to get the state of the relay from the web and then change the relay pin.  Here we implemented manual control, but there was logic involved.  When the Arduino made the request, App Engine queried for the Relay entity and simply returned the state property.  Instead, we could have changed the interface to be a simple form to get the start and end time for a grow light in a basement system - say 5:00 am and 10:00 pm.  Then we could set a conditional when the Arduino made its request, to compare the timestamp of the request with the current time.  If the timestamp is inside the operating time, the light is on, otherwise it is off.  This is just a small example of the kind of intelligence you could program in; the point is, program it on the web side, where you can make changes and see the current state of a relay.

You can find more tutorials like this on the Automating Aquaponics with Arduino page.

Related Projects
Real-Time Graphing Online
Online Temperature and Humidity

Any questions or comments are appreciated.

Live Demo of Tutorials

As we upload more software and hardware tutorials on using the Arduino to automate aquaponics, we are also including live demos of each tutorial, where it makes sense.

http://adacsprojects.appspot.com




Aquaponics: Real-Time Graphing Online


While this project was designed with aquaponics in mind, it does not require an aquaponics system, making it useful in any application that wants graph data online in real-time.  The included application, therefore, is bare-bones to make it easier to integrate into your own project.

You can find a background tutorial on the light-dependent-resistor here.

Intro
I can think of a multitude of reasons you would want to graph data from an Arduino sensor, but the idea of retrieving the SD card from whatever environment your Arduino is in, hooking it up to a computer, downloading the most recent data and finally loading the data into your favourite visualizer, well that gets old fast.  Instead, lets hook the Arduino up to the internet and graph the data in real-time.

This project focuses on collecting the light level from an aquaponics greenhouse using a light dependent resistor (LDR) and sending it to Google App Engine to be graphed using Google Chart Tools.  The cycle is as follows: the Arduino attempts to connect to the webapp and waits ten seconds before parsing the response.  If the response is 'Ok', a good connection was made and a bicolor LED is set to green.  If it could not connect, the LED is set to red.  If a good connection was made, the Arduino will read from analog pin zero (A0) and create a GET request to App Engine with the LDR reading.  App Engine will save the LDR reading and update the current environment property.

On startup, the client will request the server to query the datastore for the light data and receives a JSON array of the current day's data.  The array is parsed and an AJAX request is made to create a table using Chart Tools.  Finally, the client sets an interval for a function that requests the last data entry as another JSON array, parses the response, compares it to the last data in the table and adds the data if they are not the same reading, creating a dynamically updated display.

Software Versions
1. Arduino IDE: Arduino-1.0.3
2. App Engine SDK: Python, Linux, 1.7.4
3. Python: Python2.7
4. Ubuntu 12.04
 Parts List
1 x Arduino Uno R3
1 x Arduino Ethernet Shield R3
1 x Arduino Power Cord (optional, but gets better power than just the USB)
1 x Bicolor LED (Red/Green)
1 x 1/4W 470 ohm resistor
1 x 1/4W 10k ohm resistor
1 x Light Dependent Resistor
Breadboard jumper wires
Small breadboard

Step 1 - Create a new App on App Engine
Since all of our internet based projects use App Engine, we've created an independent tutorial to explain how to create a new app.

Step 2 - GAE Project Code
  • Download the project code here.
  • Extract the tar file in your home directory.  It will be labelled IAquaponics_LDRInside you will find the App Engine directory called myapsystem and the Arduino folder arduino.
  • The name of the GAE code is irrelevant, but you can rename it if you wish.  The rest of the instructions in this tutorial will use the original name.  If you choose to amend the directions accordingly.
  • Inside myapsystem you will find app.yaml.  Open that in your favourite text editor and edit the first line to the project name you created in Step 1; see Figure 1.
Fig 1.  Edit the highlighted text.
 
  • Take note of the bottom.  The main page of our webapp is listed last because it will catch all url requests not listed above it.  In fact, this webapp only has two pages: adacs and main.  The former is used by the Arduino and the latter is the visible interface you will see with the web browser (client).
  • Finally, under main.app, you will see login: admin.  This app uses Google Accounts as you saw in Step 1, but we are restricting application to the webapp to just you.

Step 3 - Launch the AppEngine SDK with Your Project
  • Open a terminal and launch the AppEngine SDK and your project.
python2.7 AppEngine/dev_appserver.py IAquaponics_LDR/myapsystem
  • If you were successful, the terminal will tell you the project is located at
http://localhost:8080/
  • So fire up a web browser and point it to that url.  Because we have restricted access to admin, you will need to login as admin.  The login name is irrelevant, but you will need to stay consistent in the login name you choose.  Beneath the email input box you need to check the box that says "Sign in as Administrator" - see Figure 2.  Then login. 
Fig 2.  Login as Administrator.
  • At this point you should see Figure 3.
Fig 3.  Main screen.

  • There are two important things to note.  First, the chart was built with a piece of data.  If you hover around the point (Now, 750), you will highlight the data.  If you wait 20 seconds, that will disappear.
  • More importantly you will see a text box to input your timezone offset from UTC.  To get accurate timestamps of your readings from the Arduino, you'll need your timezone before you hook up the Arduino.  In Iowa, we are six (6) hours behind, so we would submit
-6
  • Australia is eight hours ahead of UTC, so they input
8
  • Next, we are going to replicate a call the Arduino will make to the server.  In the browser's url bar, type
http://localhost:8080/adacs/lightLevel?Level=800
  • You'll be presented with a page that says 'Ok'.  Now go back using the Back button and you'll see a seemingly blank chart, like Figure 4.
Fig 4.  First data.

  • It's actually not blank.  You can see the timestamp on the bottom of the chart and you can hover above (timestamp, 800) and see your data input.
  • If you look in the terminal from which you launch the SDK and your project, you will see a recurring set of requests.  This shows the JavaScript making the AJAX requests to the datastore to get the non-existent data.

Step 4 - Go Live:  Upload Your Project to App Engine
  • If everything has worked for you so far, it's time to upload the webapp to the live server.  In a terminal, from your home directory, type
cd AppEngine/
./appcfg.py update ~/IAquaponics_LDR/myapsystem
  • You will be prompted to input your Google email and password you used to create the project from Step 1.
  • When the update is complete, go to your webapp.
http://myapsystem.appengine.com 
  • If you are not already logged into your Google account, you'll be prompted to.  Note the login is consistent for all of Google's products.
  • Once you are logged in, you should see the same screen from the SDK.  Again, input your time zone offset.
  • To test the live webapp, replicate the Arduino call just as you did in Step 3. 

Step 5 - Arduino
  • Wire up your breadboard using the Fritzing diagram in Figure 5.  The Arduino is not shown, but obviously it resides under the Ethernet Shield.  The 470 ohm resistor is used for the bicolor LED and the 10k resistor is used for the LDR.
Fig 5.  Fritzing Diagram.

  • You can find the Arduino file, LDR.ino inside the arduino folder.  Upload that to your IDE.
  • There are three places the Arduino code needs to be altered, Figs 6, 7 and 8.  Replace the highlighted code with the project name you developed in Step 1.
Fig 6.  First location.

Fig 7.  Second location.

Fig 8.  Third and final location.
  • Select the correct Arduino Board from the menu list and make sure to select the proper Serial Port.  Save and upload your code to your Uno, then open a serial monitor.
  • When the Arduino connects, the bicolor LED will turn green.  If there is a connection issue, the Serial output will tell you and the LED will turn red (see Note 1).
  • Once the Arduino connects and sends data, go back to your web browser look at your app home page.  You should see the graph update itself as it receives data from the datastore.  Figure 9 shows the our graph over a very short period of time.  The large dip at 19:23:34 was due to our hand blocking the LDR from the light source.
Fig 9.  Real-time data graph.

  • And that's it.

Notes
  1. Since working with AppEngine and Arduino Ethernet we have encountered one consistent error.  The Arduino will report that it failed to make the third request.  The first two are fine.  Every request after the third connects and works, but for some reason the third one will fail, every time and we don't know if it is due to App Engine or the Arduino.
  2. For troubleshooting and development mode, Serial output is on.  Specifically, there are two counters provided, which need to be commented out (using double slashes at the beginning of the line: //).  If you don't, the counter can grow very, very large.

You are free to modify the code as you see fit, but I have one suggestion.  Google App Engine allows for daily free quotas, if you exceed the free quotas and don't pay a monthly fee ($9), your app will be shut down.  Two free quotas you should be aware of are instance hours and read/write/small operations on the datastore.  You get 28 free instance hours, reset daily.  The Arduino should not be set to send requests to AppEngine too often, or new instances of your app may spawn.  Similarly, the browser is set to refresh every 20 seconds.  If you combine the Arduino requests with the client requests, and make them too frequently,  you may again spawn new instances.  In fact, your app is hosted on a server with other applications.  Depending on the load on the other applications, your app may spawn new instances elsewhere.  Better safe than sorry.

Read/write operations are set to 50K each and AppEngine is a little funny about how they count these, but needless to say, making a request every ten seconds, all day from both the browser and Arduino will run up a lot of operations.  Try not to go below twenty seconds on both the client and Arduino and enjoy the free quota.

If you have trouble, hit up the comments below.  And if you include this in one of your own projects, send us a screenshot, we'd love to see it.

Related Projects:
EnvDAQ with Water Temperature Sensor
Online Temperature and Humidity
Online Relay Control
Arduino Email and Text Message Alerts

Related Parts:
pH
Dissolved Oxygen
Water Temperature


Creating a New Application on Google App Engine

The first step in following our web application tutorials is to create a new application on Google App Engine.  So we don't have to repeat the steps each time, this tutorial will be included at the beginning of each project.

Step 1 - Sign Up for Google App Engine
  • If you don't already have a free Google Account,  sign-up for one.
  • Using your Google account, sign up for Google App Engine.


Step 2 - Download the AppEngine Python SDK
  • Our projects are built on Ubuntu Linux and use the Python SDK.  Projects are developed on the local machine in a sandbox using the Software Dev Kit, which you can download here.


Step 3 - Create a New Application
  • At the AppEngine homepage, you'll be presented with a list of your applications.  At the bottom, click "Create Application" and input a name for your webapp.
Create a new application.

  • All of our webapps use the default security.  Click "Create Application".

Step 4 - Update App.yaml
  • The first step in either creating a new app, or using our source code, is to modify app.yaml to use your new project name.  Open your project directory (not the SDK directory) and open or create app.yaml in your text editor of choice.
  • The first line is the application name, which you chose in Step 3.  In Fig 2, our project is called myapsystem, so replace this with your own project name.
Fig 2.  Generic app.yaml file
  • Lastly, when you chose your project name and set the security to the default setting, you still have options.  For individual projects, each page is set to login: admin that you can see in Fig 2.  If you want to host an app to anybody with a Google login, change this to login: required.  If you don't want a login, delete this line.
 Step 5 - Upload the Web Application