EcoRenovator  

Go Back   EcoRenovator > Improvements > Appliances & Gadgets
Advanced Search
 


Blog Register 60+ Home Energy Saving Tips Recent Posts Search Today's Posts Mark Forums Read


Closed Thread
 
Thread Tools Display Modes
Old 11-19-10, 06:10 PM   #1
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default Home energy data logging project

Yesterday I saw that onterio will be increasing the electric bills 46% over the next 5 years as they phase out coal and go renewable. We don't have coal to phase out but are expecting similar increases in the future. The city recently announced millions in water infrastructure updates and they will be billing that not through property taxes but on water usage. In both cases the more you use the worse the hit will be.

So today I decided it's time to get off my *** and get really serious about consumption logging and reduction. We've done pretty good on the electric front but more can be done I'm sure.

Water wise we discovered a break in the main on the cities side of the meter so they'll be digging that out and fixing it shortly. While spending all this time looking at the meter I decided to throw the reading into my spreadsheet and see how we're doing since I got a bill just a month ago. It reports that we're using a lot of water. 200 gallons a day almost which is comparable to my dead of summer watering 1000 sqft of gardens and 14 fruit trees water usage. So I waited 24 hours, and checked again. Since then we've used 52 gallons. Thats with 2 of us having long showers but no laundry or dishwasher. My best guess is they don't actually read the meter 3 times a year and instead average things for the billing. We'll see.

Anyways since I want to do data logging and realtime monitoring of our usage today I ordered an arduino mega microcontroller kit off of ebay. I'm sure it's a chinese knock off but my morals no longer care so long as it works. the unit was 30% the price of an official one. I also ordered up some hall effect current sensors from a small place in the US. Later tonight maybe I'll pick up a bunch of temp sensors.

The plan is to wire up and monitor the following
Electric dryer consumption
Electric hot water heater consumption
cold room temperature
basement back temperature
basement front temperature
bedroom 1,2,3 upstairs temperature
upstairs living room temperature
outside backyard temperature
water from mains temperature
water from hotwater tank temperature

That is a good start to let me know what is going on beyond what my kill a watt can tell me.

As I've been slowly piecing my solar hotwater setup together(read that as I've got a stack of copper pipe, some glass and a tank) I'll use the arduino as the differential controller and some more logging. In the logging it will track
solar tank top temperature
solar tank bottom temperature
trench temperature
pipe to collectors start temperature
collectors inlet temperature
collector outlet temperature
collector surface temperature
plus more I've forgotten I'm sure.

eventually I'd love to put a water meter with digital outputs in line on the main but they are not cheap last time I looked.

My goal is to use this thread to organize my thoughts and flesh out design as I SLOWLY chunk through all of this. I'm a software guy not electronics so a bunch of this will require some research. Looking at the arduino stuff the code is trivial and I've seen some pretty cool web linking so I'd like to try that. The hardware is not too brutal looking but I'm doing my best to stay in the realm of already been done and here's the step by step tutorial on how to make it work.

The pieces I've selected so far are
hall effect current sensors - Linear BiPolar Hall Effect Current Sensor +/- 80 Amps I ordered 4 shipped for $26.40

maxim ds18b20 one wire temp sensors - 2 free samples so far ebay has them shipped for under $2 each if you order more then 10

the arduino ATmega with a 128x64 LCD screen and a bunch of starter electronics stuff that may be useful since I don't have much of it. Arduino ATmega 1280 128x64 Graphic LCD Mega Starter Kit - eBay (item 150521034176 end time Dec-16-10 04:19:03 PST) $68 after shipping.

When everything arrives I'll do some basic testing and get a couple of temp probes working and measure some electrical currents that I can verify against the kill-a-watt before I hook up the 220 appliances.

I'll have to decide what I want to do wire wise for all of this stuff. for things like the solar panels cat5 may work, but multiple runs of simple twisted pair may be cheaper. We'll see what home depot is carrying.


This should be fun. I've only been wanting to do it for a couple of years now...

strider3700 is offline  
Old 11-20-10, 01:02 AM   #2
Piwoslaw
Super Moderator
 
Piwoslaw's Avatar
 
Join Date: May 2009
Location: Warsaw, Poland
Posts: 926
Thanks: 176
Thanked 101 Times in 80 Posts
Default

Re water usage, here are a few ideas:
  • I've seen analog (dial) type water meters with a wire that sends a signal every time a certain dial turns. They are cheaper than the fancy digital meters, but could be even cheaper if you DIY (similarly to the gas meter in a link you recently shared).
  • Install one for your water mains (the utility probably won't you replace their meter, so install your's after it).
  • Put another one before your hot water tank, so you'll more or less know how much hot water you're using.
  • One more meter just for the toilets. This will tell you how much can be saved by switching to lower flush models and/or reusing grey water to flush.
  • And another meter for water that goes outside (yard, trees, car washing, etc.). Now install a rain water system and watch the meter stop moving Metering water for yard usage is becoming popular here, since it allows the sewage bill to be reduced by that amount.
Piwoslaw is offline  
Old 12-07-10, 03:35 AM   #3
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

OK my arduino mega showed up this morning and having not done any electronics since about 1997 I took my time figuring things out. At this moment I have it reading the temperature using a ds18b20 temperature probe.

It turns out that the b is important because I spent 2 hours trying to figure out what was wrong based on following a tutorial for a ds18s20... The CRC code the device reports back is different. So for my and anyone else that wants to know's information here's what I needed to do.

on the ds18b20 pin 1 goes to ground on the arduino. Pin 3 is jumpered to pin 1 so it's also going to ground on the arduino. 5V on the arduino goes to one end of a 4.7kohm resistor and that resistor then goes to pin 2. pin 22 on the arduino goes to pin 2. Thats it wiring wise.

Here is the code I used, hacked directly from a tutorial then hacked up using some more notes I found elsewhere
Code:
#include <OneWire.h>

/* DS18b20 Temperature chip i/o */

OneWire  ds(22);  // on pin 22

void setup(void) {
  Serial.begin(9600);
}

void loop(void) {
  int HighByte, LowByte, TReading, SignBit, Tc_100, Whole, Fract;
  byte i;
  byte present = 0;
  byte data[12];
  byte addr[8];
  
  if ( !ds.search(addr)) {
    Serial.print("No more addresses.\n");
    ds.reset_search();
    delay(250);
    return;
  }
  
  Serial.print("R=");
  for( i = 0; i < 8; i++) {
    Serial.print(addr[i], HEX);
    Serial.print(" ");
  }

  if ( OneWire::crc8( addr, 7) != addr[7]) {
      Serial.print("CRC is not valid!\n");
      return;
  }
  
  if ( addr[0] != 0x28) {    //changed the value to 0x28 to recognize ds18b20 
      Serial.print("Device is not a DS18B20 family device.\n");
      return;
  }

  // The DallasTemperature library can do all this work for you!

  ds.reset();
  ds.select(addr);
  ds.write(0x44,1);         // start conversion, with parasite power on at the end
  
  delay(1000);     // maybe 750ms is enough, maybe not
  // we might do a ds.depower() here, but the reset will take care of it.
  
  present = ds.reset();
  ds.select(addr);    
  ds.write(0xBE);         // Read Scratchpad

  Serial.print("P=");
  Serial.print(present,HEX);
  Serial.print(" ");
  for ( i = 0; i < 9; i++) {           // we need 9 bytes
    data[i] = ds.read();
    Serial.print(data[i], HEX);
    Serial.print(" ");
  }
  Serial.print(" CRC=");
  Serial.print( OneWire::crc8( data, 8), HEX);
  Serial.println();
  LowByte = data[0];
  HighByte = data[1];
  TReading = (HighByte << 8) + LowByte;
  SignBit = TReading & 0x8000;  // test most sig bit
  if (SignBit) // negative
  {
    TReading = (TReading ^ 0xffff) + 1; // 2's comp
  }
  Tc_100 = (6 * TReading) + TReading / 4;    // multiply by (100 * 0.0625) or 6.25

  Whole = Tc_100 / 100;  // separate off the whole and fractional portions
  Fract = Tc_100 % 100;


  if (SignBit) // If its negative
  {
     Serial.print("-");
  }
  Serial.print(Whole);
  Serial.print(".");
  if (Fract < 10)
  {
     Serial.print("0");
  }
  Serial.print(Fract);

  Serial.print("\n");
  
}
You will need to install the onewire library but that is easy enough to find with google, then just extract the zip into the correct location which in my case is /sketchbook/libraries

I think my next thing will be getting multiple temp sensors working hopefully that is easy. After that I want to try out those current sensors but that will involve a bit of soldering.
strider3700 is offline  
Old 12-07-10, 02:53 PM   #4
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

ok the second temp sensor is trivial. connect pin 1 and pin3 to ground on the arduino, connect pin 2 to pin 22 the same as the other sensor. I didn't even shut the arduino down or update the code. the serial monitor just started reading the second one as soon as it was hooked up.

right now they are reporting that my office is 17.93 or 17.52 C and they are 3 inches apart on the bread board. They are sensitive enough that if I hold my hand a couple inches above them the temperature starts climbing within 5 seconds. Now to order 25 more and track down a crap load of wire to do the entire house. This is too cool.

Next step I think will be getting this data logging and accessible online. If I track down some 3 wire or an extra cat 5 cable I'll try the current sensors out but that will take some non-coding work.

<edit> Ok I just ran across this page http://www.pachube.com/ it looks like it's tailor made for what I want to do here. I'll have to poke around and see if It will do what I want or if I need to roll my own. </edit>

<edit #2 > apparently pachube is not free if you want to store data for more then 30 days. the non free options range from really expensive to insanely expensive. $15/month to $900/month with 100% storage starting at $100/month... I'll play with it a bit for now but google apps is likely in my near future </edit #2>

Last edited by strider3700; 12-07-10 at 03:05 PM.. Reason: found pachube
strider3700 is offline  
Old 12-09-10, 12:27 AM   #5
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

Ok getting data logging going on even at the simplest level turned out to be stupidly difficult for me. I've only been programming for a couple of decades now but I don't think I've ever touched the serial port so I decided to just grab an example someone else did and go with that.

My initial goal is just to get it writing every output to a local file. Nothing hard at all right?

Well I first tried a simple batch script that just monitored the serial port but that got a lot of corrupted crap or dropped data. So I figured since the arduino is using python lets just find a python script and do that. Well 4 hours and 5 examples later it wasn't working correctly. Once again lots of errors and corrupted data. I'm also not a python guy so debugging not knowing the language was a nightmare.

So I went to java which isn't my day to day but I've done a bunch of and it's damn close to c# which I mostly do these days. Then I figured since I have a java example lets learn eclipse, I've heard good things about it. Well it's pretty and shiny but it took me awhile to figure out how to add a jar to the project to make the rxtx library work. Anyways here's the code I found after I changed the port to be correct
Code:
package datalogger;

	import java.io.InputStream;
	import java.io.OutputStream;
	import gnu.io.CommPortIdentifier; 
	import gnu.io.SerialPort;
	import gnu.io.SerialPortEvent; 
	import gnu.io.SerialPortEventListener; 
	import java.util.Enumeration;

	public class SerialTest implements SerialPortEventListener {
		SerialPort serialPort;
	        /** The port we're normally going to use. */
		private static final String PORT_NAMES[] = { 
								"/dev/ttyUSB0", // Linux
								};
		/** Buffered input stream from the port */
		private InputStream input;
		/** The output stream to the port */
		private OutputStream output;
		/** Milliseconds to block while waiting for port open */
		private static final int TIME_OUT = 2000;
		/** Default bits per second for COM port. */
		private static final int DATA_RATE = 9600;

		public void initialize() {
			CommPortIdentifier portId = null;
			Enumeration portEnum = CommPortIdentifier.getPortIdentifiers();

			// iterate through, looking for the port
			while (portEnum.hasMoreElements()) {
				CommPortIdentifier currPortId = (CommPortIdentifier) portEnum.nextElement();
				for (String portName : PORT_NAMES) {
					if (currPortId.getName().equals(portName)) {
						portId = currPortId;
						break;
					}
				}
			}

			if (portId == null) {
				System.out.println("Could not find COM port.");
				return;
			}

			try {
				// open serial port, and use class name for the appName.
				serialPort = (SerialPort) portId.open(this.getClass().getName(),
						TIME_OUT);

				// set port parameters
				serialPort.setSerialPortParams(DATA_RATE,
						SerialPort.DATABITS_8,
						SerialPort.STOPBITS_1,
						SerialPort.PARITY_NONE);

				// open the streams
				input = serialPort.getInputStream();
				output = serialPort.getOutputStream();

				// add event listeners
				serialPort.addEventListener(this);
				serialPort.notifyOnDataAvailable(true);
			} catch (Exception e) {
				System.err.println(e.toString());
			}
		}

		/**
		 * This should be called when you stop using the port.
		 * This will prevent port locking on platforms like Linux.
		 */
		public synchronized void close() {
			if (serialPort != null) {
				serialPort.removeEventListener();
				serialPort.close();
			}
		}

		/**
		 * Handle an event on the serial port. Read the data and print it.
		 */
		public synchronized void serialEvent(SerialPortEvent oEvent) {
			if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
				try {
					int available = input.available();
					byte chunk[] = new byte[available];
					input.read(chunk, 0, available);

					// Displayed results are codepage dependent
					System.out.print(new String(chunk));
				} catch (Exception e) {
					System.err.println(e.toString());
				}
			}
			// Ignore all the other eventTypes, but you should consider the other ones.
		}
	public static void main(String[] args) {
		SerialTest main = new SerialTest();
		main.initialize();
		System.out.println("Started");
	}

}
I got it from here. Arduino playground - Java
I like that it's event driven it totally gets rid of all those issues I had with the other attempts. Anyways the hard part was adding the rxtx jar recognized in eclipse. I followed this partially (it's for windows I'm on linux) Using RXTX In Eclipse - Rxtx. That got it working so I can now go back to some old school reading and writing to disk...

I think the major Issue I've had with getting this part working is there are hundreds of ways to do it and none of them are amazingly well documented. The arduino people of course don't recommend one method over another and google will net you thousands of partial chunks of code pointing you in the right direction but none of them are plug and play.
strider3700 is offline  
Old 12-09-10, 06:08 AM   #6
Daox
Administrator
 
Daox's Avatar
 
Join Date: Aug 2008
Location: Germantown, WI
Posts: 5,482
Thanks: 1,117
Thanked 370 Times in 301 Posts
Default

While I haven't taken the time to really dig into your posts on this yet, please keep posting as it does look quite interesting!
__________________
Current project -
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.



To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
&
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
Daox is offline  
Old 12-09-10, 12:06 PM   #7
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

I have to say I love this thing so far. Last night it logged temperatures in my office every minute to a csv file. This morning I threw it into a spreadsheet and charted it.



The spikes every 15 minutes or so are the circulating fan for the furnace kicking on. The heat pump is not actively heating the fan is just helping to move the hot air from the woodstove around. I was never sure if it was actually effective but it is obviously pushing a bit of hot air in here.

THat last spike is also something I've noticed manually watching the sensor data. Thats me coming into the office and sitting down 18 inches from the sensor without a shirt on. People release a lot of heat.

Tonights goal will be to get the java program parsing the incoming stream better. Right now I just raw right it to a file and that is handling the new lines and putting the chunks together. I want to get it actually parsing first of all complete lines then split those lines up so that I can start doing averaging and so on inside of the program.

Also Sensor 282895C50200009B is a really crappy label. That is one of my complaints with the one wire system so far. I will need to install each sensor record it's ID in HEX then label it so that when I have 20 of them out there I know which room has which sensor so that I can apply a label to them. If not it's going to be a game of put your hand on it and sit here watching for the temp to go up then apply the label.
Attached Thumbnails
Click image for larger version

Name:	office temps.jpg
Views:	1898
Size:	11.1 KB
ID:	1135  
strider3700 is offline  
Old 12-10-10, 01:17 PM   #8
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

To not screw up my simple logging and get an easy pretty graph I had turned off the second sensor reading on the arduino rather then parse it on the java program. Last night I turned it back on and discovered I had an issue. The Event that fires saying data is on the serial port grab it would happen quicker then the java program could parse the first event so basically I'd get the entire first sensor in except for the new line character and then the second sensor would partially send garbling the results. I went to bed thinking that this would suck to handle, I'd need a second thread to process data and one to just grab the data. This morning I woke up realizing it's far far easier to just add a delay between sending each sensors data. so the send part on my arduino now looks like this.
Code:
printData(s1);
delay(100);
printData(s2);
It's a hack but it works. I'll play around with the delay to see how short I can get it without causing issues. I don't want to take 3 seconds to read in 30 sensors worth of data if possible....

<edit> A delay of 30 between each sensor write is good enough to allow me to output 100 sensors worth of readings. I've moved the delay into the print data function on the arduino. </edit>

Last edited by strider3700; 12-10-10 at 01:45 PM.. Reason: figured out an acceptable delay
strider3700 is offline  
Old 12-10-10, 01:32 PM   #9
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

And last nights office temps

You can clearly see the effects of me closing the office door right at the start and then opening it in the morning right at the end. THe gradual middle of the night climb is from me getting the fire going better.

Attached Thumbnails
Click image for larger version

Name:	Screenshot.jpg
Views:	2214
Size:	10.1 KB
ID:	1136  
strider3700 is offline  
Old 12-10-10, 01:47 PM   #10
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

My last edit reminded me. I started using the dallas temperatures library on the arduino so it's code is very different and much cleaner. It's still in bodge it together to see if it works stage though so it's far from pretty but here is the complete arduino code as it currently stands.

Code:
#include <OneWire.h>
#include <DallasTemperature.h>

// Data wire is plugged into port 22 on the Arduino
#define ONE_WIRE_BUS 22
#define TEMPERATURE_PRECISION 9
#define delayTime 2000     // the delay before restarting the loop

// Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);

// Pass our oneWire reference to Dallas Temperature. 
DallasTemperature sensors(&oneWire);

// arrays to hold device addresses
DeviceAddress s1, s2;

void setup(void)
{
  // start serial port
  Serial.begin(9600);
  
  // Start up the library
  sensors.begin();

  //printStatus();

  // assign address manually.  the addresses below will beed to be changed
  // to valid device addresses on your bus.  device address can be retrieved
  // by using either oneWire.search(deviceAddress) or individually via
  // sensors.getAddress(deviceAddress, index)
  //insideThermometer = { 0x28, 0x1D, 0x39, 0x31, 0x2, 0x0, 0x0, 0xF0 };
  //outsideThermometer   = { 0x28, 0x3F, 0x1C, 0x31, 0x2, 0x0, 0x0, 0x2 };

  // search for devices on the bus and assign based on an index.  ideally,
  // you would do this to initially discover addresses on the bus and then 
  // use those addresses and manually assign them (see above) once you know 
  // the devices on your bus (and assuming they don't change).
  // 
  // method 1: by index
  if (!sensors.getAddress(s1, 0)) Serial.println("Unable to find address for Device 0"); 
  if (!sensors.getAddress(s2, 1)) Serial.println("Unable to find address for Device 1"); 

  // method 2: search()
  // search() looks for the next device. Returns 1 if a new address has been
  // returned. A zero might mean that the bus is shorted, there are no devices, 
  // or you have already retrieved all of them.  It might be a good idea to 
  // check the CRC to make sure you didn't get garbage.  The order is 
  // deterministic. You will always get the same devices in the same order
  //
  // Must be called before search()
  //oneWire.reset_search();
  // assigns the first address found to insideThermometer
  //if (!oneWire.search(insideThermometer)) Serial.println("Unable to find address for insideThermometer");
  // assigns the seconds address found to outsideThermometer
  //if (!oneWire.search(outsideThermometer)) Serial.println("Unable to find address for outsideThermometer");

  
  // set the resolution to 12 bit
  sensors.setResolution(s1, 12);
  sensors.setResolution(s2, 12);

  
}

// function to print a device address
void printAddress(DeviceAddress deviceAddress)
{
  for (uint8_t i = 0; i < 8; i++)
  {
    // zero pad the address if necessary
    if (deviceAddress[i] < 16) Serial.print("0");
    Serial.print(deviceAddress[i], HEX);
  }
}

void printStatus(void)
{
  // locate devices on the bus
  Serial.print("Locating devices...");
  Serial.print("Found ");
  Serial.print(sensors.getDeviceCount(), DEC);
  Serial.println(" devices.");

  // report parasite power requirements
  Serial.print("Parasite power is: "); 
  if (sensors.isParasitePowerMode()) Serial.println("ON");
  else Serial.println("OFF");
  
  // show the addresses we found on the bus
  Serial.print("Device 0 Address: ");
  printAddress(s1);
  Serial.println();

  Serial.print("Device 1 Address: ");
  printAddress(s2);
  Serial.println();
  
  // print the device resolutions
  Serial.print("Device 0 Resolution: ");
  Serial.print(sensors.getResolution(s1), DEC); 
  Serial.println();

  Serial.print("Device 1 Resolution: ");
  Serial.print(sensors.getResolution(s2), DEC); 
  Serial.println();
}
// function to print the temperature for a device
void printTemperature(DeviceAddress deviceAddress)
{
  float tempC = sensors.getTempC(deviceAddress);
  //Serial.print("Temp C: ");
  Serial.print(tempC);
}

// function to print a device's resolution
void printResolution(DeviceAddress deviceAddress)
{
  Serial.print("Resolution: ");
  Serial.print(sensors.getResolution(deviceAddress));
  Serial.println();    
}

// main function to print information about a device
void printData(DeviceAddress deviceAddress)
{
  //Serial.print("Device Address: ");
  printAddress(deviceAddress);
  Serial.print(",");
  printTemperature(deviceAddress);
  Serial.println();
  delay(30);
}

void loop(void)
{ 
  // call sensors.requestTemperatures() to issue a global temperature 
  // request to all devices on the bus
  //Serial.print("Requesting temperatures...");
  sensors.requestTemperatures();
  //Serial.println("DONE");

  // print the device information
  printData(s1);
  printData(s2);
  delay(delayTime);
}

strider3700 is offline  
Closed Thread


Tags
data, logging

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 05:36 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.
Ad Management by RedTyger
Inactive Reminders By Icora Web Design