Home energy data logging project - Page 6

AC_Hacker · 03-06-11, 08:54 PM

Strider,

Last time I tuned into your project, you were still wrestling with the problem of measuring power. As I recall, you were measuring current and then multiplying by the assumed voltage, and ignoring PF issues.

I got to thinking about this and just remembered that there was a project call Tweet-a-Watt, where a guy used an off-the-shelf Kill-a-Watt and hacked into it and made it send out power consumption info via Twitter.

So, I was thinking that he must have found some way to intercept the power data from the Kill-a-Watt while it was on it's way to the LCD and decoded it and sent that on to Twitter.

I went to Tweet-a-Watt to see just how he did it, and NO I was wrong, he had a XBee...

.

...pick up the current and the voltage info from the Kill-a-Watt...

... and sent it on to a computer where a Python Script takes care of the dirty work.

Details of this approach can be found here.

Best Regards,

-AC_Hacker

strider3700 · 03-07-11, 12:17 AM

that works fine for 110V if I want to have $20-$30 sensors everywhere. Doesnt handle 220V at all which is my real hold up. I can't even get a one time reading on the 220V lines to Know how close my estimate math is...

AC_Hacker · 03-20-11, 01:53 PM

Quote:

Originally Posted by strider3700

that works fine for 110V if I want to have $20-$30 sensors everywhere. Doesnt handle 220V at all which is my real hold up. I can't even get a one time reading on the 220V lines to Know how close my estimate math is...

I'm sorry I didn't make this clearer, but I thought the most useful part for your project was the Python script that combined the current and voltage data to yield power data.

-AC_Hacker

AC_Hacker · 03-22-11, 11:10 AM

Strider,

I'm about to string some 1-wire serial cable throughout my house. I figure 12 sensors max. Since the 1-wire concept is to use a serial as opposed to parallel topography, this is going to mean a lot of doubling back of serial cable routing.

Seems to me there ought to be a better way.

How did you go about it? Any advice before I start drilling holes and running cable?

-AC_Hacker

strider3700 · 03-22-11, 12:09 PM

I'm using 4 wire cables for everything and I run a single run out to each location basically with short Y's soldered in every now and again. I found that the longer the run the lower number of devices I could run on it before I started getting garbage data back. I figured this had to do with try to push too much through 1 pin on the arduino. So my current set up is

10' to 1 temp sensor then 40' to another temp sensor that shares an arduino pin with my office sensor which is right on the board.

15' to a temp sensor sharing arduino pin with
30' to temp sensor that then continues 10' to temp sensor and 6' to temp sensor. soldering 3 wires to each of the sensor legs sucked and is ugly but works. I tried adding a 4th to add two more sensor's eventually but it wouldn't work.

when I do my long runs for the solar monitoring they will be close to 70' away from the arduino so I intend to run 1 cable to each and use a separate pin for each if necessary. The wire is really cheap and it's just going in a buried conduit so nothing to it.

strider3700 · 07-15-11, 01:11 AM

it's been a few months and time for an update.

I've been fighting most of this time with keeping the connection between the arduino and the computer going for long periods. I'd get 4 or 5 days and then the usb/serial connection wouldn't connect and it would stop logging. I tried multiple ways to handle the data transfer, arduino push, computer pull, different programs on the computer side doing the catching/pushing, nothing would give me a rock solid connection where I can set it and forget it.

I was also starting to get annoyed that my pc is running 24/7 never going to sleep because it needs to get data every second or minute depending on the method I was trying. Hitting google revealed that very few people trust the serial connection to just work. Standard practice appears to be treat the arduino as a stand alone computer not just a data collector and add either wireless or an ethernet connection to it.

So I have ordered up an ethernet shield that is basically plug and play(minor coding required) as well as a screw shield which is basically a protoboard with a bunch of screw terminals on it. My original plan was to design a circuit and etch it then add a bunch of screw terminals and the other components so I figured the $16 it cost was entirely worth the savings in time. The total cost was $61 $70 shipped with most of that being for the ethernet shield. Not sure why they are so pricey but it was consistently between $40 and $50 for the different versions from different sites.

When they arrive I'll spend some time getting the ethernet connection working. I think I'll dump directly to on online site and skip going to my own machine. It may take a bit more work but longterm that is the goal anyways.

AC_Hacker · 07-15-11, 08:05 AM

Quote:

Originally Posted by strider3700

I've been fighting most of this time with keeping the connection between the arduino and the computer going for long periods.

Strider,

Sorry you're having trouble with your data logging. I hope your Ethernet shield does the trick.

In the meantime, that little Arduino datalogging board I mentioned in this post is logging merrily away.

True, I don't have the large, branching array of sensors you have (which may be a source of your problems)... I'm just using one temp sensor for now.

This board is an Arduino with data logging stuff on board. For instance, it has a 16GB SD card on board and it just keeps logging data to that device, with a date & time stamp for each reading cycle (it has an onboard RTC). I did a calculation a while back as to how much data the 16GB card would hold, and with one sensor (it has one 1-wire temp sensor on board, and a 3-pin header for a string of sensors on the same 1-wire line). Anyway, I calculated that it would log about 225 years of data for one device, and half that for two devices, etc. Travis said he tested it out to 70 devices. So with 12 devices, it would log over 18 years.

So, it has a serial port just like your board, and you could pull data just like you are now. Or you could do a periodic data dump. Either way, it still keeps writing data to the SD card, with time stamp for each cycle. So if your computer goes down, or your serial connection fails, you're still logging data.

Right now, I have a tiny cell phone power supply that is powering it. I should put a fat cap across the power supply to keep it going in the event of temporary power lapses.

I haven't measured power consumption, but it's probably under a watt.

(NOTE-1: I measured the power at the cell phone power supply, and I'm reading 1.1ma @ 120V. If I'm calculating that right, it means that with one 1-wire device, the power draw is about 0.132watt... pretty darn low.
NOTE-1 (revised): I hooked the data logger up to a kill-a-watt and it is telling me that the power supply & data logger are pulling a total of 3 watts, which I believe. More than I wanted but way less than a separate computer. At my power rate (11 cents/KW), that's around $2.89/yr)

Travis sold me my board for $35... I think I was a beta-tester.

He is very amiable and knowledgeable.

Just a thought...

-AC_Hacker

strider3700 · 07-15-11, 12:28 PM

the ethernet shield has a slot for sd cards of some variant on it so I'll have the ability to log to that if I want. I managed to keep the arduino running and reading temps for ages and I had it store the average temps for each of the sensors indefinitely in the arduino itself not a problem. The issue has always been that I want real time( a few seconds to 1 minute delayed is fine) readings available on the internet. the serial port to dump the data automatically just wasn't stable enough. I'm not sure if it's the arduino, my OS or my computer that is causing the issue.

Ethernet will let me sleep this machine and leave the arduino running instead. It should ship early next week I beleive

David_L_G · 09-25-11, 03:37 AM

Hi,

I'm just starting out on a project to monitor temperatures at different heights on my rooftop solar hot water tank. This is a learning thing for me as I am a programmer and have not dabbled with electronics much at all. However, I've taken the plunge and ordered some DS18S20s and a USB 1-Wire adaptor as a starting point. I'm really at early days but your posts above are most interesting.

Logging is one issue - I don't want to leave my PC running all the time. One direction I think would be cool would be to have a the device which reads the 1-wire sensors to connect via ethernet to my broadband router (which is on all the time), then post the data to a website where it can be stored (if nothing else) and graphed etc (even better).

There are two unknowns there for me.
1. The hardware that will sit between the 1-wire setup and the router. I've seen things like webcontrol-8, OW-Server etc which might do the trick. Does the Arduino also have http capability.

2. The website. I don't want to spend lots of money hosting this, sureley someone has done something like this already. It would need to be a simple website which accepts GET and POST to save data readings. Simple? Any ideas on this area would be most interesting.

Thanks
David

strider3700 · 09-25-11, 10:52 AM

here's a tutorial on getting the data into a web based database
Tweet-a-Watt - Data collection
there they use a wireless method of connecting to the net

I'm going to use a wired method Arduino Ethernet shield with micro SD connector - Assembled ID: 201 - $45.00 : Adafruit Industries, Unique & fun DIY electronics and kits
but the webcode should be pretty close.

The parts are sitting on my desk but I haven't had a chance to play with them lately.

03-06-11, 08:54 PM	#51
AC_Hacker Supreme EcoRenovator Join Date: Mar 2009 Location: Portland, OR Posts: 4,004 Thanks: 303 Thanked 723 Times in 534 Posts	Measuring Power... Strider, Last time I tuned into your project, you were still wrestling with the problem of measuring power. As I recall, you were measuring current and then multiplying by the assumed voltage, and ignoring PF issues. I got to thinking about this and just remembered that there was a project call Tweet-a-Watt, where a guy used an off-the-shelf Kill-a-Watt and hacked into it and made it send out power consumption info via Twitter. So, I was thinking that he must have found some way to intercept the power data from the Kill-a-Watt while it was on it's way to the LCD and decoded it and sent that on to Twitter. I went to Tweet-a-Watt to see just how he did it, and NO I was wrong, he had a XBee... . ...pick up the current and the voltage info from the Kill-a-Watt... ... and sent it on to a computer where a Python Script takes care of the dirty work. Details of this approach can be found here. Best Regards, -AC_Hacker __________________ I'm not an HVAC technician. In fact, I'm barely even a hacker...

03-22-11, 11:10 AM	#54
AC_Hacker Supreme EcoRenovator Join Date: Mar 2009 Location: Portland, OR Posts: 4,004 Thanks: 303 Thanked 723 Times in 534 Posts	Stringing 1-Wire Serial Cable? Strider, I'm about to string some 1-wire serial cable throughout my house. I figure 12 sensors max. Since the 1-wire concept is to use a serial as opposed to parallel topography, this is going to mean a lot of doubling back of serial cable routing. Seems to me there ought to be a better way. How did you go about it? Any advice before I start drilling holes and running cable? -AC_Hacker __________________ I'm not an HVAC technician. In fact, I'm barely even a hacker...

09-25-11, 03:37 AM	#59
David_L_G Lurking Renovator Join Date: Sep 2011 Location: Clarence Gardens, South Australia Posts: 5 Thanks: 0 Thanked 0 Times in 0 Posts	Web interface via ethernet to get around logging issues? Hi, I'm just starting out on a project to monitor temperatures at different heights on my rooftop solar hot water tank. This is a learning thing for me as I am a programmer and have not dabbled with electronics much at all. However, I've taken the plunge and ordered some DS18S20s and a USB 1-Wire adaptor as a starting point. I'm really at early days but your posts above are most interesting. Logging is one issue - I don't want to leave my PC running all the time. One direction I think would be cool would be to have a the device which reads the 1-wire sensors to connect via ethernet to my broadband router (which is on all the time), then post the data to a website where it can be stored (if nothing else) and graphed etc (even better). There are two unknowns there for me. 1. The hardware that will sit between the 1-wire setup and the router. I've seen things like webcontrol-8, OW-Server etc which might do the trick. Does the Arduino also have http capability. 2. The website. I don't want to spend lots of money hosting this, sureley someone has done something like this already. It would need to be a simple website which accepts GET and POST to save data readings. Simple? Any ideas on this area would be most interesting. Thanks David

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

03-07-11, 12:17 AM	#52
strider3700 Master EcoRenovator Join Date: Dec 2008 Location: Vancouver Island BC Posts: 745 Thanks: 23 Thanked 37 Times in 30 Posts	that works fine for 110V if I want to have $20-$30 sensors everywhere. Doesnt handle 220V at all which is my real hold up. I can't even get a one time reading on the 220V lines to Know how close my estimate math is...

03-22-11, 12:09 PM	#55
strider3700 Master EcoRenovator Join Date: Dec 2008 Location: Vancouver Island BC Posts: 745 Thanks: 23 Thanked 37 Times in 30 Posts	I'm using 4 wire cables for everything and I run a single run out to each location basically with short Y's soldered in every now and again. I found that the longer the run the lower number of devices I could run on it before I started getting garbage data back. I figured this had to do with try to push too much through 1 pin on the arduino. So my current set up is 10' to 1 temp sensor then 40' to another temp sensor that shares an arduino pin with my office sensor which is right on the board. 15' to a temp sensor sharing arduino pin with 30' to temp sensor that then continues 10' to temp sensor and 6' to temp sensor. soldering 3 wires to each of the sensor legs sucked and is ugly but works. I tried adding a 4th to add two more sensor's eventually but it wouldn't work. when I do my long runs for the solar monitoring they will be close to 70' away from the arduino so I intend to run 1 cable to each and use a separate pin for each if necessary. The wire is really cheap and it's just going in a buried conduit so nothing to it.

07-15-11, 01:11 AM	#56
strider3700 Master EcoRenovator Join Date: Dec 2008 Location: Vancouver Island BC Posts: 745 Thanks: 23 Thanked 37 Times in 30 Posts	it's been a few months and time for an update. I've been fighting most of this time with keeping the connection between the arduino and the computer going for long periods. I'd get 4 or 5 days and then the usb/serial connection wouldn't connect and it would stop logging. I tried multiple ways to handle the data transfer, arduino push, computer pull, different programs on the computer side doing the catching/pushing, nothing would give me a rock solid connection where I can set it and forget it. I was also starting to get annoyed that my pc is running 24/7 never going to sleep because it needs to get data every second or minute depending on the method I was trying. Hitting google revealed that very few people trust the serial connection to just work. Standard practice appears to be treat the arduino as a stand alone computer not just a data collector and add either wireless or an ethernet connection to it. So I have ordered up an ethernet shield that is basically plug and play(minor coding required) as well as a screw shield which is basically a protoboard with a bunch of screw terminals on it. My original plan was to design a circuit and etch it then add a bunch of screw terminals and the other components so I figured the $16 it cost was entirely worth the savings in time. The total cost was $61 $70 shipped with most of that being for the ethernet shield. Not sure why they are so pricey but it was consistently between $40 and $50 for the different versions from different sites. When they arrive I'll spend some time getting the ethernet connection working. I think I'll dump directly to on online site and skip going to my own machine. It may take a bit more work but longterm that is the goal anyways.

07-15-11, 12:28 PM	#58
strider3700 Master EcoRenovator Join Date: Dec 2008 Location: Vancouver Island BC Posts: 745 Thanks: 23 Thanked 37 Times in 30 Posts	the ethernet shield has a slot for sd cards of some variant on it so I'll have the ability to log to that if I want. I managed to keep the arduino running and reading temps for ages and I had it store the average temps for each of the sensors indefinitely in the arduino itself not a problem. The issue has always been that I want real time( a few seconds to 1 minute delayed is fine) readings available on the internet. the serial port to dump the data automatically just wasn't stable enough. I'm not sure if it's the arduino, my OS or my computer that is causing the issue. Ethernet will let me sleep this machine and leave the arduino running instead. It should ship early next week I beleive

09-25-11, 10:52 AM	#60
strider3700 Master EcoRenovator Join Date: Dec 2008 Location: Vancouver Island BC Posts: 745 Thanks: 23 Thanked 37 Times in 30 Posts	here's a tutorial on getting the data into a web based database Tweet-a-Watt - Data collection there they use a wireless method of connecting to the net I'm going to use a wired method Arduino Ethernet shield with micro SD connector - Assembled ID: 201 - $45.00 : Adafruit Industries, Unique & fun DIY electronics and kits but the webcode should be pretty close. The parts are sitting on my desk but I haven't had a chance to play with them lately.