Home energy data logging project

[David_L_G]

Hey thanks. Its good to know I don't have to reinvent the wheel. I'm a bit of a Python fan too, so this could be fun!

[strider3700]

I started working on getting the ethernet shield working last night. Right away I hit an issue of the arduino no longer showing up as a serial device when connected. Lots of errors in my logs about an unknown USB device. Remove the shield plug the arduino back in and it's recognized just fine. Many hours later this morning adafruit support suggested I try powering it via battery and check the voltages. Sure enough it works when powered by 9V. Checking the voltages the arduino it's self is screwed up when it comes to the 3.3v and 5V lines.

Moral of the story, cheap chinese knockoffs aren't necessarily as good as the original Since this will long term be powered via 9V wallwart and I have no money I'll continue on with what I've got.

I still haven't done anything other then plug the two together and verify that it's at least recognized by the PC.

[Piwoslaw]

The first project with my new Arduino Duemilanove was to measure the temperature with a MCP9700 thermistor. No problem, I even got a second one going. Next, I'd like to log the data, either from the serial port or straight to the Seeed Studio SD card shield I have.

I'm just starting my journey with Arduino, and my programming background is C, not C++, C#, Java, or any other object or script stuff So here are the problems I've encountered (any help would be very welcome):

1. I have no idea how to log what's coming in through the serial port. I've searched around and found some VBScript on a few forums, but I have no idea how use that code I have to use WinXP for the next few months, but hopefuly I'll be back to Linux after that.
2. I also have no idea how to program the Arduino to use the SD card shield, or how to write to it.
3. The SD shield uses the 3.3V, 5V, GND, and digital 9-13 pins on my Duemilanove, so how can get 3.3V or 5V for the sensors I'd be logging data from? Will I have to get my solder iron out?

[AC_Hacker]

Originally Posted by Piwoslaw

Next, I'd like to log the data, either from the serial port or straight to the Seeed Studio SD card shield I have.

Piwoslaw,

I mentioned a locally-designed Arduino board in previous posts, that integrates the SD card shield.

Here is the board.


What might be more interesting for you is the firmware that you might be able to learn from, that already does a lot of the functions you want to do. Of course, you have to tweak bits of the code to work properly in your setup.

Regards,

-AC_Hacker

[David_L_G]

I have been eyeing off an Arduino "compatible" board with both Ethernet and micro SD card slot in one unit. Its called an Etherten from FreeTronics and I would be interested in opinions on this sort of thing too. Are there good reasons to instead go for a basic board then add the different shields for ethernet and SD card?

Unfortunately my post count on this forum won't allow me to post a link or image, but its from jaycar.com.au search for "etherten"

regards
David

EDIT (Piwoslaw): Here is the link: EtherTen - JayCar Electronics

[Piwoslaw]

AC - Thanks, I saw that link before and even looked for it locally, but couldn't find it. The board I got was on sale since it's been discontinued, and the SD shield was cheap when ordered with it. Now I see that I should have researched it. Anyhow, I've found firmware for my SD card shield, but I have no idea how to use it, yet. I thought that maybe someone here has used this shield and had some tips ready.

[Piwoslaw]

OK, it looks like I've solved most of the problems
First, the power supply. As mentioned earlier, Seeedstudio's SD card shield plugs into both the digital pins (D9-D13 plus GND) and the power supply pins (3.3V, 5V, and 2xGND). Luckily, the shield can be powered either from the 3.3V pin, or from D9, you choose this with a small switch. So by switching to D9 and adding the following lines to the code, the shield doesn't really need the power supply pins.

pinMode(9, OUTPUT); 
digitalWrite(9, HIGH);

Now that the shield doesn't need the power supply sockets, but my sensors do, I only pressed the digital pins into the sockets, slightly bending the pins:

It's not elegant, but it works.
Another workaround (which I may end up doing) is to use the ICSP socket to power my sensors (pin 2 = 5V, pin 6 = GND).

Next, I got the Arduino to write, then append, data onto the SD card by using the FAT16 libraries. I took one of the ready examples and hacked away.

When I was testing the data logging with two temperature sensors I started getting problems. The basic program opened a file for append, recorded both readings, closed the file, waited 1 second, 60 times. When hooked up to the computer through the USB the reading would be very jumpy - anywhere from 14°C to 19°C, but when powered by a 12V wallwart they more or less smoothed out. At least in the beginning, because when repeating the program the first sensor's readings would be around 33°C. Only the first run after uploading the sketch would be OK. I then changed the program to close then open the file every ten seconds and what I got was every tenth reading from sensor 1 was too high. So I added a 'dummy' read and delay before the real sensor read. This seems to work OK.
Now the only problem seems to be that even now and then there is a random reading. This doesn't happen every time the program is run, but when it does there is only reading which is off, for example:

...
26: 16.89 16.89
27: 16.89 16.89
28: 16.89 16.89
29: 16.89 16.89
30: 16.89 17.38
31: 16.89 28.61
32: 16.89 16.89
33: 16.89 16.89
34: 16.89 16.89
35: 16.89 16.89
36: 16.89 16.89
37: 16.89 17.38
38: 16.89 17.38
...

or

...
27: 19.82 19.82
28: 19.82 19.82
29: 19.82 19.82
30: 19.82 19.82
31: 19.82 19.82
32: 19.82 19.82
33: 48.63 19.82
34: 19.82 19.82
35: 19.82 19.82
36: 19.82 19.82
37: 19.82 19.82
38: 19.82 19.82
...

Below is the whole code I'm using, hopefully it'll help someone.

#include <Fat16.h>
#include <Fat16util.h> // use functions to print strings from flash memory

SdCard card;
Fat16 file;
float temp;

void setup(void) {
  pinMode(9, OUTPUT); 
  digitalWrite(9, HIGH);
      
  // initialize the SD card
  card.init();
  
  // initialize a FAT16 volume
  Fat16::init(&card);

  // create a new file
  char name[] = "WRITE00.cvs";
  for (uint8_t i = 0; i < 100; i++) {
    name[5] = i/10 + '0';
    name[6] = i%10 + '0';
    // O_CREAT - create the file if it does not exist
    // O_APPEND - seek to the end of the file prior to each write
    // O_WRITE - open for write
    if (file.open(name, O_CREAT | O_APPEND | O_WRITE)) break;
  }
  if (!file.isOpen()) exit;
  file.close();
  
  // clear write error
  file.writeError = false;

  // write data to file
  for (char i=0; i<60; i++) {
    if (!(i%10)) if (!file.open(name, O_CREAT | O_APPEND | O_WRITE)) break;
    file.print(i,DEC);
    file.print(": ");
    if (!(i%10)) {
      temp = analogRead(0);
      delay(100);
    }
    temp = analogRead(0)*5/1024.0;
    temp = temp - 0.5;
    temp = temp / 0.01;
    file.print(temp);
    file.print(" ");
    temp = analogRead(1)*5/1024.0;
    temp = temp - 0.5;
    temp = temp / 0.01;
    file.println(temp);
    if (!((i+1)%10)) {
      if (!file.close()) break;
      delay(900);}
      else delay(1000);
  }
  
  // close file and force write of all data to the SD card
  file.close();
}

void loop(void) {};

[Daox]

If your wires are long (perhaps even if they're short) you could be getting noise over the wire. I had this problem when I did my attic fan project. All I did was add a capacitor inline with the sensor signal wire and it smoothed the signal out. Of course this also slows response time a bit.

[Piwoslaw]

The wires were short, just to the breadboard, while the USB cable was a standard 1.8m (though this probably didn't make a difference). This evening I'll try to rig up a few 1 meter wires to start some real testing. But it's good to know about the caps. Is there a rule about the cap size vs wire length?

[Daox]

Not that I know of. I just put a small one on, I think it was 10uf. It still responds plenty fast and I'm only sampling every second anyways. From my testing (holding the sensor in my hand or blowing on it to heat it up) it responded well enough for my application.