EcoRenovator  

Go Back   EcoRenovator > Improvements > Appliances & Gadgets
Advanced Search
 


Blog Register 60+ Home Energy Saving Tips Recent Posts Search Today's Posts Mark Forums Read


Closed Thread
 
Thread Tools Display Modes
Old 12-10-10, 09:34 PM   #11
AC_Hacker
Supreme EcoRenovator
 
AC_Hacker's Avatar
 
Join Date: Mar 2009
Location: Portland, OR
Posts: 4,002
Thanks: 303
Thanked 712 Times in 532 Posts
Default

Quote:
Originally Posted by strider3700 View Post
You can clearly see the effects of me closing the office door right at the start and then opening it in the morning right at the end. THe gradual middle of the night climb is from me getting the fire going better.[/IMG]
I confess that I haven't been following this thread from the beginning, so pardon if I've missed something...


I understand the trend lines, but what causes the short-duration periodic swings?

-AC_Hacker

AC_Hacker is offline  
Old 12-10-10, 10:32 PM   #12
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

I have the fan in the forced air system set to circulate so it comes on for a few minutes every 15 minutes or so. the room with the sensor is easily the coldest in the house being the basement north west corner so slightly warmer air gets blown in when that happens. if the air is warm enough the room gradually climbs as happened last night. If the air isn't the room gradually drops like happened in the first graph.

I picked up 100 meters of cat 3 today and will put the second sensor in the room with the wood stove if I get a chance tonight. It'll be nice to see multiple temps logged and see how the rooms are different.
strider3700 is offline  
Old 12-11-10, 12:37 AM   #13
AC_Hacker
Supreme EcoRenovator
 
AC_Hacker's Avatar
 
Join Date: Mar 2009
Location: Portland, OR
Posts: 4,002
Thanks: 303
Thanked 712 Times in 532 Posts
Default

Quote:
Originally Posted by strider3700 View Post
I have the fan in the forced air system set to circulate so it comes on for a few minutes...
OK, that explains everything.

* * *

Say, I'm only marginally able to tweak Arduino code, and what you are doing is right in line with something I have wanted to do for a couple of years...

As you may know, I'm doing the Homemade HeatPump Manifesto thread, and I have wanted to build a vertical multi-sensor data logger to monitor temperatures of the earth at various depths in my GSHP loop field.

My idea is to get a fifteen foot length of pipe, copper is ok, PVC would work too, and put sensors every two feet or so, and fill the pipe up with tar or plastic or something and bury it, putting a data cable from the pipe and sensors, into a foot deep trench, and run it into the house.

Then I would monitor the temps periodically like maybe once an hour or once a day... for several years.

I understand that 1-wire sensors are uniquely numbered, and quite accurate, and for design simplicity, I would run them off of three wires, Data, Vcc, and Ground. Screw the timeing issues, wire is cheap.

So, do you think you could help me out with code to read several 1-wire therrmal sensors?

I'm guessing it's nothing much more than cut & paste & paste & paste & paste, etc.

Tell me, am I over-simplifying?

Does your code include an initialization section that polls and records the unique identifiers of all sensors?

Thanks,

-AC_Hacker

Last edited by AC_Hacker; 12-11-10 at 12:39 AM..
AC_Hacker is offline  
Old 12-11-10, 12:55 AM   #14
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

Just FYI the ds18b20 sensors will not work over 100 meters of cat 3 28 gauge 2 pair in either parasitic or standard mode. I'm not shocked and don't need them to anyways but wanted to test. It does work just fine in parasitic for 10 meters though. My longest is about 25 meters but I won't be setting that test up tonight.
strider3700 is offline  
Old 12-11-10, 01:14 AM   #15
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

Quote:
Originally Posted by AC_Hacker View Post

So, do you think you could help me out with code to read several 1-wire therrmal sensors?
Not a problem. Before I go to bed I'm hoping to have the logging to csv of multiple sensors working. The reading is already there.

Your basic steps are -

first use a breadboard and wire up 1 sensor

Here's a nice page explaining it. Arduino and DS18B20 - 1-wire digital thermometer | Ogalik

on the arduino you'll see VCC and gnd so you plug them in. Pick any of the digital pins you just need to know the number.

Then load the arduino IDE and install the dallas temps and 1 wire library - it's pretty trivial and I can point it out if needed when the time comes, basically just put the correct file from a downloaded zip in the right space.

Then if you grab my code a couple of posts back paste it into the arduino IDE change the pin number at the top to whatever you used and hit compile it should compile with no errors. click upload and then fire up the serial monitor in the arduino IDE and it should tell you the sensors ID and the temperature.

record the ID and label the sensor, repeat till you have ID's for all of them

wire two of them up like in that page I posted and restart the serial monitor. The code will work fine for two with no changes as it currently is. Soon I'll make it work for any number (up to 64 is the limit I think) of sensors without any changes and you'll be good to go.

That will give you data being dumped into the arduino serial monitor. When the java part of the program is finished I'll post it up and you can dump the data to a csv. Or you can grab the code as it currently stands and track down the rxtx library and build it yourself.

As I go whatever I create while working on this project I'll post on here and anyone can take and do whatever with it. Just be forewarned I'm posting where I'm at every couple of days and much of it is far from optimal. I'll hopefully get it pretty by the end but if you grab something now just be aware that it's a very early stage work in progress/proof of concept.

I am hoping that the java code will be good enough in the end to be useful for others that want basic data logging via an arduino. Just have the arduino return the sensor ID and a value and the java program will put that to a csv file.
strider3700 is offline  
The Following User Says Thank You to strider3700 For This Useful Post:
AC_Hacker (12-11-10)
Old 12-11-10, 02:08 AM   #16
AC_Hacker
Supreme EcoRenovator
 
AC_Hacker's Avatar
 
Join Date: Mar 2009
Location: Portland, OR
Posts: 4,002
Thanks: 303
Thanked 712 Times in 532 Posts
Default

Thanks, most of it is pretty clear.

Tomorrow, I'll dig up my arduino board and give it a try.

-Regards,

-AC_Hacker
AC_Hacker is offline  
Old 12-11-10, 04:04 AM   #17
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

Ok I spent some time wiring up a second temp sensor and installing it in my rec room which has the woodstove. It was an easy/quiet wiring job to do in the middle of the night and it should be interesting to see how the temps work out in my office vs in there.

Then I decided to get down to work on the java side to get logging that in a nice easy to use method. I don't want to datalog every second or two's data but I don't want the inaccuracy associated with only collecting data every 1,2,5... whatever minutes. So I figured I'll collect data every second and average it for whatever time period becomes useful in the future.

So I decided first of all I need a new java class to create a sensor object for each sensor. it gives me the basics, set the name, id, get the name, id. also it has a Value stored in it. I can reset that, and add to it. each time I add to it I add one to the count and when I ask for the average I just take the value/count and return that. Pretty simple.
Code:
package datalogger;

public class Sensor {
	
	private String sensorID;
    private String sensorName;
    private float value;
    private int averageCount;
    
    Sensor(String inID,String inName)
    {
     sensorID = inID;
     sensorName = inName;
     value =0;
     averageCount = 0;
    }
    
    public String getID(){
    	return sensorID;
    }
    public String getID(String inName){
    	return sensorID;
    }
    
    public String getName(){
    	return sensorName;
    }
    
    public String getName(String inID){
    	return sensorName;
    }
        
    public void setName(String inName)
    {
    	sensorName = inName;
    }
    
    public void setID(String inID){
    	sensorID = inID;
    }
    public void resetValue()
    {
    	value =0;
    	averageCount =0;
    }
    
    public void addValueToAverage(float inValue)
    {
     value += inValue;
     averageCount++;
    }
    
    public String returnAverage()
    {
      float average = value/averageCount;
      return (String.valueOf(average));
    }       
    
}
The other class got some more functions calling and using the sensor class but nothing special. I'm thinking I should really create a new class to handle the data manipulation that will just call the serialtest and let it handle getting the data from the serial port then hand it over to this other class to deal with. I won't be working on this much if at all tomorrow though. I also somewhere need to add a timer that when the timer goes off will walk through the list of sensors and get their averages then output them to the CSV as 1 line to make for easy comparison graphs.

I was also thinking that long term I should make the program look at a simple configuration file to get the sensor ID's and names to make it easier for non programmers to use. As if I don't have enough to do...
strider3700 is offline  
Old 12-13-10, 03:38 AM   #18
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

Ok I've spent too much time on this lately but it's now doing the following.

The arduino dumps back all of the sensor data every 2 seconds with a 30 millisecond delay between each.

The java program collects the data and every 60 seconds prints the average for each sensor to a csv file the clears the average so we're getting a minute by minute recording. The format going into the csv file is Date,Time,S1,S2.... with the first line being a Header which loads the "common name" for the sensor so in my case, office, Rec room. I don't know if there is a guarantee that s1 will always be office and s2 always rec room. The program will always put the values to the correct sensor name however and it won't flip part way through a run. Worst case you run the program and it lists them office, rec room and then you stop the program restart it and it will continue appending but flipped so recroom, office. The Header will be rewritten if you were to do that.... Common names are currently hardcoded. Long term none of that should matter because I want to move away from dumb csv's and into some form of database.

anyways right now it's running collecting from the two different rooms and in the morning I can generate a new chart, this one with labels and two lines... That's enough to keep me happy for awhile data wise so when I return to this project it will be to get current readings working.

Now the code if anyone is interested. It's still rough... The arduino is the same as last time I posted I believe.

Logger.java the new main class
Code:
package datalogger;

import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Timer;

public class Logger {

	private static Date today;
	private static SimpleDateFormat  formatter = new SimpleDateFormat("yyyy/MM/dd,HH:mm");
	public static List<Sensor> sensorList=new ArrayList<Sensor>();
	private static SerialHandling handler ;
	private static Timer timer;
	public  static void main(String[] args) {
		loadSensorList();
		timer = new Timer();
		setTimer();
		
		handler  = new SerialHandling();
		handler.initialize();
		
	}
		
	
	private static void setTimer(){
		timer.scheduleAtFixedRate(
				new writeData(),  // calls the writeData class run method to write the data to csv
				0,  				// start right away
				60000             // run every 60 seconds
				//5000			  // run every 5 seconds for testing	
		);
	}
	public static void parseText(String line)
	{
		//System.out.println(line);
		if (line.contains(","))
		{
			String[] split = line.split(",");					
			String sensor = split[0];
			String value = split[1];

			averageValues(sensor,value);

			String name = getSensorName(sensor);					
			today = new Date();
			String time = formatter.format(today);
			//System.out.println(name+","+time+","+value);
			System.out.println(time+","+name+","+value);
			//writeText(time+" "+name+" "+ value+newline);
			//writeText(name+","+time+","+value+newline);
		}
	}
	// fills the List with the Sensors.  Currently hardcoded ID's
	public static void loadSensorList()	
	{		
		Sensor office = new Sensor("282895C50200009B","office");
		Sensor recroom = new Sensor("28BF90C50200007D","recroom");
		sensorList.add(office);
		sensorList.add(recroom);		
	}

	public static int getSensorCount()
	{
		return sensorList.size();
	}
	
	
	// find the sensor based on the ID
	private static Sensor getSensor(String inID)
	{
		int i =0;
		boolean done = false;
		Sensor temp = new Sensor("default","default");
		while ( i < sensorList.size() && done == false)
		{
			temp = sensorList.get(i);

			if (inID.equals(temp.getID()))
			{
				done = true;						
			}		
			i++;
		}
		return temp;
	}

	// returns the sensors common Name if in the list else the sensor ID is used as the name
	private static String getSensorName(String inID){

		String name = inID; //default to the ID if not found
		Sensor temp = getSensor(inID);				
		name = temp.getName();
		return name;
	}
	
	private static void averageValues(String id, String value){

		Sensor s = getSensor(id);
		try
		{
			float f = Float.valueOf(value.trim()).floatValue();
			s.addValueToAverage(f);
		}
		catch (NumberFormatException nfe)
		{
			System.out.println("NumberFormatException: " + nfe.getMessage());
		}

		//  System.out.println(id+" value "+value+" average "+s.returnAverage());

	}
}
WriteData.java - the timer that writes to the csv every minute
Code:
package datalogger;

import java.io.BufferedWriter;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.TimerTask;


// the Timer task the calls the outputting to the CSV file
public class writeData extends TimerTask {

	private static Date today;
	private static SimpleDateFormat  formatter = new SimpleDateFormat("yyyy/MM/dd,HH:mm");
	private boolean first = true;
	
	public void run() {
		//System.out.println("Generating csv");
		String row = buildRow();
		if (row.contains("NaN")==false) // done to stop that initial hit of empty from being written I could also delay 
		{								// the timer by 60 seconds but this always works no matter what the timer timing
			writeText(row);
		}		
	}
	
	//builds the row to output to CSV file. It hardcodes the header line in the first time called.   
	private String buildRow(){
		
		//row = Date,Time,s1,s2,...
		String row = "";
		// if the first line then dump in the header
		if (first == true)
		{
			row = "Date,Time";
			for (int i=0;i< Logger.getSensorCount() ;i++)
			{
				row+=","+Logger.sensorList.get(i).getName();
			}
			first = false;
			writeText(row);
			row ="";
		}
		
		today = new Date();
		String time = formatter.format(today);
		row += time;  // add the time to the row
		
		// now add all of the average values in the sensors
		for (int i=0;i< Logger.getSensorCount() ;i++)
		{
			
			row+=","+Logger.sensorList.get(i).returnAverage();	//get the average
			Logger.sensorList.get(i).resetValue();  // clear the averages
		}
		
		//System.out.println("writing -"+row);
		return row;
	}
	
	// for writing the text strings to the file	
	public synchronized void writeText(String inString){
		String strFilePath = "/home/strider/Logs/tempData.csv";
		String newline = System.getProperty("line.separator");
		try
		{
			Writer output = new BufferedWriter(new FileWriter(strFilePath,true));

			//FileWriter always assumes default encoding is OK!
			output.write( inString+newline);
			output.close();
		}
		catch(FileNotFoundException ex)
		{
			System.out.println("FileNotFoundException : " + ex);
		}
		catch(IOException ioe)
		{
			System.out.println("IOException : " + ioe);
		}
	}	
}
strider3700 is offline  
Old 12-13-10, 03:39 AM   #19
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

stupid maximum length rule


SerialHandler.java - handles the data on the serial port and passes it formatted to Logger
Code:
package datalogger;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import gnu.io.CommPortIdentifier; 
import gnu.io.SerialPort;
import gnu.io.SerialPortEvent; 
import gnu.io.SerialPortEventListener; 
import java.util.Enumeration;



public class SerialHandling implements SerialPortEventListener {
	SerialPort serialPort;
	/** The port we're normally going to use. */
	private static final String PORT_NAMES[] = { 
		"/dev/ttyUSB0", // Linux
	};
	/** Buffered input stream from the port */
	private InputStream input;
	/** The output stream to the port */
	private OutputStream output;
	/** Milliseconds to block while waiting for port open */
	private static final int TIME_OUT = 2000;
	/** Default bits per second for COM port. */
	private static final int DATA_RATE = 9600;		
	private String line = new String();

	public void initialize() {			
		CommPortIdentifier portId = null;
		Enumeration portEnum = CommPortIdentifier.getPortIdentifiers();

		// iterate through, looking for the port
		while (portEnum.hasMoreElements()) {
			CommPortIdentifier currPortId = (CommPortIdentifier) portEnum.nextElement();
			for (String portName : PORT_NAMES) {
				if (currPortId.getName().equals(portName)) {
					portId = currPortId;
					break;
				}
			}
		}

		if (portId == null) {
			System.out.println("Could not find COM port.");
			return;
		}

		try {
			// open serial port, and use class name for the appName.
			serialPort = (SerialPort) portId.open(this.getClass().getName(),
					TIME_OUT);

			// set port parameters
			serialPort.setSerialPortParams(DATA_RATE,
					SerialPort.DATABITS_8,
					SerialPort.STOPBITS_1,
					SerialPort.PARITY_NONE);

			// open the streams
			input = serialPort.getInputStream();
			output = serialPort.getOutputStream();

			// add event listeners
			serialPort.addEventListener(this);
			serialPort.notifyOnDataAvailable(true);
		} catch (Exception e) {
			System.err.println(e.toString());
		}
	}

	/**
	 * This should be called when you stop using the port.
	 * This will prevent port locking on platforms like Linux.
	 */
	public synchronized void close() {
		if (serialPort != null) {
			serialPort.removeEventListener();
			serialPort.close();
		}
	}

	/**
	 * Handle an event on the serial port. Read the data and print it.
	 */
	public synchronized void serialEvent(SerialPortEvent oEvent) {

		if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
			try {
				int available = input.available();
				byte chunk[] = new byte[available];
				input.read(chunk, 0, available);

				// Displayed results are codepage dependent								
				//System.out.print(new String(chunk));

				//writeText(chunk);
				parseText(new String(chunk));
				//System.out.println(line);

			} catch (Exception e) {
				System.err.println(e.toString());
			}
		}
		// Ignore all the other eventTypes, but you should consider the other ones.
	}

	public synchronized void parseText(String chunk)
	{
		String newline = System.getProperty("line.separator");
		line+= chunk;

		if (line.contains("\\n") || line.contains("\\r") || line.contains(newline))
		{ 
			line = removeReturns(line);
			if (line.contains("Unable to find address for Device"))
			{
				System.out.println(" Connection failed retry"); // sometimes the arduino fails to recognize the sensor ID's  
			}			
			Logger.parseText(line);			
			line =""; // line is dealt with so clear it
		}

	}
	
	// removes the multiple return characters from the incoming line.
	private String removeReturns(String inLine){
		String newline = System.getProperty("line.separator");

		inLine = inLine.replaceAll(newline,"");
		inLine = inLine.replaceAll("\\n", "");  // replacing the newline characters with blank. 
		inLine = inLine.replaceAll("\\r", "");  // It is getting both \n and \r so both need handled.	

		return(inLine);

	}

	// for writing the raw byte array to file
	public synchronized void writeText(byte[] inChunk){
		String strFilePath = "/home/strider/Logs/tempData.csv";

		try
		{
			FileOutputStream fos = new FileOutputStream(strFilePath,true);  //true to append

			/*
			 * To write byte array to a file, use
			 * void write(byte[] bArray) method of Java FileOutputStream class.
			 *
			 * This method writes given byte array to a file.
			 */

			fos.write(inChunk);

			/*
			 * Close FileOutputStream using,
			 * void close() method of Java FileOutputStream class.
			 * 
			 */ 

			fos.close(); 

		}
		catch(FileNotFoundException ex)
		{
			System.out.println("FileNotFoundException : " + ex);
		}
		catch(IOException ioe)
		{
			System.out.println("IOException : " + ioe);
		}
	}	
}
Sensor.java - I think it's the same as before, basically everything to do with the sensor objects.
Code:
package datalogger;

public class Sensor {
	
	private String sensorID;
    private String sensorName;
    private float value;
    private int averageCount;
    
    Sensor(String inID,String inName)
    {
     sensorID = inID;
     sensorName = inName;
     value =0;
     averageCount = 0;
    }
    
    public String getID(){
    	return sensorID;
    }
    public String getID(String inName){
    	return sensorID;
    }
    
    public String getName(){
    	return sensorName;
    }
    
    public String getName(String inID){
    	return sensorName;
    }
        
    public void setName(String inName)
    {
    	sensorName = inName;
    }
    
    public void setID(String inID){
    	sensorID = inID;
    }
    public void resetValue()
    {
    	value =0;
    	averageCount =0;
    }
    
    public void addValueToAverage(float inValue)
    {
     value += inValue;
     averageCount++;
    }
    
    public String returnAverage()
    {
      float average = value/averageCount;
      return (String.valueOf(average));
    }           
}
If anyone wants/needs anything or has any questions just let me know.
strider3700 is offline  
Old 12-13-10, 01:20 PM   #20
strider3700
Master EcoRenovator
 
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
Default

And this mornings graph verifying that everything is working correctly.


It appears that taking the average of the every 2 second readings every minute has partially flattened the spikes from the circulating fan kicking on.

Attached Thumbnails
Click image for larger version

Name:	Screenshot-1.jpg
Views:	827
Size:	9.5 KB
ID:	1147  
strider3700 is offline  
Closed Thread


Tags
data, logging

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 02:10 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.
Ad Management by RedTyger
Inactive Reminders By Icora Web Design