![]() |
![]() |
#11 | |
Supreme EcoRenovator
Join Date: Mar 2009
Location: Portland, OR
Posts: 4,004
Thanks: 303
Thanked 724 Times in 534 Posts
|
![]() Quote:
![]() I understand the trend lines, but what causes the short-duration periodic swings? -AC_Hacker |
|
![]() |
![]() |
#12 |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() I have the fan in the forced air system set to circulate so it comes on for a few minutes every 15 minutes or so. the room with the sensor is easily the coldest in the house being the basement north west corner so slightly warmer air gets blown in when that happens. if the air is warm enough the room gradually climbs as happened last night. If the air isn't the room gradually drops like happened in the first graph.
I picked up 100 meters of cat 3 today and will put the second sensor in the room with the wood stove if I get a chance tonight. It'll be nice to see multiple temps logged and see how the rooms are different. |
![]() |
![]() |
#13 | |
Supreme EcoRenovator
Join Date: Mar 2009
Location: Portland, OR
Posts: 4,004
Thanks: 303
Thanked 724 Times in 534 Posts
|
![]() Quote:
* * * Say, I'm only marginally able to tweak Arduino code, and what you are doing is right in line with something I have wanted to do for a couple of years... As you may know, I'm doing the Homemade HeatPump Manifesto thread, and I have wanted to build a vertical multi-sensor data logger to monitor temperatures of the earth at various depths in my GSHP loop field. My idea is to get a fifteen foot length of pipe, copper is ok, PVC would work too, and put sensors every two feet or so, and fill the pipe up with tar or plastic or something and bury it, putting a data cable from the pipe and sensors, into a foot deep trench, and run it into the house. Then I would monitor the temps periodically like maybe once an hour or once a day... for several years. I understand that 1-wire sensors are uniquely numbered, and quite accurate, and for design simplicity, I would run them off of three wires, Data, Vcc, and Ground. Screw the timeing issues, wire is cheap. So, do you think you could help me out with code to read several 1-wire therrmal sensors? I'm guessing it's nothing much more than cut & paste & paste & paste & paste, etc. Tell me, am I over-simplifying? Does your code include an initialization section that polls and records the unique identifiers of all sensors? Thanks, -AC_Hacker Last edited by AC_Hacker; 12-11-10 at 12:39 AM.. |
|
![]() |
![]() |
#14 |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() Just FYI the ds18b20 sensors will not work over 100 meters of cat 3 28 gauge 2 pair in either parasitic or standard mode. I'm not shocked and don't need them to anyways but wanted to test. It does work just fine in parasitic for 10 meters though. My longest is about 25 meters but I won't be setting that test up tonight.
|
![]() |
![]() |
#15 | |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() Quote:
Your basic steps are - first use a breadboard and wire up 1 sensor Here's a nice page explaining it. Arduino and DS18B20 - 1-wire digital thermometer | Ogalik on the arduino you'll see VCC and gnd so you plug them in. Pick any of the digital pins you just need to know the number. Then load the arduino IDE and install the dallas temps and 1 wire library - it's pretty trivial and I can point it out if needed when the time comes, basically just put the correct file from a downloaded zip in the right space. Then if you grab my code a couple of posts back paste it into the arduino IDE change the pin number at the top to whatever you used and hit compile it should compile with no errors. click upload and then fire up the serial monitor in the arduino IDE and it should tell you the sensors ID and the temperature. record the ID and label the sensor, repeat till you have ID's for all of them wire two of them up like in that page I posted and restart the serial monitor. The code will work fine for two with no changes as it currently is. Soon I'll make it work for any number (up to 64 is the limit I think) of sensors without any changes and you'll be good to go. That will give you data being dumped into the arduino serial monitor. When the java part of the program is finished I'll post it up and you can dump the data to a csv. Or you can grab the code as it currently stands and track down the rxtx library and build it yourself. As I go whatever I create while working on this project I'll post on here and anyone can take and do whatever with it. Just be forewarned I'm posting where I'm at every couple of days and much of it is far from optimal. I'll hopefully get it pretty by the end but if you grab something now just be aware that it's a very early stage work in progress/proof of concept. I am hoping that the java code will be good enough in the end to be useful for others that want basic data logging via an arduino. Just have the arduino return the sensor ID and a value and the java program will put that to a csv file. |
|
![]() |
The Following User Says Thank You to strider3700 For This Useful Post: | AC_Hacker (12-11-10) |
![]() |
#16 |
Supreme EcoRenovator
Join Date: Mar 2009
Location: Portland, OR
Posts: 4,004
Thanks: 303
Thanked 724 Times in 534 Posts
|
![]() Thanks, most of it is pretty clear.
Tomorrow, I'll dig up my arduino board and give it a try. -Regards, -AC_Hacker |
![]() |
![]() |
#17 |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() Ok I spent some time wiring up a second temp sensor and installing it in my rec room which has the woodstove. It was an easy/quiet wiring job to do in the middle of the night and it should be interesting to see how the temps work out in my office vs in there.
Then I decided to get down to work on the java side to get logging that in a nice easy to use method. I don't want to datalog every second or two's data but I don't want the inaccuracy associated with only collecting data every 1,2,5... whatever minutes. So I figured I'll collect data every second and average it for whatever time period becomes useful in the future. So I decided first of all I need a new java class to create a sensor object for each sensor. it gives me the basics, set the name, id, get the name, id. also it has a Value stored in it. I can reset that, and add to it. each time I add to it I add one to the count and when I ask for the average I just take the value/count and return that. Pretty simple. Code:
package datalogger; public class Sensor { private String sensorID; private String sensorName; private float value; private int averageCount; Sensor(String inID,String inName) { sensorID = inID; sensorName = inName; value =0; averageCount = 0; } public String getID(){ return sensorID; } public String getID(String inName){ return sensorID; } public String getName(){ return sensorName; } public String getName(String inID){ return sensorName; } public void setName(String inName) { sensorName = inName; } public void setID(String inID){ sensorID = inID; } public void resetValue() { value =0; averageCount =0; } public void addValueToAverage(float inValue) { value += inValue; averageCount++; } public String returnAverage() { float average = value/averageCount; return (String.valueOf(average)); } } I was also thinking that long term I should make the program look at a simple configuration file to get the sensor ID's and names to make it easier for non programmers to use. As if I don't have enough to do... |
![]() |
![]() |
#18 |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() Ok I've spent too much time on this lately but it's now doing the following.
The arduino dumps back all of the sensor data every 2 seconds with a 30 millisecond delay between each. The java program collects the data and every 60 seconds prints the average for each sensor to a csv file the clears the average so we're getting a minute by minute recording. The format going into the csv file is Date,Time,S1,S2.... with the first line being a Header which loads the "common name" for the sensor so in my case, office, Rec room. I don't know if there is a guarantee that s1 will always be office and s2 always rec room. The program will always put the values to the correct sensor name however and it won't flip part way through a run. Worst case you run the program and it lists them office, rec room and then you stop the program restart it and it will continue appending but flipped so recroom, office. The Header will be rewritten if you were to do that.... Common names are currently hardcoded. Long term none of that should matter because I want to move away from dumb csv's and into some form of database. anyways right now it's running collecting from the two different rooms and in the morning I can generate a new chart, this one with labels and two lines... That's enough to keep me happy for awhile data wise so when I return to this project it will be to get current readings working. Now the code if anyone is interested. It's still rough... The arduino is the same as last time I posted I believe. Logger.java the new main class Code:
package datalogger; import java.text.SimpleDateFormat; import java.util.ArrayList; import java.util.Date; import java.util.List; import java.util.Timer; public class Logger { private static Date today; private static SimpleDateFormat formatter = new SimpleDateFormat("yyyy/MM/dd,HH:mm"); public static List<Sensor> sensorList=new ArrayList<Sensor>(); private static SerialHandling handler ; private static Timer timer; public static void main(String[] args) { loadSensorList(); timer = new Timer(); setTimer(); handler = new SerialHandling(); handler.initialize(); } private static void setTimer(){ timer.scheduleAtFixedRate( new writeData(), // calls the writeData class run method to write the data to csv 0, // start right away 60000 // run every 60 seconds //5000 // run every 5 seconds for testing ); } public static void parseText(String line) { //System.out.println(line); if (line.contains(",")) { String[] split = line.split(","); String sensor = split[0]; String value = split[1]; averageValues(sensor,value); String name = getSensorName(sensor); today = new Date(); String time = formatter.format(today); //System.out.println(name+","+time+","+value); System.out.println(time+","+name+","+value); //writeText(time+" "+name+" "+ value+newline); //writeText(name+","+time+","+value+newline); } } // fills the List with the Sensors. Currently hardcoded ID's public static void loadSensorList() { Sensor office = new Sensor("282895C50200009B","office"); Sensor recroom = new Sensor("28BF90C50200007D","recroom"); sensorList.add(office); sensorList.add(recroom); } public static int getSensorCount() { return sensorList.size(); } // find the sensor based on the ID private static Sensor getSensor(String inID) { int i =0; boolean done = false; Sensor temp = new Sensor("default","default"); while ( i < sensorList.size() && done == false) { temp = sensorList.get(i); if (inID.equals(temp.getID())) { done = true; } i++; } return temp; } // returns the sensors common Name if in the list else the sensor ID is used as the name private static String getSensorName(String inID){ String name = inID; //default to the ID if not found Sensor temp = getSensor(inID); name = temp.getName(); return name; } private static void averageValues(String id, String value){ Sensor s = getSensor(id); try { float f = Float.valueOf(value.trim()).floatValue(); s.addValueToAverage(f); } catch (NumberFormatException nfe) { System.out.println("NumberFormatException: " + nfe.getMessage()); } // System.out.println(id+" value "+value+" average "+s.returnAverage()); } } Code:
package datalogger; import java.io.BufferedWriter; import java.io.FileNotFoundException; import java.io.FileWriter; import java.io.IOException; import java.io.Writer; import java.text.SimpleDateFormat; import java.util.Date; import java.util.TimerTask; // the Timer task the calls the outputting to the CSV file public class writeData extends TimerTask { private static Date today; private static SimpleDateFormat formatter = new SimpleDateFormat("yyyy/MM/dd,HH:mm"); private boolean first = true; public void run() { //System.out.println("Generating csv"); String row = buildRow(); if (row.contains("NaN")==false) // done to stop that initial hit of empty from being written I could also delay { // the timer by 60 seconds but this always works no matter what the timer timing writeText(row); } } //builds the row to output to CSV file. It hardcodes the header line in the first time called. private String buildRow(){ //row = Date,Time,s1,s2,... String row = ""; // if the first line then dump in the header if (first == true) { row = "Date,Time"; for (int i=0;i< Logger.getSensorCount() ;i++) { row+=","+Logger.sensorList.get(i).getName(); } first = false; writeText(row); row =""; } today = new Date(); String time = formatter.format(today); row += time; // add the time to the row // now add all of the average values in the sensors for (int i=0;i< Logger.getSensorCount() ;i++) { row+=","+Logger.sensorList.get(i).returnAverage(); //get the average Logger.sensorList.get(i).resetValue(); // clear the averages } //System.out.println("writing -"+row); return row; } // for writing the text strings to the file public synchronized void writeText(String inString){ String strFilePath = "/home/strider/Logs/tempData.csv"; String newline = System.getProperty("line.separator"); try { Writer output = new BufferedWriter(new FileWriter(strFilePath,true)); //FileWriter always assumes default encoding is OK! output.write( inString+newline); output.close(); } catch(FileNotFoundException ex) { System.out.println("FileNotFoundException : " + ex); } catch(IOException ioe) { System.out.println("IOException : " + ioe); } } } |
![]() |
![]() |
#19 |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() stupid maximum length rule
![]() SerialHandler.java - handles the data on the serial port and passes it formatted to Logger Code:
package datalogger; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import gnu.io.CommPortIdentifier; import gnu.io.SerialPort; import gnu.io.SerialPortEvent; import gnu.io.SerialPortEventListener; import java.util.Enumeration; public class SerialHandling implements SerialPortEventListener { SerialPort serialPort; /** The port we're normally going to use. */ private static final String PORT_NAMES[] = { "/dev/ttyUSB0", // Linux }; /** Buffered input stream from the port */ private InputStream input; /** The output stream to the port */ private OutputStream output; /** Milliseconds to block while waiting for port open */ private static final int TIME_OUT = 2000; /** Default bits per second for COM port. */ private static final int DATA_RATE = 9600; private String line = new String(); public void initialize() { CommPortIdentifier portId = null; Enumeration portEnum = CommPortIdentifier.getPortIdentifiers(); // iterate through, looking for the port while (portEnum.hasMoreElements()) { CommPortIdentifier currPortId = (CommPortIdentifier) portEnum.nextElement(); for (String portName : PORT_NAMES) { if (currPortId.getName().equals(portName)) { portId = currPortId; break; } } } if (portId == null) { System.out.println("Could not find COM port."); return; } try { // open serial port, and use class name for the appName. serialPort = (SerialPort) portId.open(this.getClass().getName(), TIME_OUT); // set port parameters serialPort.setSerialPortParams(DATA_RATE, SerialPort.DATABITS_8, SerialPort.STOPBITS_1, SerialPort.PARITY_NONE); // open the streams input = serialPort.getInputStream(); output = serialPort.getOutputStream(); // add event listeners serialPort.addEventListener(this); serialPort.notifyOnDataAvailable(true); } catch (Exception e) { System.err.println(e.toString()); } } /** * This should be called when you stop using the port. * This will prevent port locking on platforms like Linux. */ public synchronized void close() { if (serialPort != null) { serialPort.removeEventListener(); serialPort.close(); } } /** * Handle an event on the serial port. Read the data and print it. */ public synchronized void serialEvent(SerialPortEvent oEvent) { if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) { try { int available = input.available(); byte chunk[] = new byte[available]; input.read(chunk, 0, available); // Displayed results are codepage dependent //System.out.print(new String(chunk)); //writeText(chunk); parseText(new String(chunk)); //System.out.println(line); } catch (Exception e) { System.err.println(e.toString()); } } // Ignore all the other eventTypes, but you should consider the other ones. } public synchronized void parseText(String chunk) { String newline = System.getProperty("line.separator"); line+= chunk; if (line.contains("\\n") || line.contains("\\r") || line.contains(newline)) { line = removeReturns(line); if (line.contains("Unable to find address for Device")) { System.out.println(" Connection failed retry"); // sometimes the arduino fails to recognize the sensor ID's } Logger.parseText(line); line =""; // line is dealt with so clear it } } // removes the multiple return characters from the incoming line. private String removeReturns(String inLine){ String newline = System.getProperty("line.separator"); inLine = inLine.replaceAll(newline,""); inLine = inLine.replaceAll("\\n", ""); // replacing the newline characters with blank. inLine = inLine.replaceAll("\\r", ""); // It is getting both \n and \r so both need handled. return(inLine); } // for writing the raw byte array to file public synchronized void writeText(byte[] inChunk){ String strFilePath = "/home/strider/Logs/tempData.csv"; try { FileOutputStream fos = new FileOutputStream(strFilePath,true); //true to append /* * To write byte array to a file, use * void write(byte[] bArray) method of Java FileOutputStream class. * * This method writes given byte array to a file. */ fos.write(inChunk); /* * Close FileOutputStream using, * void close() method of Java FileOutputStream class. * */ fos.close(); } catch(FileNotFoundException ex) { System.out.println("FileNotFoundException : " + ex); } catch(IOException ioe) { System.out.println("IOException : " + ioe); } } } Code:
package datalogger; public class Sensor { private String sensorID; private String sensorName; private float value; private int averageCount; Sensor(String inID,String inName) { sensorID = inID; sensorName = inName; value =0; averageCount = 0; } public String getID(){ return sensorID; } public String getID(String inName){ return sensorID; } public String getName(){ return sensorName; } public String getName(String inID){ return sensorName; } public void setName(String inName) { sensorName = inName; } public void setID(String inID){ sensorID = inID; } public void resetValue() { value =0; averageCount =0; } public void addValueToAverage(float inValue) { value += inValue; averageCount++; } public String returnAverage() { float average = value/averageCount; return (String.valueOf(average)); } } |
![]() |
![]() |
#20 |
Master EcoRenovator
Join Date: Dec 2008
Location: Vancouver Island BC
Posts: 745
Thanks: 23
Thanked 37 Times in 30 Posts
|
![]() And this mornings graph verifying that everything is working correctly.
It appears that taking the average of the every 2 second readings every minute has partially flattened the spikes from the circulating fan kicking on. |
![]() |
![]() |
Tags |
data, logging |
|
|