Analysis of Time Varying Data

[prost]

Suppose I have two measurements that vary in time, for instance, Temperature and Dew Point. I want to see if there is some kind of connection between them. I could just plot one vs. the other; result is a big amorphous blob. So I start computing some statistics. If I use CORREL in Excel (it's easy to do, that's why this isn't in the Excel forum!), my correlation is actually slightly negative, which tells me only a little (as one goes up or down, the other is likely to go the opposite direction as the first).

I vaguely remembered autocorrelation from my turbulence studies. That turns out to be a way to estimate if some variable repeats, and the period (or wavelength) of the repeating. Not really useful.

ANOVA and HSD seem to work well only if the measurements aren't sequentially related--that is, one measurement in say column A, row 823 of a spreadsheet is taken at precisely the same time as the second measurement in column B, row 823. Those two stats are out then I think.

Any other ideas?

 

[rb1957]

? you're taking two measurements that you thin are related. plot them (as an X-Y plot). the more linear the plot, the stronger the relationship between them; the more "blobby" the plot, the less they relate to one another (they're weakly related to each other).

not sure why you think ANOVA doesn't work with sequential data (but then the last time i did an ANOVA analysis was in uni.). if this is the case, why not scatter the results (so that they are not cronologically ordered), but you have to keep the correlation between column A and B (ie, more row 823 to after 231).

 

[BobM3]

In addition to autocorrelation there is cross-correlation that allows you to judge relationships between two separate waveforms. If you can get the data into the frequency domain then you have coherence measurements you can look at.

 

[IRstuff]

If you only see an "amorphous blob" then any correlation will be weak, at best. The fact that the slope of the correlated line is small is again an indication that any correlation is probably accidental.

In the case of dewpoint, you'd at least expect to see a upwardly sloping blob, since you'd expect there to be scatter in the RH relative to temperature, but for any given RH day, the dewpoint should loosely track with the ambient temperature.

TTFN

FAQ731-376

 

[prost]

ANOVA tests whether the differences in the means between several groups of data are significant. I don't think you learn too much when you are looking at data sets whose means are orders of magnitude apart, say temperature and absolute humidity (a measure of the amount of water in a volume of air). I don't know if this a valid test--I took temperature, then compared this temperature to the temperature minus 20 degrees--I get different means (by 20 deg.) but same variance. Do an ANOVA, says the data sets are significantly different, even though they are related.

As I said, the x-y plot is a blob.

I don't think I am looking for coherence in the data sets; I am looking for a correlation--data A does this, data B responds by doing this; that kind of thing--I don't think I am looking for frequency or wavelengths in data set A or data set B....but maybe I ave similar frequency distribution, does that suggest they arm--if two data sets hae related?

 

[prost]

My two variables are not temperature and dew point--they are temperature and some other thing I am measuring (not Rel. humidity)--something to do with corrosion. Anyway, I just used dew point to illustrate using a measured quantity almost everyone understands.

 

[rb1957]

so instead of an X-Y plot, you're plotting both data against time ... can you ralate differences between successive data points ? i think that'll tell you something ... maybe (tho' i doubt it) all the time A increases B decreases; probably sometimes A increases B reduces ('cause something else is getting involved)

 

[btrueblood]

Prost,

If you think the effect is lagging in time, try doing a series of x-y plots where you shift one data column forward or backward to create an x(t)-vs-y(t+delta) plot. The raw data will likely still show blobby behavior, so you may also need to filter by only plotting results from periods when x is decreasing (or increasing).

 

[GregLocock]

I agree with rb1957, if the x vs y plot looks blobby, and there is no reason from your understanding of the problem that there should be a time based relationship, then auto correlation and other time history based techniques are misleading.

If you want to test the time based theory you could break the time histories up into frames of data and calculate the coherence between them, but you will need a large number of datapoints for that, and a signal analysis textbook to tell you how to work the coherence out.



Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[Bribyk]

You could compare the slopes of your two data sets (dx/dt, dy/dt), which would eliminate time as a variable. Since you mentioned corrosion, I assume you are looking to find some kind of rate anyway. You may have to break the data up into groups, as Greg mentioned, because your slope may vary a lot when calculated with only two points.

 

[IRstuff]

But, again, why would you expect a correlation? We can easily have 90ºF/60% and 90ºF/15% days in So Cal.

The only thing that's even remotely correlatible is for a given day, that day's RH will track with temperature over small periods of time.

see:

http://en.wikipedia.org/wiki/Dew_point#Calculating_the_dew_point

TTFN

FAQ731-376

 

[GregLocock]

Prost was using dewpoint as an example, that isn't his actual data.

Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[IRstuff]

I understood that.

But, perhaps he was answering his own question by picking an example relationship that showed that his data had no correlation, rathe than one that does correlate.

TTFN

FAQ731-376

 

[prost]

OK, my response variable Y is corrosion current--I am measuring current on aluminum that is exposed to a wet environment in a box in the lab. One possible choice for my X (independent variable that is) is moisture content in the air in the exposure box in the lab. I am looking for a correlation that tells me when X, my Y measurement responds thusly. When I say there is a time correlation, I mean when we measure X, right away we measure Y, so each has the same date/time stamp as the other.

Thus far, if I compute the correlation coefficient with Excel, my corrosion current seems to correlate as well with moisture as temperature and relative humidity correspond--not at all, that is. I'll keep looking, though I suspect the few choices of X that I have (temperature, moisture, etc.) won't do--perhaps instead of Y being a function of just temperature, Y is more likely a function of BOTH temperature and water content of the air.

 

[Bribyk]

Try correcting your relative humidity (which depends on temperature) to absolute humidity. Then your variables are independent.

 

[rb1957]

my 2c ... maybe moisture content is more directly related to the rate of change of corrosion (i imagine that corrosion current is proportional to the amount of corrosion)

 

[prost]

I like that..absolute humidity against derivative of the corrosion signal in time. Let me try that, see if something more interesting evolves. tks

I tried to do a two parameter curve fit on the data, using temp. and absolute humidity. what a disaster! I get a curve fit that has a low (0.2) R-squared (correlation coefficient, that is).

 

[IRstuff]

The standard corrosion test in 810 applies both humidity AND temperature. Humidity corrosion by itself is rather slow, but increasing the temperature accelerates the corrosion.

http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel1/24/1894/00052630.pdf?arnumber=52630

http://ieeexplore.ieee.org/Xplore/l...62554&arSt=1098&ared=1104&arAuthor=Hoge,+C.E.

http://www.chamotlabs.com/ApplicationNotes/HTC/HTC34thMidwest.html

TTFN

FAQ731-376