×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

Analysis of Time Varying Data

Analysis of Time Varying Data

Analysis of Time Varying Data

(OP)
Suppose I have two measurements that vary in time, for instance, Temperature and Dew Point. I want to see if there is some kind of connection between them. I could just plot one vs. the other; result is a big amorphous blob. So I start computing some statistics. If I use CORREL in Excel (it's easy to do, that's why this isn't in the Excel forum!), my correlation is actually slightly negative, which tells me only a little (as one goes up or down, the other is likely to go the opposite direction as the first).

I vaguely remembered autocorrelation from my turbulence studies. That turns out to be a way to estimate if some variable repeats, and the period (or wavelength) of the repeating. Not really useful.

ANOVA and HSD seem to work well only if the measurements aren't sequentially related--that is, one measurement in say column A, row 823 of a spreadsheet is taken at precisely the same time as the second measurement in column B, row 823. Those two stats are out then I think.

Any other ideas?

RE: Analysis of Time Varying Data

? you're taking two measurements that you thin are related.  plot them (as an X-Y plot).  the more linear the plot, the stronger the relationship between them; the more "blobby" the plot, the less they relate to one another (they're weakly related to each other).

not sure why you think ANOVA doesn't work with sequential data (but then the last time i did an ANOVA analysis was in uni.).  if this is the case, why not scatter the results (so that they are not cronologically ordered), but you have to keep the correlation between column A and B (ie, more row 823 to after 231).

RE: Analysis of Time Varying Data

In addition to autocorrelation there is cross-correlation that allows you to judge relationships between two separate waveforms.  If you can get the data into the frequency domain then you have coherence measurements you can look at.

RE: Analysis of Time Varying Data

If you only see an "amorphous blob" then any correlation will be weak, at best.  The fact that the slope of the correlated line is small is again an indication that any correlation is probably accidental.

In the case of dewpoint, you'd at least expect to see a upwardly sloping blob, since you'd expect there to be scatter in the RH relative to temperature, but for any given RH day, the dewpoint should loosely track with the ambient temperature.

TTFN

FAQ731-376: Eng-Tips.com Forum Policies

RE: Analysis of Time Varying Data

(OP)
ANOVA tests whether the differences in the means between several groups of data are significant. I don't think you learn too much when you are looking at data sets whose means are orders of magnitude apart, say temperature and absolute humidity (a measure of the amount of water in a volume of air). I don't know if this a valid test--I took temperature, then compared this temperature to the temperature minus 20 degrees--I get different means (by 20 deg.) but same variance. Do an ANOVA, says the data sets are significantly different, even though they are related.

As I said, the x-y plot is a blob.

I don't think I am looking for coherence in the data sets; I am looking for a correlation--data A does this, data B responds by doing this; that kind of thing--I don't think I am looking for frequency or wavelengths in data set A or data set B....but maybe I ave similar frequency distribution, does that suggest they arm--if two data sets hae related?

RE: Analysis of Time Varying Data

(OP)
My two variables are not temperature and dew point--they are temperature and some other thing I am measuring (not Rel. humidity)--something to do with corrosion. Anyway, I just used dew point to illustrate using a measured quantity almost everyone understands.

RE: Analysis of Time Varying Data

so instead of an X-Y plot, you're plotting both data against time ... can you ralate differences between successive data points ?  i think that'll tell you something ... maybe (tho' i doubt it) all the time A increases B decreases; probably sometimes A increases B reduces ('cause something else is getting involved)  

RE: Analysis of Time Varying Data

Prost,

If you think the effect is lagging in time, try doing a series of x-y plots where you shift one data column forward or backward to create an x(t)-vs-y(t+delta) plot.  The raw data will likely still show blobby behavior, so you may also need to filter by only plotting results from periods when x is decreasing (or increasing).

RE: Analysis of Time Varying Data

I agree with rb1957, if the x vs y plot looks blobby, and there is no reason from your understanding of the problem that there should be a time based relationship, then auto correlation and other time history based techniques are misleading.

If you want to test the time based theory you could break the time histories up into frames of data and calculate the coherence between them, but you will need a large number of datapoints for that, and a signal analysis textbook to tell you how to work the coherence out.

 

Cheers

Greg Locock

SIG:Please see FAQ731-376: Eng-Tips.com Forum Policies for tips on how to make the best use of Eng-Tips.

RE: Analysis of Time Varying Data

You could compare the slopes of your two data sets (dx/dt, dy/dt), which would eliminate time as a variable.  Since you mentioned corrosion, I assume you are looking to find some kind of rate anyway.  You may have to break the data up into groups, as Greg mentioned, because your slope may vary a lot when calculated with only two points.

RE: Analysis of Time Varying Data

Prost was using dewpoint as an example, that isn't his actual data.

Cheers

Greg Locock

SIG:Please see FAQ731-376: Eng-Tips.com Forum Policies for tips on how to make the best use of Eng-Tips.

RE: Analysis of Time Varying Data

I understood that.  

But, perhaps he was answering his own question by picking an example relationship that showed that his data had no correlation, rathe than one that does correlate.

TTFN

FAQ731-376: Eng-Tips.com Forum Policies

RE: Analysis of Time Varying Data

(OP)
OK, my response variable Y is corrosion current--I am measuring current on aluminum that is exposed to a wet environment in a box in the lab. One possible choice for my X (independent variable that is) is moisture content in the air in the exposure box in the lab. I am looking for a correlation that tells me when X, my Y measurement responds thusly. When I say there is a time correlation, I mean when we measure X, right away we measure Y, so each has the same date/time stamp as the other.

Thus far, if I compute the correlation coefficient with Excel, my corrosion current seems to correlate as well with moisture as temperature and relative humidity correspond--not at all, that is. I'll keep looking, though I suspect the few choices of X that I have (temperature, moisture, etc.) won't do--perhaps instead of Y being a function of just temperature, Y is more likely a function of BOTH temperature and water content of the air.

RE: Analysis of Time Varying Data

Try correcting your relative humidity (which depends on temperature) to absolute humidity.  Then your variables are independent.

RE: Analysis of Time Varying Data

my 2c ... maybe moisture content is more directly related to the rate of change of corrosion (i imagine that corrosion current is proportional to the amount of corrosion)  

RE: Analysis of Time Varying Data

(OP)
I like that..absolute humidity against derivative of the corrosion signal in time. Let me try that, see if something more interesting evolves. tks

I tried to do a two parameter curve fit on the data, using temp. and absolute humidity. what a disaster! I get a curve fit that has a low (0.2) R-squared (correlation coefficient, that is).
 

RE: Analysis of Time Varying Data

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members!


Resources