time series analysis - finding the linear trend in a random walk
time series analysis - finding the linear trend in a random walk
(OP)
This is a bit of a roundabout question.
So, if you have a short series of samples, say 135, could you determine whether there is a linear signal buried in the noise? Imagine that the series is a random walk ie value(n)= value(n-1)+ a signed random variable + X. The signed random variable is of the order of +/- 0.5 say +/- 3 sigma limits. X might be .01
Apparently the answer is no, but I wondered if a technique that measured how many attempts /on average/ it took to find a random number sequence that sufficiently matched the 135 long record for a given assumed value of X was (a) known or (b) a plausible way of investigating the best value of X? Or even graph the average number of tries to meet an RMS error value vs X?
You may know the background to this question or you may not. I'll reveal all later
Here's a plot, the green curve is real world data, the other two are random walks. One of the random walks is just random, the other has a gradient superimposed on it. I am looking for a way of reliably assessing whether there is a linear trend underneath the random walk.

Cheers
Greg Locock
New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?
So, if you have a short series of samples, say 135, could you determine whether there is a linear signal buried in the noise? Imagine that the series is a random walk ie value(n)= value(n-1)+ a signed random variable + X. The signed random variable is of the order of +/- 0.5 say +/- 3 sigma limits. X might be .01
Apparently the answer is no, but I wondered if a technique that measured how many attempts /on average/ it took to find a random number sequence that sufficiently matched the 135 long record for a given assumed value of X was (a) known or (b) a plausible way of investigating the best value of X? Or even graph the average number of tries to meet an RMS error value vs X?
You may know the background to this question or you may not. I'll reveal all later
Here's a plot, the green curve is real world data, the other two are random walks. One of the random walks is just random, the other has a gradient superimposed on it. I am looking for a way of reliably assessing whether there is a linear trend underneath the random walk.

Cheers
Greg Locock
New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?





RE: time series analysis - finding the linear trend in a random walk
TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm
RE: time series analysis - finding the linear trend in a random walk
Then test how long it takes for each series to converge to a given goodness of fit to the original data. repeat several times, this gives a score for each X.
Is speed of convergence to a good fit a statistical technique with a proper name ? It sounds like something a machine learning or optimiser would use, and we use it as an indicator of the quality of our MBD models - good clean models run fast, horrible buggy cludged models like mine take forever.
Cheers
Greg Locock
New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?
RE: time series analysis - finding the linear trend in a random walk
It seems to me that it would be easier to do this the other way around, i.e., take your trend and determine its fit to a straight line and decide whether the calculated sigma is plausible or consistent with your process. The bottom line is that if you see something that looks like it has a linear trend, you can find a line that "fits," only it might be rather a cruddy fit; the measure of fit would still default to the regression coefficient and the calculated deviation from a perfect line.
TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm
RE: time series analysis - finding the linear trend in a random walk
Cheers
Greg Locock
New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?
RE: time series analysis - finding the linear trend in a random walk
TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm
RE: time series analysis - finding the linear trend in a random walk
Cheers
Greg Locock
New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?
RE: time series analysis - finding the linear trend in a random walk
TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers Entire Forum list http://www.eng-tips.com/forumlist.cfm