Reducing data set
Reducing data set
(OP)
I have a 4 column by 80,800 row data set. It's too much to graph. It's data recorded every 6 seconds, I could get rid of 90% of it and only keep one record every minute. How can I automate that?
----------------------------------------
The Help for this program was created in Windows Help format, which depends on a feature that isn't included in this version of Windows.





RE: Reducing data set
Note that Excel 2010 has removed that specific limit, but the older versions of Excel are only limited to 32k data points, so there's not that much decimation that you need to do.
TTFN

FAQ731-376: Eng-Tips.com Forum Policies
Need help writing a question or understanding a reply? forum1529: Translation Assistance for Engineers
Of course I can. I can do anything. I can do absolutely anything. I'm an expert!
There is a homework forum hosted by engineering.com: http://www.engineering.com/AskForum/aff/32.aspx
RE: Reducing data set
I tried filtering to only times that fit the pattern ??:??:00. The beginning data has points that meet that criteria but evidently the clock has a little skew so after awhile there are no data points that meet that criteria. I'll try adding a column with just sequential integers and filter on that.
----------------------------------------
The Help for this program was created in Windows Help format, which depends on a feature that isn't included in this version of Windows.
RE: Reducing data set
----------------------------------------
The Help for this program was created in Windows Help format, which depends on a feature that isn't included in this version of Windows.
RE: Reducing data set
----------------------------------------
The Help for this program was created in Windows Help format, which depends on a feature that isn't included in this version of Windows.
RE: Reducing data set
TTFN

FAQ731-376: Eng-Tips.com Forum Policies
Need help writing a question or understanding a reply? forum1529: Translation Assistance for Engineers
Of course I can. I can do anything. I can do absolutely anything. I'm an expert!
There is a homework forum hosted by engineering.com: http://www.engineering.com/AskForum/aff/32.aspx
RE: Reducing data set
=MOD(ROW(),10)
replacing the 10 with whatever sampling severity is to your taste.
(You could even use a variable instead of 10, so you could readily explore the graph's sensitivity to the value.)
RE: Reducing data set
CODE -->
Sub delete_rows() Nsave = 3 ' Number of consequitive rows to save Ndelete = 3 ' Number of consequitive rows to delete Row1 = 6 ' Number of the first row of database Nrows = 35 'Number of rows in database Niter& = Nrows / (Ndelete + Nsave) For i = 1 To Niter Row1 = Row1 + Nsave sRows$ = Row1 & ":" & Row1 + Ndelete - 1 Rows(sRows).Delete Next i End SubYakpol
RE: Reducing data set
There are many ways to do this, and there is nothing wrong with the filtered list or VBA approach, but I would probably rearrange all the data using the Index function.
Say we wanted to plot points at 1 minute intervals, which is convenient because that's every tenth point:
Suppose we want the rearranged table to start in cell B2
In row 1 above the output range enter the values 0 to 9 in starting in column B.
In Cell A2 enter 1
In Cell A3 enter =A2+10 and copy down as far as required.
In Cell B2 enter =Index(datarange, $A2 + B$1). Datarange is either a named range, or the absolute cell address of the range (must be a single column).
Copy across the 10 columns and down as far as required.
If desired, copy the entire table and paste as values, or if the data will change leave it as formulas.
You can now plot any one of the columns, or find the average of each row and plot that (or max, or min, or whatever).
Doug Jenkins
Interactive Design Services
http://newtonexcelbach.wordpress.com/
RE: Reducing data set
----------------------------------------
The Help for this program was created in Windows Help format, which depends on a feature that isn't included in this version of Windows.
RE: Reducing data set
CODE -->
Sub points() Dim freq As Integer, iRow As Integer, iCol As Integer, iRow2 As Integer freq = 10 ' frequency of getting data iRow = 2 ' starting row of data iCol = 1 ' column of data iRow2 = 2 'Starting row of data in new sheet Do Until IsEmpty(Sheet1.Cells(iRow, iCol)) Sheet1.Rows(iRow).Copy Destination:=Sheet2.Cells(iRow2, 1) iRow = iRow + freq iRow2 = iRow2 + 1 Loop End Sub