×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Reading big .csv files

Reading big .csv files

Reading big .csv files

(OP)
I'm trying (in Mathcad 11) to read a file in .csv format. File consists of 6100 3-character words, separated by commas. There are no carriage returns. (To be precise, it's an array of 3-digit hex numbers separated by commas).

The File-read component fails - error messsge suggests "check file is not in use by another programme". Works ok if file length is reduced to 4000 3-charcter words, or 6100 2-character words) so i suspect it's a file length problem.

Any ideas?
Replies continue below

Recommended for you

RE: Reading big .csv files

Split the file into two halves?

Cheers

Greg Locock

RE: Reading big .csv files

There may be something corrupted in your installation.  The file read component can easily handle 64K elements.

You might try to use the repair option in the install disk or reinstall.

TTFN

RE: Reading big .csv files

(OP)
Thanks.
Not sure what tool I'd use to split the file, though I suppose I could do a manual split using a text editor. Might be a bit error-prone doing it manually.

Yes, I agree that the file read component can read big files. I can happily read a file with 300,000 numerical entries. Problem seems to be associated with the file being alphanumeric - ie I'm tring to get a vector of strings - or maybe being comma separated.

I get idential results with Mathcad 11 at work and Mathcad 200 at home, so problem seems to be with me, not with the installation.

Any thoughts?

RE: Reading big .csv files

My posting was based on reading a CSV file with 64K 3 character elements separated by commas.

TTFN

RE: Reading big .csv files

(OP)
Right, thanks ... could you post the file (or the relevant snippet of it) so's I can try it out? Mathcad 2000 would be good.

RE: Reading big .csv files

There's not much to it and it doesn't paste well:

ARR:---file input from i:\test.csv----
length(ARR)=65536
ARR[0="abc"
ARR[65535="abc"

I didn't try to get anything more complicated than that
 

TTFN

RE: Reading big .csv files

(OP)
Thanks.
Right, I've had another look at this. The problem is only there if the file is a single line, example:
cat,cat,cat,....
If it's:
cat
cat
cat

there's no problem.
Problem file can be created by
 n:=0 .. 6000/Vn:="cat"/T:=transpose of V/save V, an then try to read it.
The failure occurs at 20,000 characters, including the commas. There's a work-around by using READBIN.

RE: Reading big .csv files

The issue as indicated by answers in the Mathcad collab to the same question is that that Excel, itself, cannot read more than 256 columns.

TTFN

RE: Reading big .csv files

(OP)
... err .... could you elucidate the bit about Excel? I puzzled as to why an Excel limitation might affect Mathcad.

I did search (but obviusly not well enough) the Collabatory. Could you point me to the relevant bit?

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members! Already a Member? Login



News


Close Box

Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close