×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

reading a large number of files

reading a large number of files

reading a large number of files

(OP)
I have to read and extract data from a large number of files:

file1.dat
file2.dat
file3.dat
.........
file100.dat

Is there a fast way of reading all these files one at a time?

My simple idea is to write all the filenames in a text file from which i will read a filename at a time. If anyone knows a faster solution im glad to hear it.

Thanx

RE: reading a large number of files

Speed of reading files is dependent on the IO method, usually, not how you get your filenames.  Nonetheless, since your filenames are sequential, you could just store the common root and append the number for each new file.

As for reading the file itself, the preference would be binary format, since that goes much faster than text.

TTFN

FAQ731-376: Eng-Tips.com Forum Policies

RE: reading a large number of files

(OP)
What if my filenames have not a common root?

If i have:

GR01.dat
GR07.dat
FR01.dat
FR02.dat
IT01.dat
IT06.dat
........

Is there a way fortan could read one by one all these...
if i dont want to write all the filenames in a .txt file?

Thanx!

RE: reading a large number of files

Presumably your data files has some unique defining characteristic, such as all having a .dat filename extension and being the only .dat files in the directory.

If so, have your program initiate a directory operation to list the files into some new file (of a different extension).  Once this file has been created, your program can open it, then read the .dat file names one by one and open them for you to extract the stuff you want.

In good ol' DOS, the directory operation would have been
DIR *.DAT /B > FILES.LST
 

RE: reading a large number of files

(OP)
What about in linux?

RE: reading a large number of files

ls -1 > files.dat

(One, not l or L)

In most dialects.

- Steve

RE: reading a large number of files

to get just the dat files in linux

ls -1 *.dat > files.txt

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members! Already a Member? Login


Resources

eBook - Functional Prototyping Using Metal 3D Printing
Functional prototypes are a key step in product development – they give engineers a chance to test new ideas and designs while also revealing how the product will stand up to real-world use. And when it comes to functional prototypes, 3D printing is rewriting the rules of what’s possible. Download Now

Close Box

Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close