×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

reading a large number of files

reading a large number of files

reading a large number of files

(OP)
I have to read and extract data from a large number of files:

file1.dat
file2.dat
file3.dat
.........
file100.dat

Is there a fast way of reading all these files one at a time?

My simple idea is to write all the filenames in a text file from which i will read a filename at a time. If anyone knows a faster solution im glad to hear it.

Thanx

RE: reading a large number of files

Speed of reading files is dependent on the IO method, usually, not how you get your filenames.  Nonetheless, since your filenames are sequential, you could just store the common root and append the number for each new file.

As for reading the file itself, the preference would be binary format, since that goes much faster than text.

TTFN

FAQ731-376: Eng-Tips.com Forum Policies

RE: reading a large number of files

(OP)
What if my filenames have not a common root?

If i have:

GR01.dat
GR07.dat
FR01.dat
FR02.dat
IT01.dat
IT06.dat
........

Is there a way fortan could read one by one all these...
if i dont want to write all the filenames in a .txt file?

Thanx!

RE: reading a large number of files

Presumably your data files has some unique defining characteristic, such as all having a .dat filename extension and being the only .dat files in the directory.

If so, have your program initiate a directory operation to list the files into some new file (of a different extension).  Once this file has been created, your program can open it, then read the .dat file names one by one and open them for you to extract the stuff you want.

In good ol' DOS, the directory operation would have been
DIR *.DAT /B > FILES.LST
 

RE: reading a large number of files

(OP)
What about in linux?

RE: reading a large number of files

ls -1 > files.dat

(One, not l or L)

In most dialects.

- Steve

RE: reading a large number of files

to get just the dat files in linux

ls -1 *.dat > files.txt

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members! Already a Member? Login


Resources

White Paper - How ESI is Helping Move New Medical Device Product to Market Quicker & More Cost Effic
Early Supplier Involvement has long been a strategy employed by manufacturers to produce innovative products. Now, it almost seems like a necessity. Because decisions made in the design phase can positively affect product quality and costs, this can help add value to OEM bottom lines. This white paper will discuss many facets of ESI, including why it’s so valuable today, what challenges limit the benefits of ESI, how cost is impacted, and more. Download Now
White Paper - Moving to a Driverless Future
This white paper describes what we see as the best practices to support a sustainable engineering process for autonomous vehicle design. It exposes how to use simulation and testing in common frameworks to enable design exploration, verification and validation for the development of autonomous cars at a system, software and full-vehicle level to drive a mature product development process for automated driving. Download Now
Research Report - How Engineers are Using Remote Access
Remote access enables engineers to work from anywhere provided they have an internet connection. We surveyed our audience of engineers, designers and product managers to learn how they use remote access within their organizations. We wanted to know which industries have adopted remote access, which software they are using, and what features matter most. Download Now

Close Box

Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close