Open and read a bin file
2

Question

Open and read a bin file

BigInch (Petroleum)

(OP)

17 Nov 06 05:44

Sorry to bother you guys, but I know nothing about bin files.  I've been trying for 3 days to extract elevation data from a SRTM3.hgt file, but can't make any sense of the values I'm getting.

The only description of the data file format I can find is,

"The is provided as two byte 16-bit signed integer data in a simple binary raster. There are no header or trailer bytes embedded in the file. The data is stored in row major order with 1201 rows, of 1201 values.  Byte order is big-endian standard with the most significant byte first. Since they are signed integers elevations can range from -32767 to 32767 meters, encompassing the range of elevation to be found on the Earth."

I have a typical file located here (700kb),

http://rapidshare.com/files/3698999/N36W005.hgt.zip.html

I would really like to know how to extract the elevations from that file using VB6.  Greatly appreciate any how-to advice.  A canned sub I could drop in would be worth its weight in gold.  Thanks.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

johnwm (Computer) · Answer 1 · 2006-11-18 03:29:49

Not quite sure what you are expecting, but here is something! You can Rem out the Debug.Print statements if the data you see is what you expect. Declare a Public array for your result in the declarations section.

CODE

Public myArray(1200, 1200) As Long

Call the sub as

CODE

UnpackElevations("c:\N36W005.hgt")

Here is the Sub which currently dispays results in the Immediate window and stores the result in the array

CODE

Public Sub UnpackElevations(mFile As String)
Dim myRow As Integer
Dim myCol As Integer
Dim myFileNum As Integer
Dim a As Byte
Dim b As Byte
Dim c As Long

myFileNum = FreeFile
Open mFile For Binary As myFileNum
    For myRow = 0 To 1200
        For myCol = 0 To 2401 Step 2
            Get #myFileNum, , a
            Get #myFileNum, , b
            c = b + 256 * a
            Debug.Print c;
            myArray(myRow, (myCol + 1) \ 2) = c
        Next myCol
        Debug.Print
    Next myRow
Close #myFileNum
End Sub

Good Luck
johnwm
________________________________________________________
To get the best from these forums read FAQ731-376 before posting

Steam Engine enthusiasts: www.essexsteam.co.uk

BigInch (Petroleum) · Answer 2 · 2006-11-18 07:38:19

Wow.  Its nice to finally make some progress.  Thanks so much.  The file is now opening and reads values that apparently fit the elevation model.  (I have to check the file's values with some actual elevation data, but I wanted to say thank you before I did that.)

Your code is very clear, but what I can't understand is what the statement c = b + 256 * a  is actually doing.  I suppose it is reversing the byte order and adding them together.  This may be important, because one little (I hope) problem is that it hangs with an overflow error (row 38), when it gets b = 0, and a = 128.  I will study up on adding bits & bytes.  Maybe I just have to add a branch for that condition.

Thanks again.  (how much does it weigh so far?

)

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

BigInch (Petroleum) · Answer 3 · 2006-11-18 07:42:28

P.S. Can't wait until I have time to have a look at your steam engine site.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

BigInch (Petroleum) · Answer 4 · 2006-11-18 09:03:31

I see that b + c * 256 is converting 2 hexidecimal digits to a decimal integer.

OK, great, now I'll try to see why the hangup is happening.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

johnwm (Computer) · Answer 5 · 2006-11-18 11:02:37

Quote:
Byte order is big-endian standard with the most significant byte first.

We are reading the bytes one by one, so the first byte * 256 plus second byte gives the 16-bit integer value. My first try overlooked the SIGNED part, so if the first byte is over 127 (ie the first bit is set) then the value is negative, read as it's 2's complement value. This means that the 'odd' results we see (that appear as overflow errors with a byte pair value of 128, 0) represent a hole in the ground 32768 metres deep!

I suspect that this 'special' value may possibly be data corruption, or more likely a deliberate flag to indicate some feature (maybe unknown/unreliable measurement?)

In the whole datafile there are quite a few (maybe around 30) of these (128,0) pairs. Other than that, the only negative numbers are very small (resolving to -1 or -2 metres) and adjacent to a load of values that resolve to 0.

To get the SIGNED values from my first post add the declaration

CODE

Dim a1 As Integer

and change

CODE

Get #myFileNum, , b
c = b + 256 * a
Debug.Print c;

to

CODE

Get #myFileNum, , b
            If a > 127 Then
            a1 = a - 256
            c = CInt(b) + CLng(256 * a1)
        Else
            c = CInt(b) + CLng(256 * a)
            End If
            Debug.Print c;

This detects the signed negatives (b>127 or first bit set) and does the appropriate conversion.

I would be interested to find out what the 128, 0 represents

And I hope you enjoy the website!

Good Luck
johnwm
________________________________________________________
To get the best from these forums read FAQ731-376 before posting

Steam Engine enthusiasts: www.essexsteam.co.uk

BigInch (Petroleum) · Answer 6 · 2006-11-18 15:17:05

Thank you so much.  I just now worked out that it was a signed integer problem.  This is great.  As you surmized, the 128 is a -32768 meter hole, but fortunately not in the earth, only in the data.  I'll have to smooth it out using  averaging of surrounding points.

The data file is a typical file from NASA's "Shuttle Radar Topography Mission", (SRTM), for which the elevations have been resolved into 3 second quadrants (appx. 90 m at the equator) between N60 and S56 degrees latitude.  I'm using N36W005.hgt, because it is close to where I live.  The title of the file indicates the lower left "pixel" centerline of coverage is at  N 36:00:00 Lat and W 5:00:00 Longitude, hgt = height).  I am trying to develop some pipeline routing and terrain visualization software by overlaying Google Earth images onto the elevation matrix.  Then I'll be able to fly the route without using a helicopter.  I already have an OGL terrain visualization program that I've been testing, but using data with only 30 second resolution, so its pretty fake looking terrain.  I was able to find ASCII files with that data, but the 03 sec ASCII files that exist now are 250 MB and a bit unwieldly.  The bins are, as you already know only about 3 MB, so much easier to use.  I also have a GPS hookup into a 2D viewer that I will integrate into the 3D system, along with some modules to download the satellite photos automatically and stitch them together, etc.  All should be ready soon.

I really appreciate your help, so if you're interested, I will send you a source code beta as soon as its cleaned up a bit.  My e-mail is in the contact section of my virtualpipeline MSN space webpage.  Let me know where I can reach you.

Now that time isn't the problem.... away from the bits and on to steam.  The neatest gift I ever got for Xmas was one of those little German steam engines.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

BigInch (Petroleum) · Answer 7 · 2006-11-18 16:42:49

This one is 50 km outside Medinah, Saudi Arabia. Its laid up at a water stop on the old Ottoman Railroad from Damascus to Medinah.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

BigInch (Petroleum) · Answer 8 · 2006-11-29 18:15:31

Here's a sample of what the HGT file contains; a partial print of the total file, after I converted the data into colored elevations. This tile shows the elevations of the Spanish Costa del Sol west of Malaga. Malaga is at the bay towards the right center. Marbella is located at the bay to the lower left. I live at the point on the ridge going southeast towards the sea at the bottom center. Distance between Malaga and Marbella is about 50 km. Dark red colors represent elevations between 2000-2500 m

Each pixel represents 3 seconds of latitude and 3 seconds of longitude. A geographically correct plot really should be compressed 80% in the horizontal direction, since at this latitude, 1 degree of longitude is 80% that of 1 degree of latitude.

Thanks for your help.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

BigInch (Petroleum) · Answer 9 · 2006-12-23 20:17:24

Thanks again.

I have now morphed the program into an OGL terrain viewer. The Elevation version above now has a line of sight module, which can be used for U/VHF antenna or forest fire tower positioning, etc. Work continues on the 3D terrain viewer using freely downloaded satellite photos from www.terraserver.com. Photos must be corrected for latitude & longitude before use, but the program will do it for you. Here's an example of the latest output. This view is from a perspective along the approach path to Gibraltar's airport... from the west.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

johnwm (Computer) · Answer 10 · 2006-12-24 03:56:05

That is a lovely piece of work BigInch - I'm happy I could help in some way. I don't do much with locomotives, but you can follow the ongoing story of restoring my traction engine on the website (which also contains contact details)

Good Luck
johnwm
________________________________________________________
To get the best from these forums read FAQ731-376 before posting

Steam Engine enthusiasts: www.essexsteam.co.uk

BigInch (Petroleum) · Answer 11 · 2006-12-25 13:16:31

Still a work in progress, but getting some useful output now. Must improve the sea presentation. I have a few more views from the west side on my web page.

I thought you were invoved in locomotive work and was quite surprized to see that I guessed wrong. I didn't expect to see the steam tractors. I also didn't think they were as complicated as they are. Quite facinating and a very complex piece of work reconditioning one. Rolling a finished product out of the shop must be a special feeling. It is really an acomplishment to be proud of.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

zeitghost (Computer) · Answer 12 · 2007-01-23 07:51:19

Someone appears to have stolen the rails from under that engine... was it Colonel Lawrence again?

BigInch (Petroleum) · Answer 13 · 2007-01-23 18:41:48

Actually I'm surprized that the engine is still there... and the fence for that matter.

BigInch worm -born in the trenches.
http://virtualpipeline.spaces.msn.com

Come Join Us!

Posting Guidelines

Open and read a bin file
2