×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Are you an
Engineering professional?
Join Eng-Tips Forums!
• Talk With Other Members
• Be Notified Of Responses
• Keyword Search
Favorite Forums
• Automated Signatures
• Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

#### Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

# Precision problem

## Precision problem

(OP)
Here's a fun one. I have a date routine that among other things it grabs the last two digits of the year from the GETDAT subroutine.  Funny thing is when 2005 rolled around it was saying the year is 04!

I fixed the problem, but I was wondering if any of you experts know what going on in the bowels of the computer that would cause this.  Here's the code snippet:

SUBROUTINE GET_DATE_TIME(DATE, TIME)
*
* Example of return values:
*    DATE = '03/13/01'
*    TIME = ' 9:18 am'
*
USE DFLIB
CHARACTER(8) DATE, TIME
CHARACTER(3) AMorPM
INTEGER(2) IYR, IMON, IDAY, IHR, IMIN, ISEC, IHUNDRETHS
*
* GET DATE
*
CALL GETDAT(IYR,IMON,IDAY)
TMP=IYR / 100.0
IYR = (TMP - INT(TMP)) * 100
ENCODE(8,'(I2.2,1H/,I2.2,1H/,I2.2)',DATE)IMON,IDAY,IYR
.
.
.

To fix it I added this to the declarations:

REAL(8) TMP

and I changed

TMP=IYR / 100.0   to  TMP = dfloti(IYR) / 100.0

The clue that tipped me off was that I assigned
TMP - INT(TMP) to a variable and looked at it in the debugger. The value was 0.049999 something, something.  In our perfect base 10 world it should have been 20.05 - 20 = 0.05.

Yes, I know it's a precision problem, but I want to understand it.  Any takers?

### RE: Precision problem

You took a perfectly reasonable and PRECISE integer and turned it into a binary floating point number.

Had you left it integer and done something like MODULO(year, 100), you would have gotten the result without any error.

TTFN

### RE: Precision problem

I agree with IRstuff's approach as the more elegant and correct way to tackle your original problem, but getting back to your question on precision, a floating point real variable is most unlikely to store 0.05 as exactly 0.05 but will store it as say 0.050001 in which case your original routine would have returned the correct year, but as you found out it had the value of 4.99999 (when multiplied by 100) and your routine then truncated it to 4, in circumstances when the MOD function is not practical it is standard practice to add 0.5 to your floating point variable before taking the integer, i.e. INT(4.99999 + 0.5) = 5 and also INT(5.00001 + 0.5 ) = 5 so any value > 4.5 and < 5.5 would return the integer value as 5

### RE: Precision problem

(OP)
Thanks guys.  I used the MOD function and it works like a dream.  I feel kinda goofy for not thinking of it.  I've used it in other applications.

I still don't understand all of the inner workings that caused this in the first place.  Yeah, yeah, I understand the principle, I'm just curious about exactly what's happening.

Sorry it took so long to get back. Thanks again.

### RE: Precision problem

There's nothing mysterious.  An integer year is exactly represented in a binary format.

Division by 100, which is not an exact binary number MUST result in an approximation in binary floating point.  It would not matter what precision you used, it would still be inexact.

TTFN

#### Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

#### Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

#### Resources

Low-Volume Rapid Injection Molding With 3D Printed Molds
Learn methods and guidelines for using stereolithography (SLA) 3D printed molds in the injection molding process to lower costs and lead time. Discover how this hybrid manufacturing process enables on-demand mold fabrication to quickly produce small batches of thermoplastic parts. Download Now
Examine how the principles of DfAM upend many of the long-standing rules around manufacturability - allowing engineers and designers to place a partâ€™s function at the center of their design considerations. Download Now
Taking Control of Engineering Documents
This ebook covers tips for creating and managing workflows, security best practices and protection of intellectual property, Cloud vs. on-premise software solutions, CAD file management, compliance, and more. Download Now

Close Box

# Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

• Talk To Other Members
• Notification Of Responses To Questions
• Favorite Forums One Click Access
• Keyword Search Of All Posts, And More...

Register now while it's still free!