Precision problem
Precision problem
(OP)
Here's a fun one. I have a date routine that among other things it grabs the last two digits of the year from the GETDAT subroutine. Funny thing is when 2005 rolled around it was saying the year is 04!
I fixed the problem, but I was wondering if any of you experts know what going on in the bowels of the computer that would cause this. Here's the code snippet:
SUBROUTINE GET_DATE_TIME(DATE, TIME)
*
* Example of return values:
* DATE = '03/13/01'
* TIME = ' 9:18 am'
*
USE DFLIB
CHARACTER(8) DATE, TIME
CHARACTER(3) AMorPM
INTEGER(2) IYR, IMON, IDAY, IHR, IMIN, ISEC, IHUNDRETHS
*
* GET DATE
*
CALL GETDAT(IYR,IMON,IDAY)
TMP=IYR / 100.0
IYR = (TMP - INT(TMP)) * 100
ENCODE(8,'(I2.2,1H/,I2.2,1H/,I2.2)',DATE)IMON,IDAY,IYR
.
.
.
To fix it I added this to the declarations:
REAL(8) TMP
and I changed
TMP=IYR / 100.0 to TMP = dfloti(IYR) / 100.0
The clue that tipped me off was that I assigned
TMP - INT(TMP) to a variable and looked at it in the debugger. The value was 0.049999 something, something. In our perfect base 10 world it should have been 20.05 - 20 = 0.05.
Yes, I know it's a precision problem, but I want to understand it. Any takers?
I fixed the problem, but I was wondering if any of you experts know what going on in the bowels of the computer that would cause this. Here's the code snippet:
SUBROUTINE GET_DATE_TIME(DATE, TIME)
*
* Example of return values:
* DATE = '03/13/01'
* TIME = ' 9:18 am'
*
USE DFLIB
CHARACTER(8) DATE, TIME
CHARACTER(3) AMorPM
INTEGER(2) IYR, IMON, IDAY, IHR, IMIN, ISEC, IHUNDRETHS
*
* GET DATE
*
CALL GETDAT(IYR,IMON,IDAY)
TMP=IYR / 100.0
IYR = (TMP - INT(TMP)) * 100
ENCODE(8,'(I2.2,1H/,I2.2,1H/,I2.2)',DATE)IMON,IDAY,IYR
.
.
.
To fix it I added this to the declarations:
REAL(8) TMP
and I changed
TMP=IYR / 100.0 to TMP = dfloti(IYR) / 100.0
The clue that tipped me off was that I assigned
TMP - INT(TMP) to a variable and looked at it in the debugger. The value was 0.049999 something, something. In our perfect base 10 world it should have been 20.05 - 20 = 0.05.
Yes, I know it's a precision problem, but I want to understand it. Any takers?
RE: Precision problem
Had you left it integer and done something like MODULO(year, 100), you would have gotten the result without any error.
TTFN
RE: Precision problem
RE: Precision problem
I still don't understand all of the inner workings that caused this in the first place. Yeah, yeah, I understand the principle, I'm just curious about exactly what's happening.
Sorry it took so long to get back. Thanks again.
RE: Precision problem
Division by 100, which is not an exact binary number MUST result in an approximation in binary floating point. It would not matter what precision you used, it would still be inexact.
TTFN