# is this a pentium, or a bug

Mark R. Bowyer (mark@ukhayq.cv.com)
Thu, 7 Mar 1996 08:34:45 PST

- Messages sorted by:[ date ][ thread ][ subject ][ author ]

    > ;10.00/4.0
    > => 2.50000000052154
    > [used 4 ticks, 0 seconds.]
    > ;1.0/2.0
    > => 0.500000000521541
    > [used 4 ticks, 0 seconds.]

    This looks like our old friend the floating point binary rounding error.  Try
    to write 1/2 in binary by hand some time and you'll understand the problem...

    On my SPARCstation, it goes:

    ;10.0/4.0
    => 2.5
    ;1.0/2.0
    => 0.5

    so I have to assume it's a problem with the maths library or hardware on your
    system not correcting for this.

    Moredhel/TC.