# floating point numbers / decode_binary()

H. Peter Anvin (hpa@trantor.zytor.com)
Mon, 16 Dec 1996 23:13:41 PST

- Messages sorted by:[ date ][ thread ][ subject ][ author ]

    Followup to:  <3.0.1.32.19961216203209.006a04c4@pop.kaiwan.com>
    By author:    Brack <slayer@kaiwan.com>
    In newsgroup: local.moo
    >
    > I thought floating point numbers only had a precision up to like 15-17
    > digits or so. It seems the largest floating point number I can use is
    > 1.79769313486231e+308. But, if I put a real long float number, such as
    > ;1.79769313486231580793728971405303415079
    > it will round off to 1.79769313486232e+308. But, I can't re-eval
    > 1.79769313486232e+308 because it will give me an overflow. Also if I try
    > 1.797693134862315807937289714053034150791 it gives an overflow tb as well.
    >

    I presume that is 1.79....e+308 you have there?  It is indeed possible
    for the normal output rounding to end up with a number that cannot be
    read back in due to overflow; this can be circumvented by using
    floatfmt() [or whatever the function is called again] using a very
    large precision (such as 9999 - the MOO server will force it down to a
    sane number.)  Unfortunately these types of effects are part of the
    nature of floating point.

    >
    > I also posted on Lambda while this list was down, I noticed if you try
    > something like decode_binary("~80~03~90") I correctly get {128, 3, 144} but
    > if I try decode_binary("~80~03~90", 1) it comes out as {-128, 3, -112}; I
    > was told it was due to using signed integer types rather than unsigned in
    > the server code.
    >

    This is a sheer bug; this should be avoided by casting to
    (unsigned char) before casting to (int):

    	*n = (int) (unsigned char) (*ch);

    [or whatever...]

    	-hpa
    --
    PGP public key available - finger hpa@zytor.com
    Key fingerprint: 2047/2A960705 BA 03 D3 2C 14 A8 A8 BD  1E DF FE 69 EE 35 BD 74
    "The earth is but one country, and mankind its citizens."  --  Bahá'u'lláh
    Just Say No to Morden * The Shadows were defeated -- Babylon 5 is renewed!!