Skip to content

Text.Megaparsec.Byte.Lexer.decimal returns wrong value for Float #479

Open
@jchia

Description

@jchia
ghci> parse (decimal @() @ByteString @_ @Float) "" "655361200000"
Right 6.5536125e11
ghci> (== 655361250000) <$> parse (decimal @() @ByteString @_ @Float) "" "655361200000"
Right True
ghci> (== 655361200000) <$> parse (decimal @() @ByteString @_ @Float) "" "655361200000"
Right False
ghci> (subtract $ fromIntegral @_ @Float 655361200000) <$> parse (decimal @() @ByteString @_ @Float) "" "655361200000"
Right 65536.0

The Float value produced is not equal to what fromIntegral @_ @Float 655361200000 returns.

It appears that decimal accumulates floating-point errors during parsing so that the final Float result is wrong. (Given the limited precision of Float, error is expected but the result is further from the perfectly accurate value than necessary.) The error probably accumulates from lossy arithmetic performed for each input character, including the (* 10).

Double is likely similarly affected, but I have not constructed a counter-example, which probably requires more digit than Float counter-example.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions