recentpopularlog in

wjy : floatingpoint   5

0.30000000000000004 - Floating Point Math
Floating Point Math Your language isn't broken, it's doing floating point math. Computers can only natively store integers, so they need some way of representing decimal numbers. This representation comes with some degree of inaccuracy. That's why, more often than not, .1 + .2 != .3.
programming  floatingpoint  math  ieee754 
november 2015 by wjy
DEC64: Decimal Floating Point
DEC64 is a number type. It can precisely represent decimal fractions with 16 decimal places, which makes it very well suited to all applications that are concerned with money. It can represent values as gargantuan as 3.6028797018963967E+143 or as measly as 1.0E-127, which makes it well suited to most scientific applications. It can provide very fast performance on integer values, eliminating the need for a separate int type and avoiding the terrible errors than can result from int truncation. DEC64 is intended to be the only number type in the next generation of application programming languages.
numbers  programming  dec64  decimal  floatingpoint 
november 2014 by wjy

Copy this bookmark:

to read