recentpopularlog in

kme : floatingpoint   12

linux - bash + arithmetic calculation with bash - Unix & Linux Stack Exchange
Since arithmetic expansion only does integer division, the proposed answer uses AWK, passing in values as variables, with '/dev/null' as the input file, and the math in a 'BEGIN' block.
bash  awk  shellscripting  floatingpoint  math  tipsandtricks  solution 
10 weeks ago by kme
python numpy machine epsilon - Stack Overflow |

An easier way to get the machine epsilon for a given float type is to use np.finfo():

<code class="language-python">print(np.finfo(float).eps)
# 2.22044604925e-16

# 1.19209e-07</code>

Also interesting, but I wonder why this works:

Another easy way to get epsilon is:
<code class="language-python">
In [1]: 7./3 - 4./3 -1
Out[1]: 2.220446049250313e-16
python  float  floatingpoint  math  solution 
may 2018 by kme
Machine epsilon - Wikipedia, the free encyclopedia
Machine epsilon gives an upper bound on the relative error due to rounding in floating point arithmetic. This value characterizes computer arithmetic in the field of numerical analysis, and by extension in the subject of computational science. The quantity is also called macheps or unit roundoff, and it has the symbols Greek epsilon epsilon or bold Roman u, respectively.
cs  programming  floatingpoint  fpweirdness  reference 
february 2013 by kme
floating point - Matlab precision: simple subtraction is not zero - Stack Overflow
Refer to the documentation for 'quantization error' or 'eps' (a representation of MATLAB's floating point precision).
matlab  floatingpoint  precision  tipsandtricks  maybesolution  dammitbrain 
january 2013 by kme

Copy this bookmark:

to read