Unit in the last place
In computer science and numerical analysis, unit in the last place or unit of least precision (ULP) is the spacing between floating-point numbers, i.e., the value the least significant digit represents if it is 1. It is used as a measure of accuracy in numeric calculations.
One definition is: In radix b with precision p, if be ≤ |x| < be+1, then ULP(x) = bmax(e,emin)−p+1.
Another definition, suggested by John Harrison, is slightly different: ULP(x) is the distance between the two closest straddling floating-point numbers a and b (i.e., those with a ≤ x ≤ b and a ≠ b), assuming that the exponent range is not upper-bounded. These definitions differ only at signed powers of the radix.
The IEEE 754 specification—followed by all modern floating-point hardware—requires that the result of an elementary arithmetic operation (addition, subtraction, multiplication, division, and square root since 1985, and FMA since 2008) be correctly rounded, which implies that in rounding to nearest, the rounded result is within 0.5 ULP of the mathematically exact result, using John Harrison's definition; conversely, this property implies that the distance between the rounded result and the mathematically exact result is minimized (but for the halfway cases, it is satisfied by two consecutive floating-point numbers). Reputable numeric libraries compute the basic transcendental functions to between 0.5 and about 1 ULP. Only a few libraries compute them within 0.5 ULP, this problem being complex due to the Table-maker's dilemma.
Let x be a positive floating-point number and assume that the active rounding attribute is round to nearest, ties to even, denoted RN. If ULP(x) is less than or equal to 1, then RN(x + 1) > x. Otherwise, RN(x + 1) = x or RN(x + 1) = x + ULP(x), depending on the value of the least significant digit and the exponent of x. This is demonstrated in the following Haskell code typed at an interactive prompt:
> until (\x -> x == x+1) (+1) 0 :: Float 1.6777216e7 > it-1 1.6777215e7 > it+1 1.6777216e7
Here we start with 0 in single precision and repeatedly add 1 until the operation does not change the value. Since the significand for a single-precision number contains 24 bits, the first integer that is not exactly representable is 224+1, and this value rounds to 224 in round to nearest, ties to even. Thus the result is equal to 224.
- p0 < π < p1
// π with 20 decimal digits BigDecimal π = new BigDecimal("3.14159265358979323846"); // truncate to a double floating point double p0 = π.doubleValue(); // -> 3.141592653589793 (hex: 0x1.921fb54442d18p1) // p0 is smaller than π, so find next number representable as double double p1 = Math.nextUp(p0); // -> 3.1415926535897936 (hex: 0x1.921fb54442d19p1)
Then ULP(π) is determined as
- ULP(π) = p1 - p0
// ulp(π) is the difference between p1 and p0 BigDecimal ulp = new BigDecimal(p1).subtract(new BigDecimal(p0)); // -> 4.44089209850062616169452667236328125E-16 // (this is precisely 2**(-51)) // same result when using the standard library function double ulpMath = Math.ulp(p0); // -> 4.440892098500626E-16 (hex: 0x1.0p-51)
Another example, in Python, also typed at an interactive prompt, is:
>>> x = 1.0 >>> p = 0 >>> while x != x + 1: ... x = x * 2 ... p = p + 1 ... >>> x 9007199254740992.0 >>> p 53 >>> x + 2 + 1 9007199254740996.0
In this case, we start with x = 1 and repeatedly double it until x = x + 1. Similarly to Example 1, the result is 253 because the double-precision floating-point format uses a 53-bit significand.
The Boost C++ libraries provides the functions
boost::math::float_advance to obtain nearby (and distant) floating-point values, and
boost::math::float_distance(a, b) to calculate the floating-point distance between two doubles.
The C language library provides functions to calculate the next floating-point number in some given direction:
long double, declared in
<math.h>. It also provides the macros
LDBL_EPSILON, which represent the positive difference between 1.0 and the next greater representable number in the corresponding type (i.e. the ULP of one).
The Swift standard library provides access to the next floating-point number in some given direction via the instance properties
nextUp. It also provides the instance property
ulp and the type property
ulpOfOne (which corresponds to C macros like
FLT_EPSILON) for Swift's floating-point types.
- David Goldberg: What Every Computer Scientist Should Know About Floating-Point Arithmetic, section 1.2 Relative Error and Ulps, ACM Computing Surveys, Vol 23, No 1, pp.8, March 1991.
- Muller, Jean-Michel; Brunie, Nicolas; de Dinechin, Florent; Jeannerod, Claude-Pierre; Joldes, Mioara; Lefèvre, Vincent; Melquiond, Guillaume; Revol, Nathalie; Torres, Serge (2018) . Handbook of Floating-Point Arithmetic (2 ed.). Birkhäuser. doi:10.1007/978-3-319-76526-6. ISBN 978-3-319-76525-9.
- Harrison, John. "A Machine-Checked Theory of Floating Point Arithmetic". Retrieved 17 July 2013.
- Muller, Jean-Michel (2005-11). "On the definition of ulp(x)". INRIA Technical Report 5504. ACM Transactions on Mathematical Software, Vol. V, No. N, November 2005. Retrieved in 2012-03 from http://ljk.imag.fr/membres/Carine.Lucas/TPScilab/JMMuller/ulp-toms.pdf.
- Kahan, William. "A Logarithm Too Clever by Half". Retrieved 14 November 2008.
- Boost float_advance.
- Boost float_distance.
- ISO/IEC 9899:1999 specification (PDF). p. 237, §126.96.36.199 The nextafter functions and §188.8.131.52 The nexttoward functions.
- "ulpOfOne - FloatingPoint | Apple Developer Documentation". Apple Inc. Apple Inc. Retrieved 18 August 2019.
- "FloatingPoint - Swift Standard Library | Apple Developer Documentation". Apple Inc. Apple Inc. Retrieved 18 August 2019.
|Look up ulp in Wiktionary, the free dictionary.|
- Goldberg, David (1991-03). "Rounding Error" in "What Every Computer Scientist Should Know About Floating-Point Arithmetic". Computing Surveys, ACM, March 1991. Retrieved from http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#689.
- Muller, Jean-Michel (2010). Handbook of floating-point arithmetic. Boston: Birkhäuser. pp. 32–37. ISBN 978-0-8176-4704-9.