Lines Matching full:precision
57 alterable precision (defaulting to 28 places) which can be as large as needed for
89 The context for arithmetic is an environment specifying precision, rounding
124 precision, rounding, or enabled traps::
132 >>> getcontext().prec = 7 # Set a new precision
178 input. Context precision and rounding only come into play during arithmetic
308 rounded (digits beyond the context precision were thrown away) and that the
373 53 or more digits of precision. For example, ``Decimal(float('1.1'))``
377 The *context* precision does not affect how many digits are stored. That is
379 ``Decimal('3.00000')`` records all five zeros even if the context precision is
772 quantize operation would be greater than precision, then an
817 the range -precision through precision. The absolute value of the second
821 length precision if necessary. The sign and exponent of the first operand
843 the range -precision through precision. The absolute value of the second
851 Return the square root of the argument to full precision.
903 Contexts are environments for arithmetic operations. They govern precision, set
930 For example, the following code sets the current decimal precision to 42 places,
936 ctx.prec = 42 # Perform a high precision calculation
938 s = +s # Round the final result back to the default precision
947 Specification. Precision is set to nine. Rounding is set to
958 Specification. Precision is set to nine. Rounding is set to
971 contexts. Changing a field (such a precision) has the effect of changing the
998 the precision for arithmetic operations in the context.
1064 context. Unlike the :class:`Decimal` constructor, the context precision,
1067 This is useful because constants are often given to a greater precision
1070 precision. In the following example, using unrounded inputs means that
1089 the context precision, rounding method, flags, and traps are applied to
1341 operation applies the context precision and rounding, so it is *not* an
1351 precision' digits.
1366 - ``modulo`` must be nonzero and have at most 'precision' digits
1370 % modulo`` with unbounded precision, but is computed more
1422 Square root of a non-negative number to context precision.
1660 Mitigating round-off error with increased precision
1665 can still incur round-off error when non-zero digits exceed the fixed precision.
1670 insufficient precision causes the breakdown of the associative and distributive
1692 expanding the precision sufficiently to avoid loss of significance:
1753 precision. Since their magnitude is zero, both positive and negative zeros are
1865 """Compute Pi to the current precision.
1881 return +s # unary plus applies the new precision
2066 Decimal though an exact conversion may take more precision than intuition would
2075 spurious result because of insufficient precision or rounding anomalies.
2078 re-run calculations using greater precision and with various rounding modes.
2079 Widely differing results indicate insufficient precision, rounding mode issues,
2082 Q. I noticed that context precision is applied to the results of operations but
2099 The solution is either to increase precision or to force rounding of inputs
