HomeSort by relevance Sort by last modified time
    Searched full:precision (Results 2826 - 2850 of 7852) sorted by null

<<111112113114115116117118119120>>

  /external/python/cpython2/Lib/test/decimaltestdata/
rounding.decTest 36 precision: 5 -- for easier visual inspection
    [all...]
  /external/python/cpython2/Lib/test/
test_random.py 234 # This should pass whenever a C double has 53 bit precision.
330 # code. Create 2000 53-bit precision random floats. Compare only
362 # at least 53 bits of precision (the underlying algorithm suffers
391 # This should pass whenever a C double has 53 bit precision.
  /external/python/cpython2/Modules/_ctypes/libffi_msvc/
README 369 all supposed to be stored in double-precision format, even if they are
370 only single-precision, but EGCS stores single-precision arguments as
371 single-precision anyway. This causes one test to fail (the `many
  /external/python/cpython2/Modules/_ctypes/libffi_osx/
README 369 all supposed to be stored in double-precision format, even if they are
370 only single-precision, but EGCS stores single-precision arguments as
371 single-precision anyway. This causes one test to fail (the `many
  /external/python/cpython2/Modules/_io/
_iomodule.h 97 precision. */
  /external/python/cpython3/Include/
pytime.h 16 /* _PyTime_t: Python timestamp with subsecond precision. It can be used to
  /external/python/cpython3/Lib/test/decimaltestdata/
copy.decTest 23 precision: 9
copyabs.decTest 23 precision: 9
copynegate.decTest 23 precision: 9
ddAbs.decTest 22 precision: 16
ddClass.decTest 23 precision: 16
ddCopy.decTest 23 precision: 16
ddCopyAbs.decTest 23 precision: 16
ddCopyNegate.decTest 23 precision: 16
ddMinus.decTest 23 precision: 16
ddPlus.decTest 23 precision: 16
dqAbs.decTest 24 precision: 34
dqClass.decTest 26 precision: 34
dqCopy.decTest 25 precision: 34
dqCopyAbs.decTest 25 precision: 34
dqCopyNegate.decTest 25 precision: 34
dqMinus.decTest 25 precision: 34
dqPlus.decTest 25 precision: 34
rounding.decTest 36 precision: 5 -- for easier visual inspection
    [all...]
  /external/python/cpython3/Modules/_ctypes/libffi_msvc/
README 369 all supposed to be stored in double-precision format, even if they are
370 only single-precision, but EGCS stores single-precision arguments as
371 single-precision anyway. This causes one test to fail (the `many

Completed in 1309 milliseconds

<<111112113114115116117118119120>>