Lines Matching refs:prec
337 * Returns value / 2 ** prec rounded to an integer.396 int prec = 0;398 for (;prec > n + 30; prec = (prec * 3)/2 - 16) {399 int msd = msd(prec);401 check_prec(prec);1329 // prec. This assumes |x| < 1/2.
