manpagez: man pages & more
info gawk
Home | html | info | man

gawk: Setting precision

 15.4.4 Setting the Precision
 'gawk' uses a global working precision; it does not keep track of the
 precision or accuracy of individual numbers.  Performing an arithmetic
 operation or calling a built-in function rounds the result to the
 current working precision.  The default working precision is 53 bits,
 which you can modify using the predefined variable 'PREC'.  You can also
 set the value to one of the predefined case-insensitive strings shown in
 ⇒Table 15.3 table-predefined-precision-strings, to emulate an IEEE
 754 binary format.
 'PREC'       IEEE 754 binary format
 '"half"'     16-bit half-precision
 '"single"'   Basic 32-bit single precision
 '"double"'   Basic 64-bit double precision
 '"quad"'     Basic 128-bit quadruple precision
 '"oct"'      256-bit octuple precision
 Table 15.3: Predefined precision strings for 'PREC'
    The following example illustrates the effects of changing precision
 on arithmetic operations:
      $ gawk -M -v PREC=100 'BEGIN { x = 1.0e-400; print x + 0
      >   PREC = "double"; print x + 0 }'
      -| 1e-400
      -| 0
      CAUTION: Be wary of floating-point constants!  When reading a
      floating-point constant from program source code, 'gawk' uses the
      default precision (that of a C 'double'), unless overridden by an
      assignment to the special variable 'PREC' on the command line, to
      store it internally as an MPFR number.  Changing the precision
      using 'PREC' in the program text does _not_ change the precision of
      a constant.
      If you need to represent a floating-point constant at a higher
      precision than the default and cannot use a command-line assignment
      to 'PREC', you should either specify the constant as a string, or
      as a rational number, whenever possible.  The following example
      illustrates the differences among various ways to print a
      floating-point constant:
           $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }'
           -| 0.1000000000000000055511151
           $ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }'
           -| 0.1000000000000000000000000
           $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }'
           -| 0.1000000000000000000000000
           $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }'
           -| 0.1000000000000000000000000
© 2000-2018
Individual documents may contain additional copyright information.