manpagez: man pages & more
man ncgen4(1)
Home | html | info | man
ncgen4(1)                      UNIDATA UTILITIES                     ncgen4(1)




NAME

       ncgen4 - From a CDL file generate a netCDF-4 file or a C program


SYNOPSIS

       ncgen4  [-b]  [-c]  [-k  kind_of_file]  [-x]  [-n] [-o netcdf_filename]
              input_file


DESCRIPTION

       ncgen4 generates either a netCDF-4 (i.e. enhanced)  file  or  C  source
       code  to create a netCDF file.  The input to ncgen4 is a description of
       a netCDF file in a small language known as  CDL  (network  Common  Data
       form Language), described below.  If no options are specified in invok-
       ing ncgen4, it merely checks the syntax of the input CDL file,  produc-
       ing error messages for any violations of CDL syntax.  Other options can
       be used to create the corresponding netCDF file, or  to  generate  a  C
       program that uses the netCDF C interface to create the netCDF file.

       ncgen4  may  be  used with the companion program ncdump to perform some
       simple operations on netCDF files.  For example, to rename a  dimension
       in  a  netCDF file, use ncdump to get a CDL version of the netCDF file,
       edit the CDL file to change the name of the dimensions, and use  ncgen4
       to generate the corresponding netCDF file from the edited CDL file.


OPTIONS

       -b     Create  a  (binary)  netCDF file.  If the -o option is absent, a
              default file name will  be  constructed  from  the  netCDF  name
              (specified  after  the netcdf keyword in the input) by appending
              the `.nc' extension.  If a file already exists with  the  speci-
              fied name, it will be overwritten.

       -c     Generate  C  source code that will create a netCDF file matching
              the netCDF specification.  The C source code is written to stan-
              dard output.

       -o netcdf_file
              Name  for  the  binary  netCDF  file created.  If this option is
              specified, it implies the "-b" option.  (This option  is  neces-
              sary because netCDF files cannot be written directly to standard
              output, since standard output is not seekable.)

       -k kind_of_file
              The -k flag specifies the kind of file to be created and, by in-
              ference,  the data model accepted by ncgen (i.e. netcdf-3 (clas-
              sic) versus netcdf-4).  The possible arguments are  as  follows.
              '1', 'classic' => netcdf classic file format, netcdf-3 type mod-
              el.  '2', '64-bit-offset', '64-bit  offset'  =>  netcdf  64  bit
              classic   file   format,  netcdf-3  type  model.   '3',  'hdf5',
              'netCDF-4', 'enhanced' => netcdf-4 file  format,  netcdf-4  type
              model.   '4',  'hdf5-nc3',  'netCDF-4 classic model', 'enhanced-
              nc3' => netcdf-4 file format, netcdf-3 type model.  Note: -v  is
              also accepted to mean the same thing as -k for backward compati-
              bility, but -k is preferred, to match the  corresponding  ncdump
              option.

       -x     Don't  initialize data with fill values.  This can speed up cre-
              ation of large netCDF files greatly, but later attempts to  read
              unwritten  data  from  the generated file will not be easily de-
              tectable.


EXAMPLES

       Check the syntax of the CDL file `foo.cdl':

              ncgen4 foo.cdl

       From the CDL file `foo.cdl', generate an equivalent binary netCDF  file
       named `x.nc':

              ncgen4 -o x.nc foo.cdl

       From the CDL file `foo.cdl', generate a C program containing the netCDF
       function invocations necessary to create an  equivalent  binary  netCDF
       file named `x.nc':

              ncgen4 -c -o x.nc foo.cdl



USAGE

   CDL Syntax Overview
       Below is an example of CDL syntax, describing a netCDF file with sever-
       al named dimensions (lat, lon, and time), variables (Z, t, p, rh,  lat,
       lon,  time), variable attributes (units, long_name, valid_range, _Fill-
       Value), and some data.  CDL keywords are in boldface.  (This example is
       intended  to  illustrate  the syntax; a real CDL file would have a more
       complete set of attributes so that the data would  be  more  completely
       self-describing.)
              netcdf foo {  // an example netCDF specification in CDL

              types:
                  ubyte enum enum_t {Clear = 0, Cumulonimbus = 1, Stratus = 2};
                  opaque(11) opaque_t;
                  int(*) vlen_t;

              dimensions:
                   lat = 10, lon = 5, time = unlimited ;

              variables:
                   long    lat(lat), lon(lon), time(time);
                   float   Z(time,lat,lon), t(time,lat,lon);
                   double  p(time,lat,lon);
                   long    rh(time,lat,lon);

                   string  country(time,lat,lon);
                   ubyte   tag;

                   // variable attributes
                   lat:long_name = "latitude";
                   lat:units = "degrees_north";
                   lon:long_name = "longitude";
                   lon:units = "degrees_east";
                   time:units = "seconds since 1992-1-1 00:00:00";

                   // typed variable attributes
                   string Z:units = "geopotential meters";
                   float Z:valid_range = 0., 5000.;
                   double p:_FillValue = -9999.;
                   long rh:_FillValue = -1;
                   vlen_t :globalatt = {17, 18, 19};
              data:
                   lat   = 0, 10, 20, 30, 40, 50, 60, 70, 80, 90;
                   lon   = -140, -118, -96, -84, -52;
              group: g {
              types:
                  compound cmpd_t { vlen_t f1; enum_t f2;};
              } // group g
              group: h {
              variables:
                   /g/cmpd_t  compoundvar;
              data:
                      compoundvar = { {3,4,5}, Stratus } ;
              } // group h
              }

       All  CDL  statements  are terminated by a semicolon.  Spaces, tabs, and
       newlines can be used freely for readability.  Comments may  follow  the
       characters `//' on any line.

       A  CDL  description consists of five optional parts: types, dimensions,
       variables, data, beginning with the keyword types:, dimensions:,  vari-
       ables:, and data, respectively.  The variable part may contain variable
       declarations and attribute assignments.  All sections may contain glob-
       al attribute assignments.

       In  addition,  after the data: section, the user may define a series of
       groups (see the example above).  Groups themselves can  contain  types,
       dimensions, variables, data, and other (nested) groups.

       The  netCDF type section declares the user defined types.  These may be
       constructed using any of the following types: enum,  vlen,  opaque,  or
       compound.

       A  netCDF  dimension  is used to define the shape of one or more of the
       multidimensional variables contained in the netCDF file.  A netCDF  di-
       mension  has  a  name  and  a size.  A dimension can have the unlimited
       size, which means a variable using  this  dimension  can  grow  to  any
       length in that dimension.

       A  variable  represents  a multidimensional array of values of the same
       type.  A variable has a name, a data type, and a shape described by its
       list  of dimensions.  Each variable may also have associated attributes
       (see below) as well as data values.  The name, data type, and shape  of
       a  variable are specified by its declaration in the variable section of
       a CDL description.  A variable may have the same name as  a  dimension;
       by  convention  such a variable is one-dimensional and contains coordi-
       nates of the dimension it names.  Dimensions need not have  correspond-
       ing variables.

       A  netCDF  attribute  contains  information  about a netCDF variable or
       about the whole netCDF dataset.  Attributes are used  to  specify  such
       properties  as units, special values, maximum and minimum valid values,
       scaling factors, offsets, and  parameters.   Attribute  information  is
       represented by single values or arrays of values.  For example, "units"
       is an attribute represented by a character array such as "celsius".  An
       attribute  has  an  associated variable, a name, a data type, a length,
       and a value.  In contrast to variables that are intended for data,  at-
       tributes are intended for metadata (data about data).  Unlike netCDF-3,
       attribute types can be any user defined  type  as  well  as  the  usual
       built-in types.

       In  CDL, an attribute is designated by a a type, a variable, a ':', and
       then an attribute name.  The type is optional and if missing,  it  will
       be  inferred from the values assigned to the attribute.  It is possible
       to assign global attributes not associated with  any  variable  to  the
       netCDF as a whole by omitting the variable name in the attribute decla-
       ration.  Notice that there is a potential ambiguity in a  specification
       such as
       x : a = ...
       In  this situation, x could be either a type for a global attribute, or
       the variable name for an attribute. Since there could both  be  a  type
       named  x  and  a  variable named x, there is an ambiguity.  The rule is
       that in this situation, x will be interpreted as a  type  if  possible,
       and otherwise as a variable.

       If  not specified, the data type of an attribute in CDL is derived from
       the type of the value(s) assigned to it.  The length of an attribute is
       the  number  of data values assigned to it, or the number of characters
       in the character string assigned to it.  Multiple values  are  assigned
       to  non-character attributes by separating the values with commas.  All
       values assigned to an attribute must be of the same type.

       The names for CDL dimensions, variables, and attributes must begin with
       an  alphabetic  character  or `_', and subsequent characters may be al-
       phanumeric or `_' or `-'.

       The optional data section of a CDL specification is where netCDF  vari-
       ables may be initialized.  The syntax of an initialization is simple: a
       variable name, an equals sign, and a comma-delimited list of  constants
       (possibly  separated  by  spaces,  tabs and newlines) terminated with a
       semicolon.  For multi-dimensional arrays,  the  last  dimension  varies
       fastest.  Thus row-order rather than column order is used for matrices.
       If fewer values are supplied than are needed to fill a variable, it  is
       extended with a type-dependent `fill value', which can be overridden by
       supplying a value for a distinguished variable attribute named  `_Fill-
       Value'.   The types of constants need not match the type declared for a
       variable; coercions are done to convert integers to floating point, for
       example.   The constant `_' can be used to designate the fill value for
       a variable.

   Primitive Data Types
              char characters
              byte 8-bit data
              short     16-bit signed integers
              int  32-bit signed integers
              long (synonymous with int)
              int64     64-bit signed integers
              float     IEEE single precision floating point (32 bits)
              real (synonymous with float)
              double    IEEE double precision floating point (64 bits)
              ubyte     unsigned 8-bit data
              ushort    16-bit unsigned integers
              uint 32-bit unsigned integers
              uint64    64-bit unsigned integers
              string    arbitrary length strings

       CDL supports a superset of the primitive data types of  C.   The  names
       for the primitive data types are reserved words in CDL, so the names of
       variables, dimensions, and attributes must not be primitive type names.
       In  declarations,  type names may be specified in either upper or lower
       case.

       Bytes differ from characters in that they are intended to hold  a  full
       eight  bits  of data, and the zero byte has no special significance, as
       it does for character data.  ncgen4 converts byte declarations to  char
       declarations  in the output C code and to the nonstandard BYTE declara-
       tion in output Fortran code.

       Shorts can hold values between -32768 and 32767.  ncgen4 converts short
       declarations to short declarations in the output C code and to the non-
       standard INTEGER*2 declaration in output Fortran code.

       Ints can hold values between -2147483648 and 2147483647.   ncgen4  con-
       verts  int declarations to int declarations in the output C code and to
       INTEGER declarations in output Fortran code.  long  is  accepted  as  a
       synonym  for int in CDL declarations, but is deprecated since there are
       now platforms with 64-bit representations for C longs.

       Int64   can    hold    values    between    -9223372036854775808    and
       9223372036854775807.   ncgen4  converts  int64 declarations to longlong
       declarations in the output C code.

       Floats can hold values between about -3.4+38 and 3.4+38.  Their  exter-
       nal representation is as 32-bit IEEE normalized single-precision float-
       ing point numbers.  ncgen4 converts float declarations to float  decla-
       rations in the output C code and to REAL declarations in output Fortran
       code.  real is accepted as a synonym for float in CDL declarations.

       Doubles can hold values between about -1.7+308 and 1.7+308.  Their  ex-
       ternal representation is as 64-bit IEEE standard normalized double-pre-
       cision floating point numbers.  ncgen4 converts double declarations  to
       double declarations in the output C code and to DOUBLE PRECISION decla-
       rations in output Fortran code.

       The unsigned counterparts of the above integer types are mapped to  the
       corresponding  unsigned C types.  Their ranges are suitably modified to
       start at zero.


   CDL Constants
       Constants assigned to attributes or variables may be of any of the  ba-
       sic netCDF types.  The syntax for constants is similar to C syntax, ex-
       cept that type suffixes must be appended to shorts and floats  to  dis-
       tinguish them from longs and doubles.

       A  byte constant is represented by a single character or multiple char-
       acter escape sequence enclosed in single quotes.  For example,
               'a'      // ASCII `a'
               '\0'          // a zero byte
               '\n'          // ASCII newline character
               '\33'         // ASCII escape character (33 octal)
               '\x2b'   // ASCII plus (2b hex)
               '\377'   // 377 octal = 255 decimal, non-ASCII

       Character constants are enclosed in double quotes.  A  character  array
       may  be represented as a string enclosed in double quotes.  The usual C
       string escape conventions are honored.  For example
              "a"       // ASCII `a'
              "Two\nlines\n" // a 10-character string with two embedded newlines
              "a bell:\007"  // a string containing an ASCII bell
       Note that the netCDF character array "a" would  fit  in  a  one-element
       variable,  since  no terminating NULL character is assumed.  However, a
       zero byte in a character array is interpreted as the end of the signif-
       icant  characters  by  the  ncdump program, following the C convention.
       Therefore, a NULL byte should not be embedded in a character string un-
       less  at  the  end: use the byte data type instead for byte arrays that
       contain the zero byte.

       short integer constants are intended  for  representing  16-bit  signed
       quantities.   The  form of a short constant is an integer constant with
       an `s' or `S' appended.  If a short constant begins with `0', it is in-
       terpreted  as  octal,  except that if it begins with `0x', it is inter-
       preted as a hexadecimal constant.  For example:
              -2s  // a short -2
              0123s     // octal
              0x7ffs  //hexadecimal

       int integer constants are intended for representing 32-bit signed quan-
       tities.   The  form of an int constant is an ordinary integer constant,
       although it is acceptable to append an optional `l' or `L' (again, dep-
       recated).  If an int constant begins with `0', it is interpreted as oc-
       tal, except that if it begins with `0x', it is interpreted as  a  hexa-
       decimal  constant  (but see opaque constants below).  Examples of valid
       int constants include:
              -2
              1234567890L
              0123      // octal
              0x7ff          // hexadecimal

       int64 integer constants are intended  for  representing  64-bit  signed
       quantities.   The form of an int64 constant is an integer constant with
       an `ll' or `LL' appended.  If an int64 constant begins with `0', it  is
       interpreted  as octal, except that if it begins with `0x', it is inter-
       preted as a hexadecimal constant.  For example:
              -2ll // an unsigned -2
              0123LL    // octal
              0x7ffLL  //hexadecimal

       Floating point constants of type float are appropriate for representing
       floating  point  data with about seven significant digits of precision.
       The form of a float constant is the same as a C floating point constant
       with an `f' or `F' appended.  For example the following are all accept-
       able float constants:
              -2.0f
              3.14159265358979f   // will be truncated to less precision
              1.f

       Floating point constants of type double are appropriate for  represent-
       ing floating point data with about sixteen significant digits of preci-
       sion.  The form of a double constant is the same as a C floating  point
       constant.   An  optional  `d'  or `D' may be appended.  For example the
       following are all acceptable double constants:
              -2.0
              3.141592653589793
              1.0e-20
              1.d

       Unsigned integer constants can be created by  appending  the  character
       'U'  or 'u' between the constant and any trailing size specifier.  Thus
       one could say 10U, 100us, 100000ul, or 1000000ull, for example.

       String constants are, like character constants, represented using  dou-
       ble quotes. This represents a potential ambiguity since a multi-charac-
       ter string may also indicate a dimensioned character value. Disambigua-
       tion  usually  occurs  by  context, but care should be taken to specify
       thestring type to ensure the proper choice.

       Opaque constants are represented as  sequences  of  hexadecimal  digits
       preceded  by  0X  or  0x: 0xaa34ffff, for example.  These constants can
       still be used as integer constants and will be either truncated or  ex-
       tended as necessary.

   Compound Constant Expressions
       In  order  to  assign values to variables (or attributes) whose type is
       user-defined type, the constant notation has been extended  to  include
       sequences  of  constants  enclosed  in curly brackets (e.g. "{"..."}").
       Such a constant is called a compound constant, and  compound  constants
       can be nested.

       Given  a type "T(*) vlen_t", where T is some other arbitrary base type,
       constants for this should be specified as follows.
           vlen_t var[2] = {{t11,t12,...t1N}, {t21,t22,...t2m}};
       The values tij, are assumed to be constants of type T.

       Given a type "compound cmpd_t {T1 f1; T2 f2...Tn fn}", where the Ti are
       other  arbitrary  base types, constants for this should be specified as
       follows.
           cmpd_t var[2] = {{t11,t12,...t1N}, {t21,t22,...t2n}};
       The values tij, are assumed to be constants of type Ti.  If the  fields
       are  missing, then they will be set using any specified or default fill
       value for the field's base type.

   Scoping Rules
       With the addition of groups, the name space for defined objects  is  no
       longer flat. References (names) of any type, dimension, or variable may
       be prefixed with the absolute path specifying a  specific  declaration.
       Thus one might say
           variables:
               /g1/g2/t1 v1;
       The  type  being  referenced  (t1) is the one within group g2, which in
       turn is nested in group g1.  The similarity of this  notation  to  Unix
       file  paths is deliberate, and one can consider groups as a form of di-
       rectory structure.

       1. When name is not prefixed, then scope rules are  applied  to  locate
              the specified declaration. Currently, there are three rules: one
              for dimensions, one for types and enumeration constants, and one
              for all others.

       2.  When  an  unprefixed  name of a dimension is used (as in a variable
              declaration), ncgen4 first looks in  the  immediately  enclosing
              group  for  the  dimension.   If  it is not found there, then it
              looks in the group enclosing this group.  This continues up  the
              group  hierarchy  until  the dimension is found, or there are no
              more groups to search.

       3. For all  other  names,  only  the  immediately  enclosing  group  is
              searched.

       When  an  unprefixed name of a type or an enumeration constant is used,
       ncgen4 searches the group tree using a  pre-order  depth-first  search.
       This  essentially means that it will find the matching declaration that
       precedes the reference textually in the cdl file and that is  "highest"
       in the group hierarchy.

       One  final  note.  Forward references are not allowed.  This means that
       specifying, for example, /g1/g2/t1 will fail if this  reference  occurs
       before g1 and/or g2 are defined.

   Special Attributes
       Special,  virtual,  attributes can be specified to provide performance-
       related information about the file format and  about  variable  proper-
       ties.  The file must be a netCDF-4 file for these to take effect.

       These  special  virtual  attributes  are not actually part of the file,
       they are merely a convenient way to set miscellaneous properties of the
       data in CDL

       The  special  attributes currently supported are as follows: `_Format',
       `_Fletcher32, `_ChunkSizes',  `_Endianness',  `_DeflateLevel',  `_Shuf-
       fle', and `_Storage'.

       `_Format'  is  a global attribute specifying the netCDF format variant.
       Its value must be a single string matching one  of  `classic',  `64-bit
       offset', `netCDF-4', or `netCDF-4 classic model'.

       The rest of the special attributes are all variable attributes.  Essen-
       tially all of then map to some corresponding `nc_def_var_XXX'  function
       as  defined  in  the  netCDF-4 API.  `_Fletcher32 sets the `fletcher32'
       property for a variable.  `_Endianness' is either  `little'  or  `big',
       depending  on  how  the  variable  is  stored when first written. `_De-
       flateLevel' is an integer between 0 and 9 inclusive if compression  has
       been specified for the variable.  `_Shuffle' is 1 if use of the shuffle
       filter is specified for the variable.  `_Storage'  is  `contiguous'  or
       `chunked'.   `_ChunkSizes'  is a list of chunk sizes for each dimension
       of the variable


   Specifying Datalists
       Specifying datalists for variables in the `data:` section can be  some-
       what  complicated. There are some rules that will ensure that datalists
       are parsed correctly by ncgen4.

       1. The top level of the datalist is assumed to be a list of 1  or  more
       items, so do not put the top level in braces (`{...}').

       2.  Sublists  that are intended to match VLEN's must always be enclosed
       in braces so that the parser knows how many items are in the VLEN.

       3. Sublists that are intended to match nested UNLIMITED dimensions must
       be  enclosed  in  braces so that the parser knows how many items are in
       the UNLIMITED dimension.  Note however, that if the UNLIMITED dimension
       is  the  first dimension, then braces should not be used (to be consis-
       tent with #1 above).

       4. Sublists for bounded size dimensions may optionally be  enclosed  in
       braces.   For  example, [2][3] may be represented as {1,2,3},{4,5,6} or
       as 1,2,3,4,5,6.  The former is useful if  you  want  to  get  automatic
       filling.   That  is,  {1,2},{4,5,6} will be treated as {1,2,_},{4,5,6},
       whereas 1,2,4,5,6 will be treated as 1,2,4,5,_.


BUGS

       The programs generated by ncgen4 when using the -c flag use initializa-
       tion  statements  to  store data in variables, and will fail to produce
       compilable programs if you try to use them for  large  datasets,  since
       the  resulting  statements may exceed the line length or number of con-
       tinuation statements permitted by the compiler.

       The CDL syntax makes it easy to assign what  looks  like  an  array  of
       variable-length  strings to a netCDF variable, but the strings may sim-
       ply be concatenated into a single array of characters.  Specific use of
       the string type specifier may solve the problem



Printed: 109-5-23        $Date: 2009/01/02 18:49:06 $                ncgen4(1)

netcdf 4.0.1 - Generated Sat May 23 08:05:36 CDT 2009
© manpagez.com 2000-2021
Individual documents may contain additional copyright information.