## Changes in CGNS 3.1

- Time-dependent Connectivity
- Zone Subregions
- General SIDS Improvements
- 64-bit Implementation
  - C Interface
  - C Portability
  - FORTRAN Interface
  - FORTRAN Issues and Portability
  - Examples

## Time-dependent Connectivity

CPEX0027 has been implemented. The added functions to support this extension are described in Zone Grid Connectivity in the Mid-Level Library.

## Zone Subregions

CPEX0030 has been implemented. The added functions to support this extension are described in Zone Subregions in the Mid-Level Library.

## General SIDS Improvements

CPEX0031 has been implemented. The added functions to support this extension are described in Flow Solution and Discrete Data in the Mid-Level Library. There are also changes to Boundary Conditions and numerous places in the SIDS.

## 64-bit Implementation

You only need to read the following if you have a 64-bit computer and need to work with problems that exceed 2 billion coordinates and/or elements. Or, if you are bored.

### C Interface

The major change in 3.1 is the introduction of the data type `cgsize_t`, which will be a 64-bit integer when building 64-bit code, and a 32-bit integer otherwise. This is defined in a new header file, `cgnstypes.h`, which is included in `cgnslib.h`. Some additional data types have also been introduced, `cglong_t` which is always 64-bits, `cgint_t` and `cgerr_t` which are 32-bit integers, and `cgid_t` which is a 64-bit floating point number (`double`). With the exception of `cglong_t` these other data types are currently not used. The `cglong_t` data type is used internally, and may be used by application code in conjunction with `DataType_t LongInteger` to write 64-bit data to the CGNS file, regardless of the compilation mode. The `cgnstypes.h` header file also defines 3 flags, which describe the current build mode, `CG_BUILD_LEGACY`, `CG_BUILD_64BIT`, and `CG_BUILD_SCOPE`, which are set to 0 or 1, depending on the build mode.

When the `--enable-legacy` option is given to `configure`, `cgsize_t` is defined as an `int` (`integer*4`), `CG_BUILD_LEGACY` is set to 1, and `CG_BUILD_64BIT` and `CG_BUILD_SCOPE` are set to 0. The `--enable-scope` and `--enable-64bit` options to `configure` are ignored. This will generate the same code as for the versions prior to 3.1.

When the `--enable-64bit` option is given to `configure`, `cgsize_t` is typedef'd as `long`, `long long` or `__int64` (depending on the machine) and the equivalent FORTRAN data type is `integer*8`. All other values are unchanged. This will set `CG_BUILD_64BIT` to 1, and `CG_BUILD_LEGACY` to 0 in `cgnstypes.h`. The value of `CG_BUILD_SCOPE` depends on the `configure` option below.
If this option is not given, then `cgsize_t` is typedef'd as `int`, and the equivalent FORTRAN data type is `integer*4`. The value of `CG_BUILD_64BIT` is set to 0.

When the `--enable-scope` option is given to `configure`, scoping of the enumeration values in cgnslib are enabled. This will prefix all the enums with `CG_`. This is disabled by default and when `--enable-legacy` is enabled. If enabled, this will set the value of `CG_BUILD_SCOPE` in `cgnstypes.h` to 1.

These options are also available for cmake users, and have the same behavior.

When building 64-bit code, the ADF file format changes, and thus will not be readable by prior versions of the library. All dimensions are stored as 64-bit integers, and the modifications to the ADF interface allow addressing and storage up to 1,000 terabytes of data. The HDF5 interface is also effected due to the the dimension sizes. I'm not sure of the HDF5 limitations to file size, but I expect it to be as large as that addressable by a 64-bit integer.

Thus, when writing a CGNS file in 64-bit mode it will only be readable by the version 3.1 software. A 32-bit compilation of version 3.1 will be able to read the 64-bit CGNS file, but will fail gracefully if the dimensions exceed those that can be handled by a 32-bit integer. With a 32-bit mode compilation, the ADF version and HDF5 interfaces are unchanged, and thus the CGNS files will be readable by earlier version software.

Regardless of how the version 3.1 software is compiled, it is backwards compatible with earlier version CGNS files.

A new data type has also been added to the CGNS MLL, `LongInteger`, which is defined in the `DataType_t` enumeration. This refers to a true 64-bit integer. Previous versions of the MLL did not explicitly allow for 64-bit integer data. This data type is automatically used for dimensions, connectivity, boundary condition points, holes and elements when compiled in 64-bit mode. It may also be used by an application to define array data as 64-bit, regardless of the compilation mode. In this case, the data should be defined as `cglong_t` in the code in order to guarantee 64-bit data being passed to the array read or write routines.

The following routines have been modified to use the `cgsize_t` data type.

    cg_zone_read(int fn, int B, int Z, char *zonename, cgsize_t *size);
    cg_zone_write(int fn, int B, const char * zonename,
    	const cgsize_t * size, CGNS_ENUMT(ZoneType_t) type, int *Z);

    cg_coord_read(int fn, int B, int Z, const char * coordname,
    	CGNS_ENUMT(DataType_t) type, const cgsize_t * rmin,
    	const cgsize_t * rmax, void *coord);
    cg_coord_partial_write(int fn, int B, int Z,
    	CGNS_ENUMT(DataType_t) type, const char * coordname,
            const cgsize_t *rmin, const cgsize_t *rmax,
            const void * coord_ptr, int *C);

    cg_section_read(int file_number, int B, int Z, int S,
    	char *SectionName,  CGNS_ENUMT(ElementType_t) *type,
    	cgsize_t *start, cgsize_t *end, int *nbndry, int *parent_flag);
    cg_section_write(int file_number, int B, int Z,
    	const char * SectionName, CGNS_ENUMT(ElementType_t) type,
    	cgsize_t start, cgsize_t end, int nbndry, const cgsize_t * elements,
    	int *S);
    cg_section_partial_write(int file_number, int B, int Z,
    	const char * SectionName, CGNS_ENUMT(ElementType_t) type,
    	cgsize_t start, cgsize_t end, int nbndry, int *S);

    cg_elements_read(int file_number, int B, int Z, int S,
    	cgsize_t *elements, cgsize_t *parent_data);
    cg_elements_partial_write(int fn, int B, int Z, int S,
    	cgsize_t start, cgsize_t end, const cgsize_t *elements);
    cg_elements_partial_read(int file_number, int B, int Z, int S,
    	cgsize_t start, cgsize_t end, cgsize_t *elements, cgsize_t *parent_data);

    cg_parent_data_write(int file_number, int B, int Z, int S,
    	const cgsize_t * parent_data);
    cg_parent_data_partial_write(int fn, int B, int Z, int S,
    	cgsize_t start, cgsize_t end, const cgsize_t *ParentData);

    cg_ElementDataSize(int file_number, int B, int Z, int S,
    	cgsize_t *ElementDataSize);
    cg_ElementPartialSize(int file_number, int B, int Z, int S,
    	cgsize_t start, cgsize_t end, cgsize_t *ElementDataSize);

    cg_field_read(int fn, int B, int Z, int S, const char *fieldname,
    	CGNS_ENUMT(DataType_t) type, const cgsize_t *rmin,
            const cgsize_t *rmax, void *field_ptr);
    cg_field_partial_write(int fn, int B, int Z, int S,
    	CGNS_ENUMT(DataType_t) type, const char * fieldname,
    	const cgsize_t *rmin, const cgsize_t *rmax,
            const void * field_ptr, int *F);

    cg_hole_info(int fn, int B, int Z, int I, char *holename,
    	CGNS_ENUMT(GridLocation_t) *location,  CGNS_ENUMT(PointSetType_t) *ptset_type,
    	int *nptsets, cgsize_t *npnts);
    cg_hole_read(int fn, int B, int Z, int I, cgsize_t *pnts);
    cg_hole_write(int fn, int B, int Z, const char * holename,
    	CGNS_ENUMT(GridLocation_t) location, CGNS_ENUMT(PointSetType_t) ptset_type,
    	int nptsets, cgsize_t npnts, const cgsize_t * pnts, int *I);

    cg_conn_info(int file_number, int B, int Z, int I,
    	char *connectname, CGNS_ENUMT(GridLocation_t) *location,
    	CGNS_ENUMT(GridConnectivityType_t) *type,
    	CGNS_ENUMT(PointSetType_t) *ptset_type,
    	cgsize_t *npnts, char *donorname,
    	CGNS_ENUMT(ZoneType_t) *donor_zonetype,
    	CGNS_ENUMT(PointSetType_t) *donor_ptset_type,
            CGNS_ENUMT(DataType_t) *donor_datatype,
    	cgsize_t *ndata_donor);
    cg_conn_read(int file_number, int B, int Z, int I, cgsize_t *pnts,
            CGNS_ENUMT(DataType_t) donor_datatype,
    	cgsize_t *donor_data);
    cg_conn_write(int file_number, int B, int Z,
    	const char * connectname, CGNS_ENUMT(GridLocation_t) location,
    	CGNS_ENUMT(GridConnectivityType_t) type,
    	CGNS_ENUMT(PointSetType_t) ptset_type,
    	cgsize_t npnts, const cgsize_t * pnts, const char * donorname,
    	CGNS_ENUMT(ZoneType_t) donor_zonetype,
    	CGNS_ENUMT(PointSetType_t) donor_ptset_type,
            CGNS_ENUMT(DataType_t) donor_datatype,
            cgsize_t ndata_donor, const cgsize_t *donor_data, int *I);
    cg_conn_write_short(int file_number, int B, int Z,
    	const char * connectname, CGNS_ENUMT(GridLocation_t) location,
    	CGNS_ENUMT(GridConnectivityType_t) type,
    	CGNS_ENUMT(PointSetType_t) ptset_type,
    	cgsize_t npnts, const cgsize_t * pnts, const char * donorname, int *I);
    cg_conn_read_short(int file_number, int B, int Z, int I,
    	cgsize_t *pnts);

    cg_1to1_read(int fn, int B, int Z, int I, char *connectname,
    	char *donorname, cgsize_t *range, cgsize_t *donor_range, int *transform);
    cg_1to1_write(int fn, int B, int Z, const char * connectname,
    	const char * donorname, const cgsize_t * range,
    	const cgsize_t * donor_range, const int * transform, int *I);
    cg_1to1_read_global(int fn, int B, char **connectname,
    	char **zonename, char **donorname, cgsize_t **range,
    	cgsize_t **donor_range, int **transform);

    cg_boco_info(int fn, int B, int Z, int BC, char *boconame,
    	CGNS_ENUMT(BCType_t) *bocotype, CGNS_ENUMT(PointSetType_t) *ptset_type,
     	cgsize_t *npnts, int *NormalIndex, cgsize_t *NormalListFlag,
     	CGNS_ENUMT(DataType_t) *NormalDataType, int *ndataset);
    cg_boco_read(int fn, int B, int Z, int BC, cgsize_t *pnts,
    	void *NormalList);
    cg_boco_write(int file_number, int B, int Z, const char * boconame,
    	CGNS_ENUMT(BCType_t) bocotype, CGNS_ENUMT(PointSetType_t) ptset_type,
    	cgsize_t npnts, const cgsize_t * pnts, int *BC);

    cg_array_info(int A, char *ArrayName,
    	CGNS_ENUMT(DataType_t) *DataType,
    	int *DataDimension, cgsize_t *DimensionVector);
    cg_array_write(const char * ArrayName,
    	CGNS_ENUMT(DataType_t) DataType, int DataDimension,
    	const cgsize_t * DimensionVector, const void * Data);

    cg_ptset_info(CGNS_ENUMT(PointSetType_t) *ptset_type,
    	cgsize_t *npnts);
    cg_ptset_write(CGNS_ENUMT(PointSetType_t) ptset_type,
    	cgsize_t npnts, const cgsize_t *pnts);
    cg_ptset_read(cgsize_t *pnts);

Note that, as in earlier versions of the library, the `donor_datatype` argument to `cg_conn_write` and `cg_conn_read` are redundant. Earlier versions required this value to be `Integer`. In 3.1 this is ignored altogether, and is taken to `Integer` in 32-bit mode and `LongInteger` in 64-bit mode. The corresponding `donor_data` is always of type `cgsize_t`.

A new convenience function has been added to the MLL:

    cg_index_dim(int file_number, int B, int Z, int *index_dim)

which gets the index dimension for the specified base and zone. The return value will be the Cell Dimension for a structured grid, and 1 for an unstructured grid.

### C Portability

If you use the `cgsize_t` data type in new code, it will work in both 32 and 64-bit compilation modes. In order to support CGNS versions prior to 3.1, you may also want to add something like this to your code:

    #if CGNS_VERSION < 3100
    #define cgsize_t int
    #endif

Existing code that uses `int` will not work with a CGNS 3.1 library compiled with the `enable-64bit` option turned on. You may want to add something like this to your code:

    #if CGNS_VERSION >= 3100 && CG_BUILD_64BIT
    #error does not work in 64 bit mode
    #endif

or modify your code to use `cgsize_t`.

### FORTRAN Interface

All integer arguments in the FORTRAN interface are taken to be `integer*4` in 32-bit mode and `integer*8` in 64-bit mode. If you have used default or implicit integers in your FORTRAN code, it should port to 64-bit mode in most cases by simply turning on your compiler option that promotes implicit integers to `integer*8`. However, be sure that you read the next section about issues. For example, all the FORTRAN code in the User's Guide examples ported unmodified when using the `-fdefault-integer-8` compiler option with `gfortran`. If you have explicitly defined your integers as `integer*4`, your code will not work in 64-bit mode. In that case, you will either need to change them to `integer` (recommended for portability) or `integer*8`.

The `cgnslib_f.h` header file has been modified to explicitly define all integer values based on the CGNS library compilation mode, `integer*8` when compiled in 64-bit mode and `integer*4` otherwise. A new integer parameter has also been added to the header, `CG_BUILD_64BIT`, which will be set to 1 in 64-bit mode and 0 otherwise. You may use this parameter to check if the CGNS library has been compiled in 64-bit mode or not, as in:

    if (CG_BUILD_64BIT .ne. 0) then
        print *,'will not work in 64-bit mode'
        stop
    endif

If you are using a CGNS library prior to version 3.1, this parameter will not be defined and you will need to rely on your compiler initializing all undefined values to 0 (not always the case) for this test to work.

### FORTRAN Issues and Portability

If your compiler supports automatic promotion of integers, and you use implicit integers, your code should port to 64-bit with the following exception.

If you use an `Integer` data type in any routine that takes a data type specification, and an implicit integer for the data, the code will fail when compiled in 64-bit mode with automatic integer promotion. An example of this would be:

    integer dim
    integer data(dim)
    call cg_array_write_f('array',Integer,1,dim,data)

This is because the MLL interprets the `Integer` data type as `integer*4` regardless of the compilation mode. The compiler, however, has automatically promoted `data` to be `integer*8`. What you will need to do to prevent this problem, is to either explicitly define `data` as in:

    integer dim
    integer*4 data(dim)
    call cg_array_write_f('array',Integer,1,dim,data)

or

    integer dim
    integer*8 data(dim)
    call cg_array_write_f('array',LongInteger,1,dim,data)

or test on `CG_BUILD_64BIT` as in:

    integer dim
    integer data(dim)
    if (CG_BUILD_64BIT .eq. 0) then
      call cg_array_write_f('array',Integer,1,dim,data)
    else
      call cg_array_write_f('array',LongInteger,1,dim,data)
    endif

The last 2 options will only work with CGNS Version 3.1, since `LongInteger` and `CG_BUILD_64BIT` are not defined in previous versions. Note also, that this does not apply to the `donor_datatype` argument to `cg_conn_write` and `cg_conn_read` as discussed above.

You may also need to be careful when using integer constants as arguments in 64-bit mode. If your compiler automatically promotes integer constants to `integer*8`, then there is no problem. This is probably the case if your compiler supports implicit integer promotion. If not, then the constants will be `integer*4`, and your code will not work in 64-bit mode. In that case you will need to do something like:

    integer*8 one,dim
    integer*4 data(dim)
    one = 1
    call cg_array_write_f('array',Integer,one,dim,data)

If you can't or don't want to use automatic integer promotion, but still want to be able to port to 64-bit code, you may include the preprocessor header file, `cgnstypes_f.h` in your code. In 64-bit mode, the `CG_BUILD_64BIT` define in the header will be set to 1, and `cgsize_t` and `CGSIZE_T` will be defined as `integer*8`, otherwise `CG_BUILD_64BIT` will be 0 and `cgsize_t` and `CGSIZE_T` will be defined as `integer*4`. By including this header file as a preprocessor include (i.e. `#include "cgnstypes_f.h"`) you may then use `cgsize_t` as variable type name. for example:

    #include "cgnstypes_f.h"
    cgsize_t cgfile,cgbase,celldim,physdim
    cgsize_t one,dim,ierr
    cgsize_t data(dim)

    call cg_open_f('file.cgns',CG_MODE_WRITE,cgfile,ierr)
    celldim = 3
    physdim = 3
    call cg_base_write_f(cgfile,'base',celldim,physdim,cgbase,ierr)
    ...
    one = 1
    #if CG_BUILD_64
      call cg_array_write_f('array',Integer,one,dim,data)
    #else
      call cg_array_write_f('array',LongInteger,one,dim,data)
    #endif

Notice, that you will not be able to use integer constants, but your FORTRAN code will be portable to 64-bit mode.

Lastly, you may use the header to simply limit the code to 32 or 64-bit modes, such as:

    #include "cgnstypes_f.h"
    #if CG_BUILD_64BIT
    # error can only be built in 32-bit mode
    #endif

### Examples

All the distributed C examples have been modified to use `cgsize_t`. and will work in both 32 and 64-bit modes. They are also fully compatible with previous CGNS versions.

#### C Test Program

There is a new C example in the `tests` subdirectory, `test64c.c`, that you may use for simple 64-bit mode testing. To compile it, do `make test64` from within the `tests` subdirectory. The program allows you to write any number of coordinates or tetrahedral elements using whatever amount of memory you specify. To get the list of available options, enter:

    test64c -help
    or
    test64c --

Without any options, the program will try to use 16 billion bytes of memory, and write 4 billion float coordinates and 500 million tetrahedra. The number of coordinates and tetrahedra default to the number that will fit within the available memory, in this case (16b / 4) and (16b / 32). To specify the amount of memory, use the `-m` option. For example, to use 4 billion bytes, run as

    test64c -m4b

In this case, the program will write 1 billion coordinates and 125 million tetrahedra, by default. I recommend that you use less than the full amount of available memory, otherwise swapping will result in your elapsed run time going from minutes to hours (or even a day).

To specify the number of coordinates or elements to write, use the `-c` and `-e` options. The number may be followed by a qualifier (`b` for billion, `m` for million, or `k` for thousand). For example, to write 10 billion coordinates and 400 million tetrahedra using 2 billion bytes of memory, use:

    test64c -m2b -c10b -e400m

If the memory required for either the coordinate or element data exceeds the memory available, the program will use partial writes to write the data. In the case above, the coordinates will be written 500m and the elements 62.5m at a time. A value of 0 may also be used for the number of coordinates or elements, in which case they are not written. Be aware, that 1 billion coordinates and elements require a liitle over 4 Gb and 32Gb of disk space, respectively.

To control the output file format, use `-adf` to force ADF and `-hdf5` for HDF5 output. The last option is `-test`, which will cause the file to be reopened in read-only mode and the data read and checked for accuracy.

#### FORTRAN

The examples `cgread.F` and `cgwrite.F` in the `tests` subdirectory have been modified to support 64-bit compilation. The `cgnstype_f.h` header was included in these, and the `cgsize_t` definition for variables was used as described above. This makes the code portable to 64-bit without requiring any additional compiler options.

Another example, `test64f.F` has been created in this directory. This program explicitly defines the integer variables as `integer*8` and then tests on the `CG_BUILD_64BIT` variable in order to prevent the program from running (and failing) unless built in 64-bit mode.

These FORTRAN examples do not really require 64 bits to run, they are provided to help with porting of your code. To build these, do `make fortran` from within the `tests` subdirectory.