Ocamlnet 4 Reference Manual : Netxdr

# Module Netxdr

    module Netxdr: sig .. end

External Data Representation

This module supports the "external data representation", or XDR for short. XDR is a means to pack structured values as strings and it serves to transport such values across character streams even between computers with different architectures.

XDR values must be formed according to an XDR type. Such types are usually written in a notation that is close to the C notation of structured types. There are some important details where XDR is better than C:

- direct support for strings
- arrays may have fixed or variable length
- unions must have a discriminator
- no pointers. The notation `*t` is allowed, but means something different, namely "t option" in O'Caml notation.
- recursive types are possible and behave like recursive types in O'Caml. For example,

           struct *list {
             int value;
             next list;
           }

  is a list of integer values and equivalent to

           type intlist = intlist_el option
            and intlist_el = { value : int; next : intlist }

See RFC 1014 for details about XDR.

This module defines:

- XDR types
- XDR type terms
- XDR type systems
- XDR type term systems

In "type terms" you can see the components from which the type has been formed, while a "type" is an opaque representation that has been checked and for that some preprocessing has been done.

A "type system" is a collection of several types that have names and that can refer to previously defined types (i.e. a sequence of "typedef"s). As with simple types, there is an extensive and an opaque representation.

A typical way of using this module is to define an "XDR type term system" by simply writing an O'Caml expression. After that, this system is validated and you get the "type system". From now on, you can refer to the types defined in the system by name and get the corresponding "XDR types". Once you have an XDR type you can use it to pack or unpack an XDR value.

Values possible for XDR types. This is straight-forward, except the "_fast" variants:

    type xdr_value =

`|` | `XV_int of `Netnumber.int4``
`|` | `XV_uint of `Netnumber.uint4``
`|` | `XV_hyper of `Netnumber.int8``
`|` | `XV_uhyper of `Netnumber.uint8``
`|` | `XV_enum of `string``
`|` | `XV_float of `Netnumber.fp4``
`|` | `XV_double of `Netnumber.fp8``
`|` | `XV_opaque of `string``
`|` | `XV_string of `string``
`|` | `XV_array of `xdr_value array``
`|` | `XV_struct of `(string * xdr_value) list``
`|` | `XV_union_over_int of `(Netnumber.int4 * xdr_value)``
`|` | `XV_union_over_uint of `(Netnumber.uint4 * xdr_value)``
`|` | `XV_union_over_enum of `(string * xdr_value)``
`|` | `XV_void`
`|` | `XV_enum_fast of `int`` | `(*` | The integer is the _position_ in the `X_enum` list, sorted by enum values (ascending). For example, if we have `X_enum [ "A", 4; "B", 2; "C", 6 ]` the element "B" has the position 0, because 2 is the lowest number | `*)`
`|` | `XV_struct_fast of `xdr_value array`` | `(*` | The array elements are in the same order as declared in `X_struct` | `*)`
`|` | `XV_union_over_enum_fast of `(int * xdr_value)`` | `(*` | The integer is the _position_ in the `X_enum` list. "position" means the same as for `XV_enum_fast` | `*)`
`|` | `XV_array_of_string_fast of `string array`` | `(*` | To be used with an `X_array` or `X_array_fixed` with an inner type of `X_string` | `*)`
`|` | `XV_mstring of `Netxdr_mstring.mstring``
`|` | `XV_direct of `exn * int * (exn -> xdr_value)``

    val xv_true : xdr_value

    val xv_false : xdr_value

See `x_bool`

    val xv_none : xdr_value

    val xv_some : xdr_value -> xdr_value

See `x_optional`

Terms that describe possible XDR types:

- `X_int`: integer (32 bit)
- `X_uint`: unsigned integer
- `X_hyper`: hyper (64 bit signed integer)
- `X_uhyper`: unsigned hyper
- `X_enum [x1,i1; ...]`: `enum { x1 = i1, ... }`
- `X_float`: float (32 bit fp number)
- `X_double`: double (64 bit fp number)
- `X_opaque_fixed n`: `opaque[n]`
- `X_opaque n`: `opaque<n>`
- `X_string n`: `string<n>`
- `X_mstring(name,n)`: `_managed string<n>` (see below)
- `X_array_fixed (t,n)`: `t[n]`
- `X_array (t,n)`: `t<n>`
- `X_struct [x1,t1;...]`: `struct { t1 x1; ...}`
- `X_union_over_int ([n1,t1;...], None)`: `union switch(int) {case n1: t1; ...}`
- `X_union_over_int ([n1,t1;...], Some t)`: `union switch(int) {case n1: t1; ...; default t}`
- `X_union_over_uint ([n1,t1;...], None)`: `union switch(unsigned int) {case n1: t1; ...}`
- `X_union_over_uint ([n1,t1;...], Some t)`: `union switch(unsigned int) {case n1: t1; ...; default t}`
- `X_union_over_enum (e, [n1,t1;...], None)`: `union switch(e) {case n1:t1; ...}` where e is an enumeration type
- `X_union_over_enum (e, [n1,t1;...], Some t)`: `union switch(e) {case n1:t1; ...; default t}` where e is an enumeration type
- `X_void`: void

The `X_type` constructor is only useful for types interpreted relative to a type system. Then it refers to a named type in this system.

The `X_param` constructor includes a reference to an arbitrary type which must first be given when packing or unpacking values. (A "lazy" type reference.) Additionally, the values for parameters may be encrypted or decrypted.

Example how to define a recursive type:

`X_rec ("a", X_array ( X_struct ["value", X_int; "next", X_refer "a"], 1))`

Managed strings are represented as `X_mstring(name,n)`. The `name` refers to the preferred factory for managed strings (needs to be passed to the XDR unpacker). Values for managed strings are objects of type `Netxdr_mstring.mstring`.

The term `X_direct(t,read,write,size,expand)` is generated by ocamlrpcgen at positions suitable for direct mapping. In this case, the XDR byte representation is directly mapped to the final Ocaml value bypassing the intermediate representation defined in this module. `t` is the mapped type. The function `read` is called as `read s cursor len` in order to map the XDR bytes at `!cursor` in `s`. The `cursor` must be advanced to the end position. `len` is the number of valid bytes in `s` (i.e. `s.(len-1)` is the last one). The result of `read` is an arbitrary exception which needs to be postprocessed by ocamlrpcgen-generated code. The function `write` is called as `write x s cursor`: The value encapsulated in exception `x` is written to string `s` at position `cursor`. Like for `read` the cursor is advanced by the number of written bytes. There must be enough space in `s`. The function `size` can be used to determine the number of written bytes beforehand.

    type xdr_type_term =

`|` | `X_int`
`|` | `X_uint`
`|` | `X_hyper`
`|` | `X_uhyper`
`|` | `X_enum of `(string * Netnumber.int4) list``
`|` | `X_float`
`|` | `X_double`
`|` | `X_opaque_fixed of `Netnumber.uint4``
`|` | `X_opaque of `Netnumber.uint4``
`|` | `X_string of `Netnumber.uint4``
`|` | `X_mstring of `string * Netnumber.uint4``
`|` | `X_array_fixed of `xdr_type_term * Netnumber.uint4``
`|` | `X_array of `xdr_type_term * Netnumber.uint4``
`|` | `X_struct of `(string * xdr_type_term) list``
`|` | `X_union_over_int of `(Netnumber.int4 * xdr_type_term) list * xdr_type_term option``
`|` | `X_union_over_uint of `(Netnumber.uint4 * xdr_type_term) list * xdr_type_term option``
`|` | `X_union_over_enum of `xdr_type_term * (string * xdr_type_term) list * xdr_type_term option``
`|` | `X_void`
`|` | `X_type of `string``
`|` | `X_param of `string``
`|` | `X_rec of `(string * xdr_type_term)``
`|` | `X_refer of `string``
`|` | `X_direct of `xdr_type_term * (Stdlib.Bytes.t -> int Stdlib.ref -> int -> exn) * (exn -> Stdlib.Bytes.t -> int Stdlib.ref -> unit) * (exn -> int) * (exn -> xdr_value)``

    type xdr_type

This is the validated version of `xdr_type_term`. Note that it does not contain `X_type` constructors, i.e. is completely expanded. It is allowed that an `xdr_type` contains `X_param` constructors (parameters). The set of occurring parameters can be determined very quickly for an `xdr_type`.

    type xdr_type_term_system = (string * xdr_type_term) list

Bind names to types. In a correct system you can only refer to previously defined types, i.e. the type called `n` must be defined in the list before it can be used via `X_type n`. It is possible to use this module without the means of type systems, but often code is more readable if types are defined in an environment allowing bindings to names.

    type xdr_type_system

A validated type system.

    val x_bool : xdr_type_term

Common abbreviation for boolean types. Has values `xv_true` and `xv_false`.

    val x_optional : xdr_type_term -> xdr_type_term

Common abbreviation for optional types. Has values `xv_none` and `xv_some v`.

    val x_opaque_max : xdr_type_term

Common abbreviation for opaque data of arbitrary length

    val x_string_max : xdr_type_term

Common abbreviation for strings of arbitrary length

    val x_mstring_max : string -> xdr_type_term

Common abbreviation for mstrings of arbitrary length

    val x_array_max : xdr_type_term -> xdr_type_term

Common abbreviation for arrays of arbitrary length

    exception Dest_failure

raised if the dest_* function are applied to non-matching xdr_value

    val dest_xv_int : xdr_value -> Netnumber.int4

    val dest_xv_uint : xdr_value -> Netnumber.uint4

    val dest_xv_hyper : xdr_value -> Netnumber.int8

    val dest_xv_uhyper : xdr_value -> Netnumber.uint8

    val dest_xv_enum : xdr_value -> string

    val dest_xv_enum_fast : xdr_value -> int

    val dest_xv_float : xdr_value -> Netnumber.fp4

    val dest_xv_double : xdr_value -> Netnumber.fp8

    val dest_xv_opaque : xdr_value -> string

    val dest_xv_string : xdr_value -> string

    val dest_xv_mstring : xdr_value -> Netxdr_mstring.mstring

    val dest_xv_array : xdr_value -> xdr_value array

    val dest_xv_array_of_string_fast : xdr_value -> string array

    val dest_xv_struct : xdr_value -> (string * xdr_value) list

    val dest_xv_struct_fast : xdr_value -> xdr_value array

    val dest_xv_union_over_int : xdr_value -> Netnumber.int4 * xdr_value

    val dest_xv_union_over_uint : xdr_value -> Netnumber.uint4 * xdr_value

    val dest_xv_union_over_enum : xdr_value -> string * xdr_value

    val dest_xv_union_over_enum_fast : xdr_value -> int * xdr_value

    val dest_xv_void : xdr_value -> unit

    val map_xv_enum_fast : xdr_type -> xdr_value -> int32

Works for both `XV_enum` and `XV_enum_fast`

    val map_xv_struct_fast : xdr_type -> xdr_value -> xdr_value array

Works for both `XV_struct` and `XV_struct_fast`

    val map_xv_union_over_enum_fast : xdr_type -> xdr_value -> int * int32 * xdr_value

Works for both `XV_union_over_enum` and `XV_union_over_enum_fast`. Returns the triple `(k,i,x)`:

- `k`: Position of the selected value in the `T_enum` array
- `i`: value of the enum
- `x`: selected arm of the union

    exception Xdr_format of string

Format error found during unpacking a string

    exception Xdr_format_message_too_long of xdr_value

The message is too long and cannot be packed into a string

    exception Xdr_failure of string

Usually a problem during packing

You must use these two functions to obtain validated types and type systems. They fail with "validate_xdr_type" resp. "validate_xdr_type_system" if the parameters are incorrect.

    val validate_xdr_type : xdr_type_term -> xdr_type

    val validate_xdr_type_system : xdr_type_term_system -> xdr_type_system

    val params : xdr_type -> string list

return the `X_param` parameters contained in the type

Get the unvalidated version back:

- Note that `X_type` constructions are always resolved

    val xdr_type_term : xdr_type -> xdr_type_term

    val xdr_type_term_system : xdr_type_system -> xdr_type_term_system

You can expand any type term relative to a (validated) type system. For example: `expanded_xdr_type sys1 (X_type "xy")` extracts the type called "xy" defined in sys1. Expansion removes all X_type constructions in a type term.

    val expanded_xdr_type : xdr_type_system -> xdr_type_term -> xdr_type

    val expanded_xdr_type_term : xdr_type_term_system -> xdr_type_term -> xdr_type_term

    val are_compatible : xdr_type -> xdr_type -> bool

`are_compatible`: currently not implemented

    val value_matches_type : xdr_value ->
           xdr_type -> (string * xdr_type) list -> bool

Is the value properly formed with respect to this type? The third argument of this function is a list of parameter instances. Note that all parameters must be instantiated to compare a value with a type and that the parameters instances are not allowed to have parameters themselves.

Encrypted parameters are not supported here.

### Packing and unpacking

`pack_xdr_value v t p print`: Serialize v into a string conforming to the XDR standard where v matches t. In p the parameter instances are given. All parameters must be given, and the parameters must not contain parameters themselves. The fourth argument, print, is a function which is evaluated for the pieces of the resultant string. You can use pack_xdr_value_as_string to get the whole string at once.

`unpack_xdr_value s t p`: Unserialize a string to a value matching t. If this operation fails you get an Xdr_format exception explaining what the reason for the failure is. Mostly the cause for failures is that t isn't the type of the value. Note that there are some implementation restrictions limiting the number of elements in array, strings and opaque fields. If you get such an error this normally still means that the value is not of the expected type, because these limits have no practical meaning (they are still higher than the usable address space).

Encryption: The `encode` and `decode` functions can be used to encrypt/decrypt parameters (placeholders in the type marked with `X_param pname` for a parameter name `pname`). The `encode` argument may list for each parameter an encoding function:

     encode = [ pname1, encoder1; pname2, encoder2; ... ]

The functions `encoder` are called with the XDR-packed parameter value, and return the encrypted value.

Likewise, the `decode` argument may list for each parameter a decoding function:

     decode = [ pname1, decoder1; pname2, decoder2; ... ]

The call style of the decoder functions is a bit more complicated, though. They are called as

     let (xdr_s, n) = decoder s pos len

meaning that the `decoder` starts decoding at position `pos` of string `s`, and that at most `len` bytes can be decoded. It returns the decoded string `xdr_s` (which is then unpacked), and in `n` the number of consumed input bytes is returned.

Exceptions raised in encoders or decoders fall through unmodified.

    type encoder = Netxdr_mstring.mstring list -> Netxdr_mstring.mstring list

see text above

    type decoder = Stdlib.Bytes.t -> int -> int -> Stdlib.Bytes.t * int

see text above

    val pack_xdr_value : ?encode:(string * encoder) list ->
           xdr_value ->
           xdr_type ->
           (string * xdr_type) list -> (Stdlib.Bytes.t -> unit) -> unit

    val pack_xdr_value_as_bytes : ?rm:bool ->
           ?encode:(string * encoder) list ->
           xdr_value ->
           xdr_type -> (string * xdr_type) list -> Stdlib.Bytes.t

    val pack_xdr_value_as_string : ?rm:bool ->
           ?encode:(string * encoder) list ->
           xdr_value ->
           xdr_type -> (string * xdr_type) list -> string

rm: If true, four null bytes are prepended to the string for the record mark

Changed in Ocamlnet-3.3: these functions raise `Xdr_failure` in case of errors.

    val pack_xdr_value_as_mstrings : ?rm:bool ->
           ?encode:(string * encoder) list ->
           xdr_value ->
           xdr_type ->
           (string * xdr_type) list -> Netxdr_mstring.mstring list

The concatanated mstrings are the packed representation

Changed in Ocamlnet-3.3: this function raises `Xdr_failure` in case of errors.

    type xdr_value_version = [ `Ocamlrpcgen | `V1 | `V2 | `V3 | `V4 ]

Selects which version of `xdr_value` is returned by `unpack_xdr_value`. During the development of Ocamlnet several incompatible changes were made, but by selecting a certain version these changes can be hidden from the caller.

- ``V1`: This version refers to the original `xdr_value` definition, which only included `XV_int`, `XV_uint`, `XV_hyper`, `XV_uhyper`, `XV_enum`, `XV_float`, `XV_double`, `XV_opaque`, `XV_string`, `XV_array`, `XV_struct`, `XV_union_over_int`, `XV_union_over_uint`, `XV_union_over_enum`, and `XV_void`.
- ``V2`: This version is available since the `rpc-0.4` distribution, and added the tags `XV_enum_fast`, `XV_struct_fast`, and `XV_union_over_enum_fast`.
- ``V3`: In Ocamlnet-3.0 the tag `XV_array_of_string_fast` was added.
- ``V4`: In Ocamlnet-3.5 the tag `XV_direct` was added.
- ``Ocamlrpcgen`: This refers to the version that must be used if the returned `xdr_value` is processed by code generated with `ocamlrpcgen`.

The default is still ``V1`, for ultimate backward compatibility. The switch `fast=true` selects ``Ocamlrpcgen` (it was always used for this purpose, despite its name).

    val unpack_xdr_value : ?pos:int ->
           ?len:int ->
           ?fast:bool ->
           ?prefix:bool ->
           ?mstring_factories:Netxdr_mstring.named_mstring_factories ->
           ?xv_version:xdr_value_version ->
           ?decode:(string * decoder) list ->
           Stdlib.Bytes.t ->
           xdr_type -> (string * xdr_type) list -> xdr_value

    val unpack_xdr_value_l : ?pos:int ->
           ?len:int ->
           ?fast:bool ->
           ?prefix:bool ->
           ?mstring_factories:Netxdr_mstring.named_mstring_factories ->
           ?xv_version:xdr_value_version ->
           ?decode:(string * decoder) list ->
           Stdlib.Bytes.t ->
           xdr_type -> (string * xdr_type) list -> xdr_value * int

`prefix`: whether it is ok that the string is longer than the message (default: false)

`mstring_factories`: when a `T_mstring(name,_)` type is found, the factory is looked up in this hash table under `name`, and if this fails under the name `"*"`. If there is no such factory, unpacking fails! (Default: empty table.)

`xv_version`: Selects a certain version of the returned `xdr_value` terms. See `Netxdr.xdr_value_version` for details. Set this to ``Ocamlrpcgen` if you decode the `xdr_value` with ocamlrpcgen-generated decoders.

`fast`: Setting this to true is a deprecated way to set `xv_version=`Ocamlrpcgen`.

The variant `unpack_xdr_value_l` returns not only the decoded value, but also the actual length in bytes.

The exceptions `Xdr_format` and `Xdr_format_message_too_long` may be raised.

    val unpack_xdr_value_str : ?pos:int ->
           ?len:int ->
           ?fast:bool ->
           ?prefix:bool ->
           ?mstring_factories:Netxdr_mstring.named_mstring_factories ->
           ?xv_version:xdr_value_version ->
           ?decode:(string * decoder) list ->
           string ->
           xdr_type -> (string * xdr_type) list -> xdr_value * int

Same for strings