Fermi Physics Class Libraries Package: CLHEP-Random

# ZOOM ZOOM

# ZOOM Validated CLHEP Random Package

HEP Random is a collection of random number engines and a variety of distributions which use those engines to generate pseudo-random numbers.

The ZOOM validation consists of

- Verifying correct statistical properties of the engines and distributions
- Testing the original code and correcting minor bugs
- Providing a few additional methods requested by the Run II user community
- Completing the set of distributions, to include the full list Review of Particle Properties.

The source code for headers and implementations of the ZOOM version of CLHEP Random may be browsed here .

The extensions, corrections and validations on this page are completed and reflected in the Random sub-package of the ZOOM distribution of CLHEP.

### What is new:

10/23/98
New Distribution: RandMultiGauss Multivariate Gaussian Distribution, as per page 170 of the new Review of Particle Physics. Returns a HepVector of random variates according to a given HepVector means mu and HepSymMatrix covariance about the mean S.

10/23/98
Tests of Distributions:

testRandDists has Tests based on generating a large number of variates and verifying that known properties of each distribution hold. Distributions tested thus far are:

RandGauss
  Mean and first four moments.
  Counts at 1, 2, 3, 4, and 5+ sigma.
RandMultiGauss
  Mean.
  Covariance matrix.

9/17/98
Cleanup of the new random engines:

Ranlux64
  The earlier CLHEP implementation of this did not accurately follow Luscher's prescription. Though it passed all the usual tests, it would likely have a shorter period that the correct algorithm. The implementation has been corrected to match Luscher's ranlxd.
Hurd288Engine, Hurd160Engine, TripleRand, Mtwist
  These have had several cleanups associated with the translation from integer bit streams to doubles.
(float) and (unsigned int)
  Methods returning random variables of single precision are included. As compared with flat(), for engines which take the trouble to ensure randomness out to 53 bits of mantissa, (float) and (unsigned int) run faster by generating fewer random bits.

8/25/98
Several new random engines have been added:

TripleRand
  is a combination of the DualRand and the Hurd288 engines, with a period of over 2**400. This is intended as a small-state (60 bytes) reasonably fast "mother of all generators" for applications that are fairly paranoid.
Hurd288Engine
Hurd160ENgine
  are a pair of linearly interconnected shift register engines, based on the paper by William Hurd (not to be confused with GNU Hurd) in IEEE Transactions on Computers, C23 No. 2 page 146. This method is claimed to remove some unsatisfactory properties in the Tausworthe methods, stemming from the latter using trinomial primitive polynomials. We implement 160 and 288 bit instances, based on lines 9 and 10 of table I in that paper.

Older "Whats New" Items

### Testing for Proper Behavior of CLHEP Random sub-package

- Random Engine Validation
  We have applied the "DIEHARD" suite of statistical tests for uniformly distributed random numbers to each of the engines in CLHEP Random.
  Results show that several engines pass the full suite of tests:
  - DualRand
  - Hurd288Engine
  - MTwistEngine
  - RanecuEngine
  - RanluxEngine on luxury level 2 or higher
  - Ranlux64Engine
  - RanshiEngine
  - TripleRand
  Other engines fail one or more tests of randomness; in particular RandEngine and RanluxEngine on level zero fail several significant tests and should not be considered reliably random. RanshiEngine and Hurd288Engine are about as fast as these (and much faster for 32-bit interger generation) and should be considered as substitutes.

- Distribution Generator Validation
Validation of proper behavior of each distribution is mathematically subtle in some cases. This validation is has been done for:

- RandGauss
- RandMultiGauss

Run testRandDists to execute these tests.

### ZOOM-supplied Extensions

A couple of extensions to the basic interface are useful to our community. These are in the area of internally saving and restoring state, and support for large collections of generators without getting identical sequences.

(In no case will these extensions modify or eliminate existing correct functionality.
Any bug fixes which may change behavior will be indicated explicitly.)
Changes (fed back to CLHEP) are discussed here.

- Additional Constructors
  - The default constructors for random engines tracks starting seeds, so that multiple instances of engine instantiated by default constructors give different starting points in the random sequence. The following code will work and give to each distribution an independent engines with a unique starting seed:
    `RandGauss g1 (JamesRandom j1);
    RandGauss g2 (JamesRandom j2);
    RandGauss g3 (JamesRandom j3);`

  - The constructors for all engines now follow the same pattern. Along with the default constructor and a copy consturctor to duplicate the state,
    - Each engine has a constructor taking a single long argument. This exhibits the identical behavior to that constructor in CLHEP; in new engines the long is used in a direct way to form the state.
    - Each egine has a constructor taking two int arguments. For engines in CLHEP that already had such a constructor (top access a seed table) the behavior is left intact. For all other engines, this constructor establishes the starting state based on those ints.
    In generators with 64-bits of state or more, the states which can be established by default constructor, by supplying one long, and by supplying two ints are all distinct from one another.

  - Each distribution class has a default constructor, which instantiates its own (default) instance of RanLuxEngine. The following code would give an array of standard-parameter Gaussian distributions, with independent engines with different starting seeds for each distribution:
    `ZMRandGauss gs [100];`

  - Distributions have a form of constructor that accepts a set a parameters to use as defaults for that distribution instance, when fire() is done without supplying parameters.
    `ZMRandGauss g(5.0,2.5);
    double x = g.fire();
    // x is distrbuted with mean = 5, sigma = 2.5
    double y = g.fire(3, 2);
    // y is distrbuted with mean = 3, sigma = 2
    `
    Of course, if parameters are explicitly supplied to an invocation of fire(), those supplied parameters are used. And if no parameters are provided when constructing the distribution instance, the standard values are used as before.
    `ZMRandGauss g(e, 0.0, 1.0);
    // same as:
    ZMRandGauss g(e);`
- Additional Methods
  - Saving and restoring engine state to file with user-supplied name.
    `RanluxEngine e;
    e.saveStatus ( "myFile.conf" );
    e.restoreStatus ( "yourFile.conf" );`

  - Engine state stream output via << and input via >> . There are also constructors accepting an istream to determine the state.
    `RanluxEngine e;
    cin >> e;
    cout << e;
    RanLuxEngine e2 ( cin );
    `

  - The flat() methods in all added engines supply double precision numbers random out to the last bit. If the underlying engine works with 32-bit numbers, this requires a bit of extra work. We will provide conversion constructors for each engine, which can be used as in:

        DualRand e;
        double x; float f; unsigned int u;
        x = double(e);        // Exactly equivalent to x = e.flat();
        f = float(e);         // Equivalent to f = e.flat();
                              // but somewhat faster
        u = unsigned int(e);  // Equivalient to u=pow(2.0, 32)*e.flat()
                              // but supplies the 32-bits obtained from
                              // the algorithm kernel, without the "middleman"

    A shorter syntax will also work, but is disparaged because it looks too mysterious:

        x = e;                // Will come out x = double(e);
        f = e;                // Will come out f = float(e);
        u = e;                // Will come out f = unsigned int(e);

    For consistency, the double, float and unsigned int operators are also be provided for the original engines, but these save no time.

- Additional Distributions

  The package now include the following distributions:

  - All eight distributions in table 28.1 of the Review of Particle Properties. Of these, Uniform, Poisson, and Gaussian were already present in HEP Random. The ZOOM work adds:
    1. Binomial -- RandBinomial
    2. Chi-squared -- RandChiSquare
    3. Student's t -- RandStudentT
    4. Gamma -- RandGamma
    5. Multivariate Gaussian -- RandMultiGauss
  - The two special cases of these (Exponential and Breit-Wigner) provided in the original CLHEP Random.

- Additional Random Engines
  DualRand
    A new random engine, implementing one which is in wide use in the Lattice Gauge Monte Carlo community. It is a combination of a (127,97) Tausworthe shift register generator and ordinary 32-bit integer congruence. This passes the statistical tests in the Diehard suite, and is rather fast. It also has the property that two instances based on different seeds will give truly different streams, and not two separated segments of the same long stream.
  Ranshi
    A huge-state (2K bytes) generator based on F. Gutbrod's method of that same name. It repeatedly simulates throwing spinning balls onto a large collection of other spinning balls. One instance of Ranshi uses as a state one 32-bit "red ball" and 512 32-bit "black ball" spins. Techniques to avoid "trap" sequences are employed. Though the properties of this process have not been studied with any rigor, the huge state lends confidence in a good random sequence of long period. Despite the huge state, the time taken to generate a single random is excellent.
  Ranlux64
    A 64-bit variant of the Ranlux engine, again from Europe. This (along with all the ZOOM-supplied additional engines) supplies doubles which are random out to all 53 bits of mantissa, rather than just 32 as the original Ranlux does.
  MTwist
    The Mersenne Twister engine. This is a huge-state (~2.5 Kbytes) engine which has mathematically proven outstanding randomness properties and a period of 2**19,937 -1. Our implementation is from the algorithm in the paper; there is a culture on the web page of speed improvements but it is rather fast as it is, and we wanted as few risks of incorrectness as possible. This is probably the best huge-state engine available.
  TripleRand
    is a combination of the DualRand and the Hurd288 engines, with a period of over 2**400. This is intended as a small-state (60 bytes) reasonably fast "mother of all generators" for applications that are fairly paranoid. The Hurd288 engine implements a set of linearly interconnected shift register engines, based on the paper by William Hurd (not to be confused with GNU Hurd) in IEEE Transactions on Computers, C23 No. 2 page 146. This method is claimed to remove some unsatisfactory properties in the Tausworthe methods, stemming from the latter using trinomial primitive polynomials. We implement 160 and 288 bit instances, based on lines 9 and 10 of table I in that paper.

  ### Corrections

  Several minor changes were made in the behavior of these classes, either to conform to the clearly intended behavior, or to conform to normal semantics a user would expect in C++:
  - When engines were constructed, they were supposed to be given zero-terminated arrays of seed numbers as arguments. They were not, and in the case of RanLuxEngine this made a difference. In principle, this could lead to access violations; in typical use, the only effect would be that two identically seeded distributions would after a short while begin to diverge. This bug has been fixed, for all the distribution classes.

  - The previous behavior of default constructors of random engines was such that if the user instantiated two intances of the same engine class using the default constructor each time, the sequences obtained would be identical. This is very trappy, and contradicts the most probable purpose of using this syntax. We now provide a different starting state each time. The user can get two matching sequences by explicitly supplying a specific seed, or by copying the engine object itself.

  ## Validation and Extensions in Progress

- Additional Methods

  - Default constructors will be provided for each of the distributions. These will conform to the following semantics: When a distribution is instantiated without specifying a specific engine, a new instance of a good random engine will be constructed, and will be owned only by the new distribution instance.

  - Copy constructors and assignment operators will be provided for each distribution. The semantics for what engine is used when copying a distribution are as yet undecided. The three possibilities are:
    1. A clone of the same class of engine as used in the original distribution is instantiated and associated with the copy distribution. Thus the original and the copy distributions will have two completely independent streams.
    2. The copy distribution will share the same engine as the original. Thus firing either will advance the state of the other.
    3. A copy of the engine associated with the original will be made and associated with the copy distribution. Thus the distribution and its copy will fire identical streams of randoms.
    We are leaning toward the first of these possibilities. we welcome comments as to which choice people feel is most useful.

- Additional Distributions
  - Distributions for fluctuation of ionization energy loss:
    1. Landau distribution.
    2. Vavilov distribution, an approximation-free version of the Landau distribution with energy cutoff.
       This version will be quicker than the CERNLIB versions of Vavilov.
    3. Vavilov distribution corrected for spin-1/2 particles.

- Additional Engines
  RanUniEngine
    Another engine in use in the Lattice Gauge community. This comes from Marsaglia, and appears to be a lagged addition Fibonacci, added mod 1 to a simple c*d mod m. Since the set of other engines do not include a lagged Fibonacci, we will likely incorporate this one.

  The following engines are also logical candidates for inclusion but we presently do not plan to provide them:

    Ran2 from Numerical recipes, which is related to Ranecu.
    Ran4 from Numerical recipes, which is related to the DES encryption standard.
    GFSR, a 4-tap feedback shift register discussed by R. Ziff in Computers in Physics, July 1998.

- Validation
  There have been problems in the past understanding the results of the DIEHARD tests.

  There is work in progress to:

  - put this Marsaglia suite into clear C++;
  - correct a suspected flaw in the Minimum Distance test, possibly introduced in the course of going from the original Fortran through f2c;
  - add a couple of other tests which would have been impractical before the available computing power was at today's level.

  ZOOM Home Page - Fermilab at Work - Fermilab Home

  Last modified: October 23, 1998