OBuild Design
=============

Genesis
-------

Obuild started on a bank holiday after xmas, as an experiment to make the
simplest OCaml build system. The mains goals are to:

* provide a good user experience.
* provide a building black box, no mocking around generic rules.
* provide features in the highest level possible.
* the cleanest build possible, with by-products hidden from the user.
* provide good defaults, and standardize names as much as possible.
* expose everything that the user of the build system needs in one place.
* be simple to build to prevent any bootstrapping problem.

One of the main influences was Haskell Cabal, which provides to all Haskellers
a simple way to provide a build system to a project with a single file. This
applies well for the myriad of OCaml options too.

Simple to build
---------------

Obuild is buildable with just the compiler and the compiler standard library.
This make bootstrapping very easy: all you need is the OCaml compiler installed.

This creates some pain for developers of obuild, as lots of basic functions
available in others libraries need to written again as part of obuild.  As the
initial development was done really quickly, some functions are not as
performant (CPU or memory-wise) as they could be. This can be fixed as problem
becomes apparent in scaling.

Simple to use
-------------

Each project is described really simply in a one place, in a user friendly format.
a central .obuild file is used, and provide high level description of your project.
Along with some meta data (name, authors, description, etc), it defines the library,
and\/or executable that the project want to have, from which inputs (source
files, modules).

All dependencies is fully autogenerated internally and used to recompile only
the necessary bits.

Along with library and executable, test and benchmark can be defined, so as to
provide a easy way to test or bench some part of your project. It also provides
a standard on how to build and execute tests and benchmarks. Later on,
because of this integration it could provide way to make nice reports (html/javascript)
based on output of benchs and tests.

Standardizing documentation
---------------------------

Providing a standard to generate and install documentation, is one of the sub goals of
obuild. This will go a long way to provide documentation in a centralized place for all
libraries in the ocaml world.

Internal META parsing
---------------------

ocamlfind is the current de-facto standard for the installed package querying.
ocamlfind is usually injected on the command line to ocamlopt,ocamldep,ocamlc
which special flags (-syntax, -package), that ocamlfind will re-write to call
the program with something that the program can understand. All the informations
for this transformation is stored in META files.

Unfortunately this design prevent META caching, and each time ocamlc/ocamlopt
is used it will reparse the META files. This also causes problem if ocamlfind
does not exists when used as a program, or if the library is not installed when
used as a library.

Because of those 2 reasons, obuild got a tiny (0.4 KLoC) rewrite of the
necessary part of findlib (3 KLoC). It is not as generic as the original
library.

Internal Design
---------------

By being a single program that knows about how things are supposed to be built,
obuild is in the unique position of caching more and going faster.

At the moment, there is lots of parsing redundancy for example:

* a file using campl4, result in camlp4 being called at minimum 2 times as a preprocessor:
  one for running ocamldep, and one for running ocamlc. Each time camlp4 is called,
  it result in the camlp4 grammar files being reloaded, and camlp4 will output the
  same things again. It get worse if you are also compiling other version.
  (e.g. a native version, a bytecode version with debug, a native version with
  debug, a native version with profiling).  This lead to camlp4 called up to 7
  times, to produce the exact same thing.

* without a .mli file, bytecode and native version will also cause ocamlc and ocamlopt
  to parse an .ml file twice. This could be made better. 
 
If the ocaml compiler add support for providing an already parsed .ml
file to ocamlc or ocamlopt, obuild could take advantage of this really easily.

Also provided compilation re-entrancy and ability to use the ocaml compiler as
a library, obuild could use the compiler as a library.

Librification
-------------

Obuild has been designed to be used as a library eventually.
The code is shifting towards using pure structure and functions, so
that things can be reused. there is some global state, that will be
eventually reduced to provide better control of each parts.

One of the possible development of this, would be to provide an optional daemon
that monitor file changes, and automatically rebuild on demand without having
to re-analyze the whole project.

Some other possible scenario is to have other programs use the project file
format, either to provide tools to write them or tools that read them.
