* Parallel copy-on-write streaming (PCOWS)

Copy-on-write can be offered by an operating system to provide
efficient parallelisation for streaming operations typical in biology
where chunks of data can be processed independently from the same
starting point. 

PCOWS compares to a regular thread pool with the difference that data
is not communicated through messages or pipes, but simply through
memory. The only catch is that PCOWS threads can not easily communicate
back. The Ruby parallel gem communicates back through pipes but that
means the main thread can not proceed until all the pipes complete.

PCOWS was first explored in the bio-ruby VCF tool (bio-vcf 0.7 series)
and then replaced by the parallel gem (bio-vcf 0.8
series). Performance, unfortunately, deteriorated to the extent that
PCOWS got retrofitted.

PCOWS basically reads a file and chunks it on the main thread. Every
chunk gets fed to a copy-on-write version of the Ruby interpreter
which contains the full state up to the point of forking - this is an
inexpensive procedure. Each chunk gets processed to a file. The main
thread tracks these files and issues a separate thread for harvesting
the ordered files so each gets piped to STDOUT in order.

The reader thread tracks This means the reader thread is only held up
by the number of allowed forks running at a time.

Based on this description the interface can be expressed as a reader
creating chunks of state that get passed on to a chunk processor named
'run' which gets a callback 'func'. Every time a chunk gets processed
the reader checks the size of the thread pool and also checks whether
output has become available.

The threads communicate simply through a file. Each thread writes to
STDOUT which has been redirected to a temporary file with extension
'.part'.  When the thread is complete, the file gets renamed by
removing the '.part'. Files that lack the '.part' get printed to the
real STDOUT and deleted on completion. I.e.:

1. STDOUT for a worker is redirected to a tempfile with extension '.part'
2. Worker thread writes to STDOUT
3. On completion '.part' gets removed from the filename
4. The main thread checks the next file to be printed based on filenames
5. A print thread gets invoked
6. The print thread removes the file

The only communication between PCOW threads and the main running
thread is therefore the renaming of the temporary file. This will also
work on network mounted systems. For performance it can make a
difference to use a fast disk for temporary files, though the Linux
file caching will make it likely that buffers are still in RAM.

