<This is the content of a letter sent in response to a question on the
 differences between p4 and pvm>

There are lots of differences between p4 and pvm.  Let me summarize and
then go into more details.

pvm is small and provides a simple way to pack several different types
(integers and doubles, for example) into a single message.  It is optimized
for a heterogenous environment.

p4 is larger than pvm and provides sends and receives, as well as a variety
of shared memory operations.

There are a number of technical details that you should know.  p4's  
messaging model is implemented with a variety of services; for example, 
on an Intel Touchstone, fast Intel-specific routines are used.  pvm's
implementations (to date) are all based on sockets.

PVM's implementation requires that data be copied one more time than 
p4's (it has to be staged until the message buffer is flushed).  pvm also
uses a daemon on each processor, so that a message passes through 2 
intermediate processes before reaching the destination process.  p4 sends
messages directly from sender to receiver; connections are opened as
needed.  In my tests on 2 SPARC IPX's, p4 was roughly twice as fast as
pvm.

Finally, there is the issue of ease-of-use.  My model of a parallel 
job is

    foo -np 4 <other arguments>

where a sequential job is

   foo <other arguments>

Neither p4 nor pvm make this simple, but it is possible to support in p4
(I have code that you may be interested in).  Under pvm, there is currently
no way to interrogate the pvm daemon, making it impossible to just
connect to the daemon and run.  The pvm model seems to be

    generate a file of hosts
    start up a pvmd
    Move your program (or create a link) to where pvm expects to find the
       executable (p4 doesn't require the executable to be in any 
       particular place).
    Start your job (note that pvm doesn't propagate the enviroment, such
       as command line arguments, working directory, and the like)
    (probably) kill the pvm daemon so that you can use pvm later.

You CAN avoid the steps of setting up and killing the pvm daemon ONLY if
you can use all of the machines that may be in the pvm set.  For example,
if I say that you can use my workstation only during certain times of day,
or only for jobs that are small and short, you'd need to create a special
pvm daemon for each such use.  Thus, you can only create a pvm daemon and
let it sit around if it is always acceptable to run jobs on any of the
processors the daemon knows about.  There has been some talk of using a
load-average monitor to improve this, but our experiance has been that
that is not sophisticated enough.

An advantage of pvm is that there is a large, well funded group working on
it.  It is possible that they will fix some of these problems, though
others are intrinsic in the design of the system.  One possibility is 
to use a system that allows you to switch between p4, pvm, and others;
I've written such a system (which also handles the initial interface,
and the in the pvm case, handles propagating the environment to the
other workers in a parallel application).  Let me know if you would like
more information.
