
this is a first cut at a caching, swapping, shared COW memory manager
for gimp16.  i would like this to be integrated onto the trunk at some
point soon after the 1.2 release.

=====================================================================
                      Copy On Write Shared Memory
=====================================================================

Smart Pointers
--------------

to do COW shared memory, we use smart pointers for the image data.
these are small structures that encapsulate the actual data pointer.
user code obtains and keeps pointers to these structures rather than
the data itself.

a smart pointer never moves after it's been allocated.  however, the
internal variables, including the wrapped data pointer, may change at
any time.  in particular, the piece of memory it points to, or the
location of that memory, may change over time.  always use the macro
wrappers when dealing with a smart pointer.

the smart pointer object has a set of routines which implement a
lifecycle for the data.  this lifecycle is a small state machine.



Lifecycle States
----------------

 UNINIT
  - the smart pointer has not been constructed

 UNALLOC
  - the smart pointer itself is inited, but the encapsulated data
    pointer does not refer to any storage

 SHARABLE
  - the smart pointer refers to storage and will allow other smart
    pointers to refer to it as well

 UNSHARABLE
  - the smart pointer refers to storage but will not allow other smart
    pointers to refer to it.

 SHARED
  - the smart pointer refers to storage which other smart pointers
    also refer to.



Lifecycle State Transitions
---------------------------

UNINIT     -> UNALLOC
  d_init created an SP

UNALLOC    -> UNINIT
  d_uninit destroyed an SP

UNALLOC    -> SHARABLE
  d_alloc reserved storage for the SP

SHARABLE   -> UNALLOC
  d_unalloc killed storage for an SP

SHARABLE   -> SHARABLE
  d_read used an unused/readonly SP
  d_release let go of a readonly SP
  d_split tried to split an unused/readonly SP

SHARABLE   -> UNSHARABLE
  d_write used an unused/readonly SP

SHARABLE   -> SHARED
  d_join added an SP to an unused/readonly SP

UNSHARABLE -> UNSHARABLE
  d_read used a readwrite SP
  d_write used a readwrite SP
  d_release let go of a readwrite SP (not the last use though)
  d_join made a copy of us for the other SP and left us alone
  d_split tried to split a readwrite SP

UNSHARABLE -> SHARABLE
  d_release let go of a readwrite SP (the last use)

SHARED     -> SHARABLE
  d_write made a copy and detached one of our SP, we had 1 left
  d_split made a copy and detached one of our SP, we had 1 left

SHARED     -> SHARED
  d_read used an unused/readonly SP
  d_write made a copy and detached one of our SP, we had 2+ left
  d_release let go of a readonly SP
  d_join added an SP to an unused/readonly SP
  d_split made a copy and detached one of our SP, we had 2+ left

(new)      -> SHARABLE
  d_join made a copy of a readwrite SP and made it sharable
  d_split made a copy of an unused/readonly SP and made it sharable

(new)      -> UNSHARABLE
  d_write made a copy and detached us from our shared memory




Smart Pointer Routines w/ State Transitions
-------------------------------------------

d_init         UNINIT     -> UNALLOC

d_uninit       UNALLOC    -> UNINIT [1]

d_alloc        UNALLOC    -> SHARABLE

d_unalloc      SHARABLE   -> UNALLOC

d_read         SHARABLE   -> SHARABLE
               UNSHARABLE -> UNSHARABLE
               SHARED     -> SHARED

d_write        SHARABLE   -> UNSHARABLE
               UNSHARABLE -> UNSHARABLE
               SHARED     -> SHARED + UNSHARABLE [2]
               SHARED     -> SHARABLE + UNSHARABLE [2]

d_release      SHARABLE   -> SHARABLE
               UNSHARABLE -> UNSHARABLE
               UNSHARABLE -> SHARABLE [3]
               SHARED     -> SHARED

d_join         SHARABLE   -> SHARED
               SHARED     -> SHARED
               UNSHARABLE -> UNSHARABLE + SHARABLE [4]

d_split        SHARABLE   -> SHARABLE
               UNSHARABLE -> UNSHARABLE
               SHARED     -> SHARED + SHARABLE
               SHARED     -> SHARABLE + SHARABLE [5]

[1] can be called from any state, but bad things may happen

[2] does a copy-on-write, new copy is unsharable, old one is
    left unmodified, unless it now has only one smart pointer.
    then it's made sharable instead of shared

[3] the last release of unsharable leaves it sharable

[4] joining to unsharable will make a new sharable copy, old one is
    left unmodified

[5] splitting a shared memory always makes a new sharable copy.  if
    the old one now has only a single smart pointer, it reverts to
    sharable as well.



=====================================================================
                        MemMgr and ShmMgr
=====================================================================

MemMgr does the alloc, free, swapping and caching of the memory
referred to by the smart pointers.  there will be interfaces to MemMgr
to control the swapping and caching behavior.

ShmMgr manages OS shared memory segments.  each segment will have one
or more ShmPtr created for it which define regions inside the segment.
the intent is to create a GimpShmBuffer subclass of GimpBuffer that
will construct itself around a ShmPtr and manage the image data
contained within.


=====================================================================
                        Image Buffers
=====================================================================

the image buffer code uses a small and specialized inheritance
mechanism.  it's basically a stripped down version of the one
used by the gtkobject system.

image buffers manage a three dimensional block of memory.  the block
is measured in (pixels x pixels x bytes).

the z dimension is described by a Tag.  every image buffer has a Tag
assigned at creation which never changes.

the x and y dimensions are each described by a pair of integers,
'period' and 'phase'.  these numbers describe the tiling of the memory
in the image.  period is the tile size in that dimension, and phase is
the offset into the first tile where the image begins (ie: partial
left/top tiles).  there may also be partial right/bottom tiles, do the
math on width and height to find out.  partial tiles are always fully
allocated.  ie: all tiles in a buffer have memory chunks which are
identical in size.

since the image tiling structure is fairly arbitrary, a method called
'focus' is used.  this takes a region-of-interest on the image and
focusses on it.  ie: it calculates the actual contiguous area in the
upper left corner of the ROI and returns the boundaries.  the lower
right parts of the roi may be clipped if they rest on another chunk of
memory.  the info is passed back in a GimpPortion structure which is
the primary means of communication with the other routines.

the memory in each tile has a lifecycle which is a superset of the
underlying copy-on-write memory lifecycle.  in particular, the
read/write/release group has a new member, update.  this is basically
just a read, except you want to modify the memory.  useful to update a
shared memory chunk if it gets corrupted.

another addition is the concept of a 'valid' state.  this is
introduced between the alloc and use of the memory.  by default,
memory is validated by memsetting it to zero.  interfaces will exist
in the subclasses to register a custom validator.

the alloc/validate/use cycle is chained together internally so that
using an unallocated tile will automatically alloc and validate it.
these automatic actions can be suppressed if required.

interfaces will need to be written to exploit the COW abilities of the
memory manager.  these will reside in the subclasses.



=====================================================================
                Comparison to Existing Image Buffers
=====================================================================

this section describes the existing (Dec 14 1998) gimp 1.1 image
memory management as i see it.  the existing interfaces are grouped
and described according to the design principles of the new package.
therefore, the various tile_* header files are thoroughly mixed
together.



Public Interfaces
-----------------

these routines are used throughout gimp.  the new design must provide
public interfaces with equivalent functionality.  these routines are
grouped here according to the design of the new system.


  System Initialization
  ---------------------

    tile_swap_add
    tile_swap_exit
      tile swap has not yet been implemented in the new design.  these
      interfaces will reside in the MemMgr class.



  Image Creation/Destruction/Attributes
  -------------------------------------

    tile_manager_new
    tile_manager_destroy
    tile_manager_level_width
    tile_manager_level_height
    tile_manager_level_bpp
      a tile_manager_new equivalent is provided by each buffer
      subclass.  the others routines apply to all buffers and are
      present in buffer.h as gimp_buffer_{delete,width,height,depth}.



  Image Data Allocation/Initialization/Accessing
  ----------------------------------------------

      all these routines relate to the image memory lifecycle.  see
      the section on that for a description of the new functionality.

    tile_manager_get_tile
      this is the gimp_buffer_use method with the associated autoalloc
      and autovalidate.  after the use succeeds, do a gimp_buffer_data
      to get the dimensions of the tile you just used.

    tile_release
      this is a combination of use, validate and alloc (with the
      release, invalidate and unalloc args)  all must be done
      separately in the new system.

      the idea of a separate refcount on the tile which destroys the
      tile when it reaches zero now lives in the bowels of the MemMgr.
      image data is freed by the MemMgr when no MemPtr (and hence no
      Tile or anyone else for that matter) refers to it.  the tile
      itself is never shared by anyone, just the image data.

    tile_invalidate_tile
      this is a combination of invalidate and unalloc.  the two are
      separate in the new scheme.

    tile_is_valid
      replaced by gimp_buffer_query

    tile_mark_valid
      what good is this routine?

    tile_manager_get_tile_coordinates
      tile_manager_get_tile_coordinates exists because the
      initialization interface for tilemanagers uses tiles, not
      coordinates.  image buffers do everything in terms of
      GimpPortion which is in pixels already.

    tile_manager_set_validate_proc
    tile_manager_set_user_data
    tile_manager_get_user_data
      these will be in the subclass specific routines which register a
      custom validator.
      
    tile_manager_get_async
      i haven't thought about this yet.

    tile_manager_map_tile
    tile_manager_map_over_tile
      GimpTileBuffer will add methods to replace the map routines.
      it's unlikely there is any benefit in using COW for the
      GimpFlatBuffer class.  perhaps for the brush/pattern dialogs?
      in any event, the COW interface exported by a tiled buffer will
      probably be different than that for a non-tiled buffer.  thus
      these routines belong in the subclasses, not the base class.
      obviously you won't be mapping over GimpShmBuffers.


  Image Data Usage
  ----------------

    tile_data_pointer
    tile_ewidth
    tile_eheight
    tile_bpp
    tile_size
       these routines are replaced by the GimpPixelArray structure and
       the gimp_buffer_data() routine.




Private Interfaces
------------------

these routines are used only inside tile*.c.  since they are not
exported to the world, no attempt is made to relate them to public
methods of the new design.  rather, they are listed to complete the
catalog of existing functionality.  the new design should somehow
provide these services as well.

    tile_init
    tile_attach
    tile_detach
    tile_manager_validate
    tile_invalidate
    tile_manager_invalidate_tiles


    tile_swap_in
    tile_swap_in_async
    tile_swap_out
    tile_swap_delete
    tile_swap_compress
    tile_cache_insert
    tile_cache_flush
    tile_cache_set_size
      these will all be internal to MemMgr since memory is now shared,
      not tiles.  they'll be triggered by d_read, d_write, d_release
      primarily.


Protected Interfaces
--------------------

these routines are used internally and shared with xcf.c and plugin.c.
both xcf and plugins arguably require privileged access to the
internal image structure.

    tile_lock
    tile_alloc
    tile_manager_map
    tile_manager_get  (plugins too)
    tile_swap_remove










