(Message inbox:31)
Return-Path: dshr@sun.com
Received: by expo.lcs.mit.edu; Mon, 6 Feb 89 12:47:48 EST
Received: by ATHENA.MIT.EDU (5.45/4.7) id AA21584; Mon, 6 Feb 89 12:47:38 EST
Received: from snail.Sun.COM by Sun.COM (4.1/SMI-4.0)
	id AA11194; Mon, 6 Feb 89 09:49:55 PST
Received: from devnull.sun.com by snail.Sun.COM (4.1/SMI-4.0)
	id AA13577; Mon, 6 Feb 89 09:46:40 PST
Received: by devnull.sun.com (4.0/SMI-4.0)
	id AA14389; Mon, 6 Feb 89 09:44:41 PST
Message-Id: <8902061744.AA14389@devnull.sun.com>
Date: Mon 6 Feb 1989 09:36:43 EST
From: David Rosenthal <dshr@sun.com>
Subject: Usenix talk & paper
To: xpert@athena.mit.edu

Last week,  I presented a paper at Usenix that David Lemke and I had
written on the problem of choosing and using Visuals.  Several people
asked me to post the talk,  and it is attached below.  Also,  we found
a bug in the paper printed in the Proceedings,  and I will be posting
a shar file with the corrected source for the paper.  The bug was that
the code in the Proceedings chooses the largest available Visual,  and
you should choose the SMALLEST avaialable Visual that will do the job,
so as not to waste resources.

	David.





                San Diego Usenix Talk Script


                   David S. H. Rosenthal






Introduction

Good morning.  I'm  Sgt.  Rosenthal  of  the  Softies,   the
Software  Police.   Even  though  its  early,   you'd better
listen up and listen good, because what I'm  going  to  tell
you  this morning can keep some of you out of a lot of trou-
ble.

Officer Lemke and I work in the X Division, and  we've  been
out  there on the net making undercover buys of X games pro-
grams.  When we test them back in the labs, we almost always
find recreational bugs.

The Zero-Tolerance Bugs Policy

I'm  here  today  to  talk   to   you   about   the   Quayle
administration's  new  ``zero-tolerance''  policy  on  bugs.
Nancy Reagan's "Just Say No" to bugs program  was  all  very
well,  but it didn't stop the wave of bugs that engulfed the
nation  during  her  husband's  administration.    The   new
administration has decided to get tough on bugs,  and that's
what the zero-tolerance policy is all about.

Surveys tell us that many of you have pretty tolerant  atti-
tudes to recreational bugs.  I'm here to change that.  There
are three main reasons why the policy  focusses  on  recrea-
tional bugs:

+  In most of the cases we see,   programs with recreational
   bugs go on to develop more serious hard bugs.

+  The sharing of  software  involved  in  recreational  bug
   usage can lead to the transmission of viruses,  worms and
   other parasites.

+  And,  of course,  the bottom  line  is  that  bug-free  X
   software isn't just a good idea,  its the law.

Penalties for Bug Offences

Under the zero-tolerance policy the  penalties  for  even  a
small recreational bug can include:

+  confiscation of resources,

+  X protocol errors,

+  termination of your program,

+  visions of bizarre colors on your screen,

+  and above all making the designers of X unhappy.

You do not want to make the X designers  unhappy.   If  they
get  too  unhappy  with the way you're mis-using their crea-
tion, they may just get together and build you a  whole  new
industry-standard  window system.  Look how much work it was
last time they did this.  Think what  they've  learnt  since
then.

The Neighbourhood Bug-Watch Program

Everyone deserves a bug-free environment, but  achieving  it
isn't  easy.  The scourge of bugs has affected every stratum
of society.  People you trust implicitly,  role-models  like
the  folks at MIT and elsewhere who work long hours into the
night selflessly donating their software to the  poor,  even
these people may be into recreational bugs.  And you may not
find out until its too late.  When we in  the  Softies  stop
you  out  there  on the net, it won't do you any good to say
``This code isn't mine,  I'm just holding it for a friend''.

Types of Bug to Look Out For

X bugs you should be looking for can be divided into:

+  The narcotics that leave programs catatonic.   A  warning
   sign to look for is a program that starts ignoring Expose
   events.  You've heard plaintive complaints from people on
   xpert  that  they  draw  and draw and nothing is visible;
   they are suffering from narcotics abuse.

+  The stimulants that drive  programs  into  a  frenzy.   A
   warning  sign  is  a  program  that  tries  to argue with
   authority figures like window managers.

+  The steroids that cause programs (and especially servers)
   to swell up and get huge.

+  The hallucinogens that cause swirling visions in  bizarre
   colors.

This morning I'm going to review the hallucinogens and their
symptoms, and we need to start with some basic physiology.

Basic X Physiology

When your X client draws,  it does so in a Drawable, a  rec-
tangle  of  memory locations containing numbers called pixel
values.  The drawing  operations  like  CopyArea,  PolyLine,
PutImage and PolyText8 all change these pixel values.

Some of these drawables may be windows, and some of the win-
dows may be mapped.  Every few milliseconds,  the screen you
are looking at is refreshed by reading  the  pixels  of  the
visible  parts  of  the  the mapped windows, and using these
values to index into a colormap.  The colormap cell  indexed
by the pixel value contains a red,  a green and a blue value
that is fed to the corresponding gun of the screen.

The details of this process  can  differ  between  different
types  of hardware.  The differences are exposed; X gives no
guarantee of portability and you have to be aware  of  these
differences:

+  Is there one gun (monochrome) or three (color)?

+  How many entries in a colormap?

+  Is there one colormap,  or one colormap per gun?

+  How many colormaps are simultaneously accessible?

+  Can a client write the colormap entries?

+  How many bits in each colormap entry for each gun?

All these differences are collected together into a  concept
called  a  Visual, and the possible Visuals divided into six
classes:

+  StaticGray.   The  pixel  value  indexes  a   predefined,
   read-only  colormap.   For  each colormap cell,  the red,
   green and blue values are the  same,   producing  a  gray
   image.

+  StaticColor.  The  pixel  values  indexes  a  predefined,
   read-only  colormap.  The red,  green and blue values for
   each cell are server-dependent.

+  TrueColor.  The pixel value is  divided  into  sub-fields
   for  red,   green  and  blue.   Each sub-field separately
   indexes the appropriate primary of a  predefined,   read-
   only  colormap.  The red,  green and blue values for each
   cell are server-dependent, and are selected to provide  a
   nearly linear increasing ramp.

+  GrayScale.  The pixel value indexes a colormap which  the
   client  can  alter,   subject to the restriction that the
   red,  green and blue values of each cell must  always  be
   the same,  producing a gray image.

+  PseudoColor.  The pixel value indexes  a  colormap  which
   the client can alter.  The red,  green and blue values of
   each cell can be selected arbitrarily.

+  DirectColor.  The pixel value is divided into  sub-fields
   for  red,   green  and  blue.   Each sub-field separately
   indexes the appropriate primary of a  colormap  that  the
   client can alter.

When you implement an X server for an individual workstation
configuration,  you have to decide which Visuals you want to
make available to your clients.  Sometimes,  the  choice  is
obvious.  For example,  a normal monochrome workstation like
a Sun 3/50 would naturally export  a  single  depth  1  Sta-
ticGray  Visual.   A  simple  color  workstation  like a Sun
3/160C with an 8-bit-per-pixel frame  buffer  and  a  single
256-entry  colormap  would naturally export a single depth 8
PseudoColor Visual.

As hardware gets more complex,  the choice may be to  export
several  Visuals for a single screen.  For example,  the Sun
3/110C in effect has a 10-bit deep frame buffer.  8  of  the
bits  index  a  single  256-entry colormap,  one of the bits
acts as a simple monochrome framebuffer,   and  one  of  the
bits selects whether the 8-bit color or the 1-bit monochrome
image is visible at this pixel.  In this case,   the  normal
choice would be to export two Visuals, a depth 8 PseudoColor
Visual and a depth 1 StaticGray Visual.  Even  more  complex
hardware  may  need  to export many Visuals to allow clients
full access to its capabilities.

If you're a sneaky server implementor,  you may even  export
Visuals  that  you  don't strictly have.  For example,  both
the MIT server for  the  DEC  QDSS  color  display  and  the
X11/NeWS  server  for  Sun  color hardware export both Pseu-
doColor and  StaticColor  Visuals  even  thought  they  both
really  have  only  a  PseudoColor display.  The StaticColor
Visuals use otherwise unused entries in the hardware  color-
map,   allowing  simple color applications to achieve better
colormap sharing than they would  on  a  single  PseudoColor
Visual.

What this means to you as an X client programmer is that you
are likely to be faced with a choice of Visuals, and that as
hardware evolves to give you bigger and better thrills, this
choice  is  likely  to expand greatly.  You have to choose a
suitable Visual for your client, and unless you choose  with
great  care, you will be increasingly at risk for hallucino-
genic bugs.

Hallucinogenic Bugs and Their Symptoms

Hallucinogenic bugs achieve  their  effects  by  interfering
with the Visual mechanism.  Here's a list of the common hal-
lucinogens and their symptoms:

+  A program can assume that the default Visual is the  only
   available Visual.

   Supose you had a color application that made this assump-
   tion  and  a  Sun  3/110C.  The server implementors might
   have chosen to make the  depth-1  monochrome  Visual  the
   default,   in  which case the application would fail even
   though the hardware actually supported color.

+  A program can assume that all Visuals with more than  two
   Colormap cells are color.

   Suppose you  had  a  color  application  that  made  this
   assumption  and  a  Sun  3/160GS  (grey-scale).  It might
   chose a foreground and a background color which mapped to
   the  same shade of grey,  and the output would be invisi-
   ble.

+  A program can assume that all Visuals with more than  two
   Colormap cells have writable Colormaps.

   This is a very common assumption.  Clients making it fail
   with  an  Access error when they try to write a read-only
   colormap cell.

+  A program can assume that Colormaps (and  especially  the
   default Colormap) are infinitely large,  so that attempts
   to allocate private cells in them will always succeed.

   Clients that make this  assumption  will  sometimes  fail
   with  Alloc  errors,  and sometimes succeed, depending on
   the number of cells they ask for and the number of  cells
   that other clients have left available in the colormap.

+  A program can assume that all  Colormaps  work  with  all
   Visuals.

   Clients that install Colormaps in windows other than  the
   one  they  were  created for may get a Match error if the
   Visuals don't correspond.

Traditional Values Keep Your Colors Sober

At this point,  you're probably saying ``this  isn't  a  big
deal, all my friends do bugs like this''.  You're wrong; the
only clients that can ignore the  question  of  Visuals  are
those  that  use the BlackPixel() and WhitePixel() macros to
paint a black and white image in the  default  Visual.   All
other  clients must pay some attention to the details of the
Visual(s) they are using,  if  they  want  their  output  to
appear in sober, everyday hues.

When you choose a color,  you really choose an  RGB  triple.
For  example,   you  say  ``I'd  like  this text to come out
blue'', and what you really mean  is  that  you'd  like  the
image  of  the  text  when its refreshed on to the screen to
have the RGB triple [001].  Working backwards:

+  this means that the colormap cell  used  in  the  refresh
   process must have the RGB triple [001] in it,

+  and this means that the pixel value in  the  window  must
   have  indexed to a cell in the colormap with the RGB tri-
   ple [001] in it,

+  and this means that the foreground pixel value in the  GC
   that  you  used  to draw the text must have been one that
   would index to a cell with the RGB triple [001] in it,

+  and this means that the foreground pixel value  you  sup-
   plied  to a CreateGC or ChangeGC call that got you the GC
   that you used to draw the text must have  been  one  that
   would index to a cell with the RGB triple [001] in it,

OK,  enough of the "for the lack of a nail the horseshoe was
lost"  stuff.  You need to have some way to convert your RGB
value into a pixel value that will map  back  into  the  RGB
value.

Where Do Pixel Values Come From?

I expect you've overheard whispered conversations among your
friends  about  where pixel values come from and how you can
get one.  I'm here to dispel the myths,  and assure you that
there are just three legal ways you can get a pixel value:

+  You can give the server a colormap and an RGB value, or a
   text name for an RGB value, and ask it to give you back a
   pixel value that will index to a read-only cell  in  that
   colormap  that  has  the closest available match for that
   RGB value.

+  You can ask the server to reserve you a  private,   writ-
   able  cell  in  a Colormap.  If this succeeds, the server
   will give you a pixel value for your  private  cell,  and
   you can set whatever RGB value you like.

+  Or there are various ways in  which  you  can  compute  a
   pixel  value  from an RGB triple by predicting the values
   in the colormap:

   +  If the server has a TrueColor Visual it  will  provide
      linear ramps.

   +  The  so-called  ``Standard  Colormaps''  also  provide
      linear ramps.

   +  The connection  handshake  process  tells  you  Black-
      Pixel() and WhitePixel() for the default Visual.

Be warned that values you obtain any other way are  illegal,
and may be bugs.

Which Method Is Right For You?

+  For beginners,  and the simplest clients,   using  Black-
   Pixel() and WhitePixel() in the default Visual is best.

+  Using read-only colormap cells and letting the server  do
   the  conversion  will  work  on  any Visual and maximises
   sharing with other  clients.   It  is  the  technique  of
   choice unless:

   +  you have a lot of colors,

   +  the exact  representation  of  colors  is  of  primary
      importance to you,

   +  or you want to play colormap tricks.

+  Clients that want to  display  an  RGB  image  with  many
   colors  should use the prediction technique in one of the
   Standard Colormaps, as soon as the experts can  agree  on
   how to make the technique really work.

+  Colormap tricks  aren't  actually  illegal,  but  they're
   risky  and  we  advise against them.  If you want to play
   these tricks you should make  sure  beforehand  that  you
   have  a  dynamic Visual, and create a private Colormap so
   as not to disturb your neighbours.

Are You Safe if You Use a Toolkit?

I expect you'll hear people saying ``I don't have  to  worry
about  this kind of bug,  I use a Toolkit''.  Unfortunately,
this is just another of the myths about bugs.  Not  that  we
don't advise people to use Toolkits; used in accordance with
the manufacturer's instructions a Toolkit can keep you  safe
from  many common bugs and save you a great deal of time and
trouble.  But the sad fact is that  Toolkits  don't  protect
against  hallucinogenic bugs, and this hasn't had the publi-
city it deserves - the  Intrinsics  manual  doesn't  mention
Visuals.

How Do Widgets Get Their Visuals?

At present,  Visuals are hereditary - Widgets inherit  their
Visuals  from  their  parents.   The  window for a Widget is
created when the Widget is realized, and the Visual  has  to
be  bound  to  the  window  then.   At  the  root  of  every
application's tree of Widgets is a Shell Widget; the Intrin-
sics  define this to inherit its Visual from its parent (ie.
the root window),  so it will have the default Visual.   All
the  Widgets  on the X11R3 tape have realize procedures that
inherit their Visuals from their parents, so that by  induc-
tion all Widgets have the default Visual.

If you're creating a Widget that others may use, it is  your
responsibility  to  ensure  that  it gets a suitable Visual.
Simply trusting your parent to do this  for  you  is  not  a
suitable way to discharge this responsibility

How Can You Protect Your Widgets From Hallucinogens?

You can do this by equipping them with  a  suitable  realize
procedure.   Based  on  your knowledge of the color require-
ments of the Widget you are defining,  you should choose one
of  the legal methods for obtaining pixel values.  Then, you
should write a realize procedure that:

+  Ranks the Visuals in  preference  order,  and  finds  the
   smallest  best Visual that will do the job.  Code to do a
   similar task is in the paper.

+  Obtains a suitable Colormap  for  the  conversion  method
   selected,  by:

   +  Using one of the Colormap  description  properties  on
      the  root  window,  if  you're  using  the  prediction
      method.

   +  Choosing the default Colormap,  if that  will  do  the
      job.

   +  Creating a private Colormap using the selected  Visual
      and the root window of the screen.

+  Creates a window in the selected  Visual,   and  supplies
   the selected Colormap as one of its attributes.

+  Sets the window and the colormap into the core attributes
   of the Widget.

Remember,  educating your Widgets about Visuals  right  from
the start is the key to keeping them bug-free for life.

Don't Let Your Widgets Accept Colormaps From Strangers

You're probably asking  ``If  colormaps  are  attributes  of
Widgets,   why shouldn't I let them be Resources that can be
changed by the user?'' The problem is that the bright  shiny
colormap  that the nice user-man is offering your Widget may
be a hallucinogenic bug.  It might not belong  to  the  same
Visual as your Widget's window, and when the trusting little
Widget tries to set the window's colormap attribute,   wham!
it gets a Match error.

Unfortunately,  there is no way to test a  colormap  before-
hand to see if it is compatible with the window whose attri-
bute you're trying to set.  Given a colormap ID,  the proto-
col  doesn't provide a way to find the Visual it was created
for.  So,  its better to let your Widgets choose  their  own
colormaps,  and not to provide any way for the user to over-
ride their choice.

If you have to accept a colormap from someone else, and  you
don't  know the Visual, you can create your own colormap for
the Visual you want, read all the entries from  the  strange
map,  and  create  corresponding  entries  in  your own map.
There are problems with this approach:

+  The entries have to be read back to the client  and  then
   shipped  back  to the server,  because there is no way of
   copying entries between colormaps of different Visuals.

+  There is no way of discovering which cells in the  source
   colormap are sharable and which private.  This means that
   the  create-and-copy  destroys  the  sharability  of  the
   entries.

+  The whole idea of creating a new colormap  works  against
   the sharing of resources that was probably the reason for
   trying to accept a strange colormap in the first place.

How You Can Get Your Programs Off Bugs

+  Understand the X display and Visual mechanism.

+  Understand the legal methods of converting RGB triples to
   pixel values.

+  Chose a method and a Visual that match your application.

+  Don't let someone else choose your Visual for you.

+  Don't accept colormaps from strangers.

What To Do If You Find Bugs In Your Program

+  Isolate the program immediately.

+  Do not pass the program on to others.

+  Examine the program's assumptions carefully.

+  Call for help from the xperts.
