Notes on bi-directional simultanious I/O: (from barrett@asgard.cs.colorado.edu)

Problems Discussed:

   - How do you prevent a write from blocking when select(2) returns true?
   - How do you detect when the slave side of the pty closes (logout) ?
   - How do you prevent the deamon from creating zombie processes ?
   - Portability Notes

Porting Notes:

A strong attempt has been made to use POSIX compatable functions as much
as possible.  Obviously, POSIX does not define everything, so here is what is
non-posix:

  utmp.c:  Contains the most machine specific code.  The only function it 
	   exports is setlogin, which is used to update the /etc/utmp file 
	   used by the getlogin library call and the who program (finger) and 
	   the like.  You can delete the call to setlogin in netlogind.c if 
	   this file causes problems.

   tty.c:  the constant ONLCR is needed to set the tty into the right "cooked"
	   state for the remote shell.  This shouln't be a problem, but you
	   may need to play around with the #define _POSIX_SOURCE and the 
	   include file order.  The rest is completly POSIX!  Amazing!

The files "txfr.c", "socket.c", "pty.c" all will have BSD-style calls there.
The reason is that POSIX doesn't have these OS calls required for networking.

  txfr.c:  Makes extensive use of select.  There are comments in the file if
	   you find the data gets constipated or the program aborts with
	   read or write failure.

socket.c:  Contains all the BSD socket style calls.  Should work on any BSD
	   style networking system.

pty.c:	   It appears that pty's should work on both BSD and system V.  This
	   code shouldn't cause any problems.

Anything else should port straight away.  If you have problems, the code has
been designed to be easily debuggable.  If you set the run-time option "-d",
the program will not do a fork when the "accept" socket call completes.  This
should allow you to examine almost anythnig you wish.  Typically the "-l"
and "-p" and "-f" options are very helpful here.  You don't have to run the
program as root.

If you are using gcc on the decstations or sparcs:

The decstations have a problem compiling psignal.c with gcc.  Use the dec
compiler.

The sun sparc machines define the memcmp stuff in <memory.h>.  This is 
non-standard.  I discovered this only when compiling with gcc -Wall.  The
sun compiler silently compiles everything fine.

Both machines report a lot of implicitly defined functions.  I don't know
why the headers for gcc can't do this.  Let me know if you find a fix.

Dectecting Close on Slave Side of Pty (for netlogind):

We need a way to tell from the master side, if the slave side
of the pty has been closed.  The BSD pty man page doesn't say.  The HP-UX man 
pages states "...exceptional condition refers to an open(2), close(2), or 
ioctl(2) request pending on the slave side...".   But, this only will occur
if TIOCTRAP is enabled on the master pty pty(7).  HP's pty TIOCTRAP is an 
hp-only feature.  No such features exist on BSD, although it may be that
setting the baudrate on the slave side to B0 will cause the master to see
it--I can't count on this because the shell doesn't appear to do this on
exit.  Thus, SIGCHLD will have to be used.  Note that System V's SIGCLD is NOT 
the same as POSIX's SIGCHLD.

One problem is that SIGCHLD isn't triggered in POSIX if sysconf(_SC_JOB_CONTROL)
is zero.  The select statement in netlogind will not abort if the child shell
dies and SIGCHLD is not caught.  The solution appears to be just to assume
that System V is being used if job control is not defined, and that SIGCLD(sic)
will then be triggered.  Note that SIGCHLD will still be defined, it just need
not be triggered when a used on a non-job control system.  Thus, for system
V, there appears to be no way to force a select on the master side of a pty
to detect a close on the slave side.  As for daemons causing zombie processes,
it appears to be impossible to stop on a POSIX system with no job control, but
it CAN be solved on a System V system without job-control by setting SIGCLD
to SIG_IGN before forking.

Athother $64,000 question is can select ensure that no blocks will occur for 
read and write to sockets and ptys.  Four independent questions must be 
answered: read/tty, read/sock, write/tty, write/sock for *blocking* 
descriptors!  The answer to all questions is YES for non-blocking descriptors.
Does a write block if there isn't enough room in a blocking fd?  The answer is 
apparently yes, but no implication is made about atomic writes except for 
send(2) implies that atomic writes are used for it.  The result?  Use FNDELAY 
and on BSD select I/O (or O_NONBLOCK on POSIX).

There is a "solution" to the above.  Attempt to use POSIX semantics of 
O_NONBLOCK with select.  POSIX states that writes are atomic if smaller
than PIPE_BUF.  For larger sizes, write may return a partial write, or 
-1 errno EAGAIN.  This is another serious problem.  If it happens, then select
for write will immediately return true and the write will continue to return
EAGAIN until the buffer flushes to a small enough value to succeeed.  We
will be sucking the CPU dry on systems which do this.  The only "solution"
is to have the select "poll" by inserting a delay (say 100ms) after the
write returns eagain.  The idea would be to insert a select for all events
except the one which returned EAGAIN with a 100ms delay, which would be
invoked wwhen EAGAIN occurred.  After that select finished, just return to
the main select loop.  This would still allow other events to be processed
immediately, and would limit the CPU interrupts to 10 / sec at the cost of
adding upto a 100ms propagation delay into the write for that descriptor.

I have chosen to assume that EAGAIN will never happen, and let the program
abort if it does.  That way, if this code is ported to a machine which has
the above "feature", you will find out real quickly and can implement the
above "solution".

Another problem has to do with SOCK_STREAM sockets.  Is there a propagation
delay for keystrokes sent for example?  TCP_NODELAY documented in TCP(4P)
talks about his.

Here is the related information from BSD:

   - read calls return as much data as possible [PS1:6-25]

   - nonblocking io (FNDELAY) applies ONLY to ttys and sockets [fcntl(2)-2]

   - Non-blocking sockets and tty fd's are created with the FNDELAY fcntl flag.
     [PS1:8-25] [tty(4)-2]

   - Writes to non-blocking sockets or to a terminal will write as much as 
     possible.  If no bytes are writable, EWOULDBLOCK is returned. 
     [PS1:8-25] [PS1:6-19]

   - A recv from a non-blocking socket will return EWOULDBLOCK if no input 
     is available.  [recv(2)-1]

   - Reads to a non-blocking control terminal return EWOULDBLOCK if no input
     is available.  Partial reads will succeed return the number of bytes 
     available [tty(4)-2] [PS1:6-19].
   
   - Select returns true when an input or output may be achieved without
     blocking. [PS1:6-19]
   
   - an accept will block unless non-blocking io is enabled.  [PS1:6-34]


PS1:8-25:

5.2 Non-blocking Sockets 

   fcntl(sock, F_SETFL, FNDELAY)

   accept connect send recv read write, all will return -1 and EWOULDBLOCK.

   "If an operation such as send cannot be done in its entirety, but partial
   writes are sensible (for example, when using a stream socket), the data
   that can be sent immediately will be processed, and the return amount will
   indicate the amount actually sent."

PS1:6-13:  Signals

   A process can request notification via a SIGIO signal when input or output
   is possible on a descriptor, or when a non-blocking operation completes.

PS1:6-19:  Select

   - A descriptor selects for input in an input oriented operation such as 
     read or receive is possible, or if a connection request may be accepted
     (see section 2.3.1.4)

     (2.3.1.4) Accepting Connections: ...the [accept] call will wait unless
     non-blocking I/O has been enabled.
     
   - A descriptor selects for output in an output oriented operation such as
     write or send is possible, or if an operation that was "in progress",
     such as connection establishment, has completed (see section 2.1.3).

     (2.1.3): A process that wishes to do non-blocking operations on one of its
     descriptors sets the descriptor in a non-blocking mode as descripbed in
     section 1.5.4.  Thereafter, the read call will return a specific
     EWOULDBLOCK error indication if there is no data to be read.  The process
     may select the associated descriptor to determine when a read is possible.

     Output attempted when a descriptor can accept less than is requested wil
     either accept some of the provided data, returning a shorter than normal
     length, or return an error indicating that the operation would block.
     More output can be performed as soon as a select call indicates the object
     is writable.

     Operations other than data input or output may be performed on a
     descriptor in a non-blocking fashion.  These operations will return with a
     characteristic error indicating that they are in progress if they cannot
     complete immediately.  The descriptor may then be serected for wwrite to
     find out when the operatin has been completed.  When select indicates the
     descriptor is writable, the operation has completed.  Depending on the
     nature of the descriptor and the operation, additional activity may be
     started or the new state may be tested.

   - A descriptor selects for an exceptional condition if a condition that
     would cause a SIGURG signal to be generated exists (see section 1.3.2),
     or other device-specific events have occurred.

   Operations on non-blocking descriptors will either complete immediately,
   note an error EWOULDBLOCK, partially complete an input or output operation
   returning a partial count, or return an error EINPROGRESS noting that the 
   requested operation is in progress. A descriptor which has signalling 
   enabled will cause the specified process and/or process group to be
   signalled, with a SIGIO for input, output, or in-progress operation
   complete, or a SIGURG for exceptional conditions.  

   For example, when writing to a terminal using non-blocking output, the
   system will accept only as much data as there is buffer space for and
   return; when making a connection on a socket, the operation may return
   indicating that the connection establishment is "in progress".  The
   select facility can be used to determine when further output is possible on
   the terminal, or when the connection establishment attempt is complete.

recv(2)-1:

   "The select(2) call may be used to determine when more data arrives."

write(2)-1:

   When using non-blocking I/O on objects such as sockets that are subject to
   flow control, write and writev may write fewer bytes than requested.

fcntl(2)-1:

   FNDELAY Non-blocking I/O: if no data is available to a read call, or if a
   write operation would block, the call returns -1 with the error EWOULDBLOCK.

fcntl(2)-2:

   BUGS: The asychronous I/O facilities of FNDELAY and FASYNC are currently
   available only for tty and socket operations.

--- 27-Jun-94 ---

Received man pages and include files for attempt to port to UNICOS on
the Cray, and Solaris OS and IRIX. 

IRIX and Solaris can't deal with ut_host in utmp.c.

IRIX and UNICOS can't open the pty: "no pty's available".
Solaris reports: 

could not acquire fd 0 (/dev/ttyp0) as controlling terminal--Inappropriate ioctl for device"

