CLHS: Issue DEFPACKAGE Writeup

#

## Issue DEFPACKAGE Writeup

    Issue:         DEFPACKAGE

    References:    CLtL section 11.7.
                   Issue: IN-PACKAGE-FUNCTIONALITY
                   Issue: MAKE-PACKAGE-USE-DEFAULT
                   Issue: PACKAGE-DELETION

    Category:      ADDITION

    Edit history:  Version 1, 12-Mar-88, Moon
                   Version 2, 23-Mar-88, Moon, changes based on discussion
                   Version 3, 27-Sep-88, JonL
                    (remove :import, :shadowing-import; allow :export to work on
                     imported and inherited; update references to in-package, etc.)
                   Version 4,  1-Oct-88, Masinter
                   Version 5, 6-Oct-88, Moon
                   Version 6, 6-Oct-88, JonL (little nits)
                   Version 7, 2-Nov-88, JonL
    		 (incorporate further discussion; simplify handling of
    		  syntactic errors; specify ordering constraints).

    Problem description:

    Many users incorrectly think that package operations can be performed
    in any order.  CLtL (p.192) contributes to this misconception.
    Programmers need direction on the ordering constraint, especially for
    creating packages, since doing things out of order can lead to
    confusing or even intractable problems.

    If the definition of a package is scattered throughout a program as a
    number of individual forms, it is very easy to read a symbol before the
    package setup needed to read that symbol correctly has been accomplished.
    Three examples: an inherited symbol that should have been shadowed might
    be accessed; a single-colon prefix might be used for a symbol that hasn't
    yet been exported; an internal symbol might be created afresh where a
    symbol that will later be imported or inherited was intended.  These
    problems can be difficult to understand or even to recognize; in some
    cases it is difficult or impossible to correct the situation in the
    same Lisp image.

    Proposal (DEFPACKAGE:ADDITION):

    Add a DEFPACKAGE macro to the language.  In the description below,
    'package-name' and 'symbol-name' can be a symbol or a string; if a
    symbol, only its name matters, not what package it is in.

    The syntax of DEFPACKAGE is

      (DEFPACKAGE package-name {option}*)

    where each option is a list of a keyword and arguments.  Nothing in a
    DEFPACKAGE form is evaluated.

    Standard options for DEFPACKAGE are listed below.   Except for :SIZE,
    options may appear more than once (this is useful primarily for
    :IMPORT-FROM and :SHADOWING-IMPORT-FROM).

    (:NICKNAMES {package-name}*)
            Set the package's nicknames to the specified names.

    (:USE {package-name}*)
            The package is to "use" the other designated packages; that is,
            it will inherit from them.  The default value for this option
            should be the same as it is for MAKE-PACKAGE (also see the issue
            MAKE-PACKAGE-USE-DEFAULT).

    (:SHADOW {symbol-name}*)
            Create the specified symbols in the package being defined,
            making them shadows, just as the function SHADOW would do.

    (:SHADOWING-IMPORT-FROM package-name {symbol-name}*)
            Find the specified symbols in the specified package, import
            them into the package being defined, and place them on the
            shadowing symbols list.  In no case will symbols be created in
            any package other than the one being defined; a continuable error
            is signaled if no symbol is accessible in the package named
            package-name for one of the symbol-names.

    (:IMPORT-FROM package-name {symbol-name}*)
            Find the specified symbols in the specified package and import
            them into the package being defined.  In no case will symbols be
            created in a package other than the one being defined; a continuable
            error is signaled if no symbol is accessible in the package named
            package-name for one of the symbol-names.

    (:EXPORT {symbol-name}*)
            Find or create symbols with the specified names and export them.
            Note an interaction with the :USE option, since inherited symbols
            can be used rather than new ones created;  note also an interaction
            with the :IMPORT-FROM and :SHADOWING-IMPORT-FROM options, since
    	imported symbols can be used rather than new ones created.

    (:INTERN {symbol-name}*)
            Find or create symbols with the specified names.  Note an
            interaction with the :USE option, since inherited symbols
            can be used rather than new ones created.  This option is useful if
            an :IMPORT-FROM or a :SHADOWING-IMPORT-FROM option in a subsequent
            call to DEFPACKAGE (for some other package) expects to find these
            symbols accessible but not necessarily external.

    (:SIZE integer)
            Declare the approximate number of symbols expected in the package.
            This is an efficiency hint only, so that the package's table will
            not have to be frequently re-expanded when new symbols are added
            to it (e.g., by reading in a large file "in" that package).

    The order in which the options occur in a DEFPACKAGE form is irrelevant;
    but since the effects of the entry-making options are context-sensitive,
    the order in which they will be executed is as follows:
      (1) :SHADOW and :SHADOWING-IMPORT-FROM
      (2) :USE
      (3) :IMPORT-FROM and :INTERN
      (4) :EXPORT
    Shadows are established first, since they may be necessary to block
    spurious name conflicts when the use link is established.  Use links are
    established next so that :intern and :export may refer to normally
    inherited symbols.  The :export is done last so that it may refer to
    symbols created by any of the other options; in particular, shadows and
    imported symbols can be made external.  Note also the prescription on CLtL
    p.178 to cover the case of calling EXPORT on an inherited symbol.

    DEFPACKAGE creates the package as specified and returns it as its value.
    It has no other side effects; e.g., it does not alter the value of *PACKAGE*.
    The function COMPILE-FILE should treat top-level DEFPACKAGE forms the
    same way it treats the other package-effecting functions (see CLtL p.182).

    If the specified name already refers to an existing package, then the
    name-to-package mapping for that name is not changed.   At most, the
    existing package will be modified to reflect the new definition;  it is
    undefined what happens if the new definition is at variance with the
    current state of that package.  If one of the specified nicknames already
    refers to an existing package, then an error is signaled just the same
    as MAKE-PACKAGE would.  See the issue PACKAGE-DELETION for undoing the
    name-to-package mapping.

    Some DEFPACKAGE errors are, however,  purely syntactic.
      (1) An error should be signaled if :SIZE appears than once.
      (2) Since extended options might be allowed by other implementations,
          an error should be signaled if an option is present that is not
          actually supported in this implementation.
      (3) The collection of symbol-name arguments given to the options
          :SHADOW, :INTERN, :IMPORT-FROM, and :SHADOWING-IMPORT-FROM must
          all be disjoint; additionally, the symbol-name arguments given to
          :EXPORT and :INTERN must be disjoint. If either condition is
          violated, an error should be signaled.
    Name conflict errors will, of course, be handled by the underlying calls to
    USE-PACKAGE, IMPORT, and EXPORT.

    Examples:

    ;;; An example that "plays it super-safe" by using only strings as names;
    ;;;  does not even assume that the package it is read in to "uses" LISP;
    ;;   *never* creates any symbols whatsoever in the package that it is read
    ;;     in to.

    (LISP:DEFPACKAGE "MY-PACKAGE"
      (:NICKNAMES "MYPKG" "MY-PKG")
      (:USE "LISP")
      (:SHADOW "CAR" "CDR")
      (:SHADOWING-IMPORT-FROM "VENDOR-COMMON-LISP"  "CONS")
      (:IMPORT-FROM           "VENDOR-COMMON-LISP"  "GC")
      (:EXPORT "EQ" "CONS" "FROBOLA")
      )

    ;;; A similar call, mostly using symbols rather than strings as names.
    ;;; Expects to be read in to a package that "uses" LISP and *may* create
    ;;;  random internal symbols in that package (such as MY-PACKAGE etc).

    (defpackage my-package
      (:nicknames mypkg :MY-PKG)		;remember CL conventions for case
      (:use lisp)				; conversion on symbols
      (:shadow CAR :cdr #:cons)
      (:export "CONS")			;yes, this is the shadowed one.
      )

    Rationale:

    The availability of DEFPACKAGE encourages putting the entire package
    definition in a single place.  It also encourages putting all the package
    definitions of a program in a single file, which can be loaded before
    loading or compiling anything else that depends on those packages; such a
    file can be read in the USER package, avoiding any initial state issues.

    In addition, DEFPACKAGE allows a programming environment to process
    the whole package setup as a unit, providing better error-checking and
    more assistance with package problems, by dint of global knowledge of
    the package setup.

    Current practice:

    Symbolics Common Lisp (SCL) has always had a DEFPACKAGE, and users
    prefer it to individual calls to EXPORT, IMPORT, SHADOW, etc.  The SCL
    version of DEFPACKAGE has quite a few additional options, but none of
    them appears to be necessary to propose for Common Lisp at this time.
    This proposal is incompatible with Symbolics DEFPACKAGE in some ways
    that will probably not cause major problems.

    Cost to Implementors:

    Small--DEFPACKAGE can be implemented simply as a bunch of
    calls to existing functions.

    Cost to Users:

    Small, this is upward compatible.

    Cost of non-adoption:

    Packages continue to be difficult to use correctly.

    Benefits:

    Guide users away from using packages in ways that get them into trouble.

    Esthetics:

    Neutral.

    Discussion:

    It has been suggested that the "Put IN Seven EXtremely Random USEr
    Interface COmmands" mnemonic described in CLtL p.191 could be removed;
    and with possibly a few exceptions, the special handling of them by
    COMPILE-FILE could be removed.  As this would be an incompatible change,
    it is not part of this proposal.  However, a new mnemonic can be offered,
    to help remember the ordering constraints mentioned above:
              I REmember Six USEr Interface Expressions
    Each word in the sentence corresponds to one operation listed below:
       I				IN-PACKAGE	;"foot" to stand on
       REmember			REQUIRE		;ensure pre-requisite packages
       Six				SHADOW		;block multiple-inheritances
       USEr				USE-PACKAGE	;go for it!
       Interface			IMPORT		;bring in "foreign" symbols
       EXpressions			EXPORT		;a "face" to show to others.

    It is noted that DEFPACKAGE cannot be used to create two "mutually
    recursive" packages, such as:
        (defpackage my-package
          (:use lisp your-package)	      ;requires 'your-package' to exist first
          (:export "MY-FUN"))
        (defpackage your-package
          (:use lisp)
          (:import-from my-package "MY-FUN") ;requires 'my-package' to exist first
          (:export "MY-FUN"))
    However, nothing prevents one from using the package-effecting functions
    such as USE-PACKAGE, IMPORT, and EXPORT to establish such links (which
    ought to be very rare) after a more standard use of DEFPACKAGE.

    The macroexpansion of DEFPACKAGE could usefully canonicalize the names
    into strings, so that even if a source file has random symbols in the
    DEFPACKAGE form, the compiled file would only contain strings.

    Frequently additional implementation-dependent options take the
    form of a keyword standing by itself as an abbreviation for a list
    (keyword T); this syntax should be properly reported as an unrecognized
    option in implementations that do not support it.

    Definition forms in Common Lisp usually just establish a name-to-object
    mapping; there is little precedent for them to modify other global-context
    state.  For this reason, we didn't want DEFPACKAGE also to "go" into the
    new package.  If it did so, like IN-PACKAGE, then the following reasonable
    file would become confused, because it wouldn't all be in one package:
       (in-package "USER")
       (defpackage "WATER"      (:use "LISP")             (:export "FISH"))
       (defpackage "ALCHEMY"    (:use "LISP" "PHLOGISTON) (:export gold))
    Should the token 'gold' be read while in the USER package, or in the
    the WATER package?

    The issue IN-PACKAGE-FUNCTIONALITY recommends that IN-PACKAGE  be
    incompatibly changed to recognize only existing packages, not to create
    them.  IN-PACKAGE would then not accept any keyword arguments.

    The function MAKE-PACKAGE might also be extended to take all the keywords
    that DEFPACKAGE does. This could be the subject of a separate cleanup.

Copyright 1996-2005, LispWorks Ltd. All rights reserved.