From: craigf@cory.Berkeley.EDU (Craig Federighi)
Newsgroups: comp.lang.tcl
Subject: Tcl Future Directions Session #2 Notes
Date: 14 Jun 1993 21:45:04 GMT
Organization: University of California, Berkeley
Lines: 269
Message-ID: <1virh0$1iv@agate.berkeley.edu>
NNTP-Posting-Host: cory.berkeley.edu

These are the notes from the second "future directions" session at the
Tcl/Tk conference at Berkeley, June 10-11 1993.  Other Berkeley people
have been posting (or will post) notes for the other three sessions.
The focus of this session was managing extensions to Tcl/Tk.

       Tcl Future Directions Session #2, Thursday June 10 4:00-5:30
       ------------------------------------------------------------
       Notes taken by Craig Federighi.

Managing Extensions
----------------------------------

John Ousterhout led a discussion on managing Tcl and Tk extensions, and 
proposed several possible solutions.  The following outline is based on his 
notes.  Lines tagged with a >> are not part of the official handout.

1. Introduction
    Goal:
        * Make it easy to mix and match various extensions to Tcl and Tk
          (both C code and Tcl scripts)
    Problems:
        * Name conflicts
        * Installation is non-uniform and clumsy
        * Proliferation of binaries
    Solutions:
        * Naming conventions
        * Installation conventions
        * Dynamic Linking, Better auto-loading
        
2. Naming Conventions
    Problem:
        * Each person assumed he/she is the only one building 
          extensions
        * Different packages use same names for global variables
          and commands, e.g. "send"
    Possible Solution #1: module mechanism
        * Tcl provides mechanism for static variables and procedures
            >> these are invisible outside of the module
        * Still doesn't solve problem for new commands and global
          procedures.
            >> Disagreement was actually expressed about this assertion.
               A brief description of a module mechanism that eliminates
               this problem was discussed.  See below.
    Possible solution #2:  single command with extensions
        * Like string command: "string index", etc.
        * Still need to find a unique command name, unique
          variable names.
    Possible solution #3:
        * For each application or extension, pick a short prefix: e.g.:
            expect_
            xp_
            tk_
            dp_
        * Use prefix in all global names (variables, commands, procedures):
            xp_send
            tk_priv
            dp_rpc
        * Suggestions for uniformity:
            - Only one underscore per name
            - Use capitalization at internal word boundaries
        * Examples: tk_menuBar, not tk_menu_bar or tk_menubar
            >> Here a significant debate began
              - The group decided that developers could do whatever they
                wanted after the first underscore.  J.O. would simply
                offer the mixed case naming as a suggestion.
              - Developers that used proper prefixing for their modules
                would be considered good members of the community
              
            >> Internal identifiers:
              - Several suggestions were put forward for internal (private)
                identifiers within a module.  Suggestions:
                  - _tk_foo
                  - tk:foo
                J.O. will pick one and offer it as a suggestion.
            >> Non-uniformity, ugliness, and end users
              - Many participants were not entirely satisfied by the prefix
                scheme.  Specifically:
                  - Some Tcl/Tk commands seem to have an unfair advantage,
                    i.e. the button widget in Tk is "button," not "tk_button,"
                    while a button in a separate package would have to
                    have a prefix.
                  - Some expressed distaste at having to type and read
                    code filled with extensions.
                  - A few participants expressed concern about having to
                    make end users of commercial products deal with a bunch
                    of prefixes.
            >> Modules revisited
              - One participant (me) pointed out that a Module interface
                that supported renaming (such as used in Modula 3) could
                eliminate most of the above problems.  The idea is as follows:
              Imagine three packages: DP (UCB Tcl-DP), Expect, and Tk.
              Authors of these packages could write code like:
              module DP {
                # External variables
                xvars conn
                
                # externally visible proc
                xproc Send {a1 a2} { ... }
                
                # private (internal proc)
                proc DoSomething {} {...} 
              }
              
              The client can use modules as follows:
              import Tk.all Expect {DP.Send as RPC}
              
              # client calls:
              
              # Tk.all means all Tk commands are imported to be called
              # without a module prefix:
              button .foo
              
              # Expect package is imported so that external identifiers
              # must be prefaced with the module name
              Expect.Send "a string"
              
              # The Send command is imported from  Tcl-DP under the
              # name RPC and can be called without a prefix
              RPC $host ls
              
              Now suppose that someone on the net releases a new package
              that also used the prefix "DP."  We're still okay:
              
              import Tk.all Expect {"UCB-DP" as DP} {"Other-DP" as ODP}
              DP.Send ...
              ODP.do ...
              
              Several participants liked this approach.  J.O. felt that
              the prefix scheme was much simpler and reduces renaming-induced
              confusion when reading others code.  We were all encouraged
              to read the source code for "sh" for a clear illustration
              of this problem.
    Classes in Prefixes:
        * Establish a central registry for prefixes.
            >> J.O. felt that this wouldn't be necessary for a while,
               but at least one participant thought that the registry
               should be established immediately.
    Solution #4:  Object-oriented commands
        * Like Tk widgets
        * One command to create object, returns object name: 
            "button .b"
        * Use object name as command name, put action as 
          first argument: ".b invoke"
        * Avoids command space pollution: only one new command
          (plus object commands).
        * Can provide uniform actions for many different kinds of objects.
        * Must allocate unique object names (similar to choosing unique
          prefix).
        >> Participants pointed out that unique names must be generated
           for object names.  The group concluded that the module prefix
           should be used as a prefix in all dynamically generated object
           identifiers.  Tk reserves the "." prefix for object instance 
           names.
           
3. Installation
    Scripts are easy:
        * Put .tcl files in a directory.
        * Create "tclIndex" file
        * Add directory to "auto_path"
          >> the environment variables "TCL_LIBRARY" and "TK_LIBRARY"
             serve this purpose
    C code is hard
        * Where to put source code?
        * Must compile extensions.
        * Must add code to "wish" main program by hand.
        * Must make new binary.
        * Different packages install differently.
        * Incompatible versions

    Source Code Management
        * Pick a directory to hold sources for Tcl, Tk, and 
          extensions.
        * Each package or application is a subdirectory of this directory:
            /usr/local/src/tcl:
              tcl7.0
              tk3.3
              ...
              expect2.1
            >> under each package directory would be a lib/ include/ and
               bin directory.
            >> This will be set up so that all of the public headers, 
               libraries, Tcl scripts, and binaries are installed in the 
               appropriate places (e.g. /usr/local/bin).
            >> Many expressed the need to support multiple architecture
               environments.
        * Keep version name in a directory name, so there can be multiple
          versions of the same package.
            >> have each module define a Tcl variable containing its version
               number, e.g. expect_version, so that applications
               can verify that they have the right version of 
               included libraries.
        * Use GNU "autoconfig" for configuration
            >> A question was raised as to how the use of a GNU
               tool might affect the distribution of commercial software.
               J.O. wasn't sure what the legal specifics were, but believed
               this probably isn't a problem.
        * Create a library as well as an application

    Incorporating Extensions
      In package:
        * Define one C initialization procedure
            Expect_Init
            Dp_Init
        * Init proc takes single argument: Tcl interpreter
        * Calls Tcl_CreateCommand to create new command(s) for
          package, performs and other initialization for package.
      
      To user package in application:
        * Create procedure TclAppInit that calls all relevant 
          initialization procedures, invokes application startup script.
        * Link with relevant libraries.
        * No need to modify main(): it calls Tcl_AppInit; Tcl and Tk provide
          default Tcl_AppInit
            >> Modules can insure that other modules that they depend on
               are already initialized by calling their initprocs
               explicitly before performing their own initialization.
            >> Module writers should make sure that module initialization
               procs are idempotent so that redundant init calls will
               no cause problems.
            >> Modules that need to execute shutdown code before app 
               closes can register a Tcl proc to get called just after
               the root window is destroyed.
            >> No facility is currently provided for per-interpreter
               initialization (for when an app creates multiple interpreters).
            >> Clients wanting to make ambitious changes to Tk (e.g. change
               the event loop) will need to write their own main() proc.
            >> C++ users will need to compile main() under C++, or rename
               the Tcl main to something else and then call it from a user
               main() the was compiled under C++.
              
4. Dynamic Linking
    Goals:
      * Avoid proliferation of binaries.
      * More flexible: can add new packages dynamically without recompiling.
      * Shared libraries save memory.
    How?
      * New Tcl command:
        "load libraryName initProc"
        >> It looks as if this man be further simplified to be just
           "local libraryName"; the C initProc name will be automatically
           derived from the name of the library.  e.g. libdp.a -> dp_Init()
      * J.O. will solicit implementations for various systems and include
        them in Tcl releases.
      * Auto-load support (next slide).
      * Must resolve differences in how to compile shared libraries 
        for different systems.
      >> J.O. surveyed the audience for Unix platforms that supported
         dynamic linking.  Most (AIX, Solaris, OSF, HPUX, NeXTSTEP) support
         "shared-libraries" but fewer (Solaris, NeXTSTEP...) support
         dynamic linking.  SCO Unix supports neither.  This list obviously
         needs to be further flushed out on the news group.
         
5. Changes to Auto-Loading
    Current approach:
      * tclIndex files have fixed format:
        tl_dialog    dialog.tcl
        (proc name)  (file to source)
      * index files are parsed, not evaluated.
    New approach for Tcl 7.0
      * Index files will be evaluated:
          set auto_index(tk_dialog) \
            "source $dir/dialog.tcl"
      * Result: 3-4x faster, more flexible.
        >> Faster because Tcl interpreter (written in C) used to do parse
           rather than a parse routine written in Tcl.
      * Should accommodate TclX style of auto-loading?
        >> TclX guys said they would support Tcl7.0 form
      * Can invoke "load" instead of "source" to auto-load
        shared libraries.


