THIS DOCUMENT HAS NOT BEEN UPDATED TO REFLECT CURRENT DESIGN!!!!
..................................................................


This document contains various notes for a redesign of the protocol
handling of LWP.  The goal is to be able to support HTTP/1.1 and
multiple connections and at the same time be able to support the old
interfaces.  The old interface will be built on top of the new
event-driven model.

All communication will be based on IO::Socket::INET or on the Net::
modules where they are applicable.  Perhaps we ought to build a
Net::HTTP module to deal with basic HTTP communication?  All timeout
handling should be based on a central event-loop.  No more alarm
calls.  When Perl get an official event-loop we will use it and I
believe that Nick is working on it together with Tk.  Using the same
event-loop is also a requirement for happy Tk/LWP based applications
in the future.

There can be multiple connections to the same or different servers.
Requests can be pipelined to HTTP/1.1 servers.

Marc's LWP::ParallelUA already do a lot of what has been planed.  This
is a much more abitious rewrite plan for LWP.

------------------------------------------------------------------

LWP::UserAgent

    %servers{"$host:$port"}
          # need $port since we don't index the protocol
    $scheduler

    $default_headers (UserAgent, From,....)
    $timeouts
    $cookie_jar... 
          # Might be generalized to something that modifies/monitor
          # requests/responses???)
          #   - a list of request hook objects and
          #   - a list of response hooks objects (objects that can
          # modify a request on the way out or a response before
          # it is delivered to the user.  These hook objects can
          # also extract information in order to use them for
          # later requests/responses.  Default headers, cookies,
          # and autentication info and proxying decisions
          # can all be implemented by these hooks.

    $proxies
       # also something to support proxy authentication

    # the main route for adding requests are:
    $ua->$scheduler->spool();
    
    # the old $ua->request ($ua->simple_request) interfaces will be
    # emulated somehow.  Something like this perhaps?
         $ua->$scheduler->spool($req);
         LWP::EventLoop::wait_for_completion();
         return $res;


LWP::Request
    # same as HTTP::Request, but also hold $ua->request parameters.
    # Should it then be a subclass or perhaps we should encapsulate
    # a HTTP::Request with this additional info.
    $arg       (undef, $file, \&callback)
    $size      (read size hint)
    $redirect  (redirect behaviour, scalar or callback)
    $prot      (might override protocol choosen, make HTTP/1.0 request to
                a HTTP/1.1 server or HTTP/1.1 request to an unknown server)
    $proxy
    $timeout


LWP::Server

    # This holds all the information that the UA needs to keep about
    # the servers it communicates with.  This is the place to remember
    # various state (not cookies) about authetication and capabilities
    # of servers.
    #
    # There should be some support of making (some of) this information
    # persistent between LWP runs.

    $netloc     'www.sn.no:8080'
    $protocol   "HTTP/1.0", "HTTP/1.1", "FTP",...  # whatever
    $pipeline   bool
    $keep_alive bool
    $max_connections    # limit connections to this server

    $last_request       time() (support nice robots and such)
    $num_requests       # counter (don't really know why we want this)

    @pending_request    # a queue of things to do
    @connections        # the active things

    # support authentication without unnecessary round-trips
    %credentials
    %realms

    $robot_rules        # if applicable


LWP::Connection (subclassed for various protocols)

    $server

    # These are attributes needed to support HTTP/1.1.
    $socket
    $current_request
    @pending_responses
    $current_response      $response (might be defined to be first of @pending)
    $last_activity         time()
    $pipeline              #send new request without waiting for response first
    $keep_alive            #seconds to live while idle
    $request_limit (max_request, decremented for each request)

    # When a connection has been created it will have a reference to
    # it's server.  It will then start eating of the @pending_request
    # queue.  Various parameters govern how greedy it will be.

    Class methods to kill all connections (user hit the big red button)


LWP::Auth

    # Knows about different authentications schemes.  Different subclasses
    # for Basic/Digest/....
        # will be loaded and invoked on demand.  Takes a response as
        # parameter and will then return a response or a request.  If
        # it is a request it will be rescheduled.  Perhaps we could
        # handle redirects with the same kind of module.  Something
        # which is invoked for any client errors or redirects.



LWP::EventLoop
   - register filehandles (mostly sockets) for readable/writeable callbacks
   - timeout callbacks
   - a start could be wrapper around Tk's MainLoop
       (I tried this but it looks like Tk don't like to have the
        same handle readable and writable at the same time.  This makes
        it almost useless for HTTP/1.1.)

   $loop->loop_until($something);  # let things happen for a while



LWP::Scheduler

   # I am not sure which one of these should be represented
   @active_servers
   @connections

   $ua  # makes callbacks (circular references are not good to have)
   $max_connections  # limit total number of connections from this client

   # this might in fact be the $ua, but a separate scheduler class
   # should make it easier to make subclasses that have a different
   # scheduling policy

   # These are called by connections when the corresponding event
   # happens.  This might result in rescheduling.
   $sched->no_more_pending($connection);  # but not closed (keep alive)
   $sched->closed($connection);

   $sched->spool($request);
       # will lookup server (create one if it does not exists) and add
       # this request as @pending_request (then call $sched->_sched)

   $sched->_sched
       # will pick some servers (with @pending_request) and create
       # connections for them.

   # The scheduler will also set up timer callbacks so that it will
   # sometimes wakeup and kill of idle connections (keep-alives).
          # perhaps the connections should take care of this
          # themselves.  yes, I think so!

   $sched->kill_server();  # kills all connections (and pending requests)
                           # to the given server.
   $sched->kill_all();
