======================================================================== 453 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 2652; Fri, 29 Jun 90 20:09:46 EDT Date: Fri, 29 Jun 90 17:08:37 PDT To: "MARK HINNEBUSCH, FCLA" Sender: Distribution List Reply-to: Distribution List From: "Lennie Stovel" Subject: Z39.50 Implementers Group meeting notes, 6/24/90-6/25/90 Z39.50 Implementers Group Meeting Stouffer Concourse Hotel St. Louis, Missouri June 4-5, 1990 Final Agenda 1. Name of this Group 2. Status Reports 3. Management of Changes, Protocol Changes 4. PICS Proforma Review and Adoption 5. Profile Review and Adoption 6. Document Numbering Scheme 7. Scheduling/Activation of Subcommittees 8. Network Layer Addressing 9. Diacritics and Alternate Character Sets 10. Images, Graphics, Fonts, Large Data Objects 11. Z39.50 Over SNA LU6.2 12. Revisit Layering Over TCP/IP 13. Explain/Help 14. Document Identifiers 15. Add Number of Result Sets to Init Response PDU 16. Make Present a Separate Service 16a. Various Protocol Enhancements 17. Union Catalog Records 18. Search Values for Language in the Attribute Set 19. New Attributes 20. Extension to Retrieve a Set of Attribute Values 21. Stoplists 22. Review Attribute and Error Message Sets 23. Resource Control Request Records Standardization/Registration 24. Testing and Implementation 25. Availability of Code (Code Sharing) 26. ACM SIGCOM 27. Changes to Protocol 28. Revisit ZIG 90-007 and Other Issues 1. Name of the Group The name of the group will be the "Z39.50 Implementers Group". 2. Status Reports (Highlights) Florida Center for Library Automation They will be acquiring the IBM OSI/CS product before the end of June . They are now reading the manuals. Detailed design will follow. University of California/Penn State University In the design phase of their joint project. Coding will begin soon. Penn State is converting from a proprietary protocol. University of California, Berkeley They are working with Thinking Machines on full-text retrieval project. It is anticipated that coding will be completed by the end of summer (1990) of full-text search and retrieval over TCP/IP (a crude implementation). OCLC OCLC is going to send a copy of their Z39.50 software to the National Library of Canada (in approximately 6 months), the University of California, and Penn State. Ralph stated that their Z39.50 implementation (a Z39.50 variant) would be available for testing (as a Target) by the 4th quarter of 1990 (over OSI stack). They hope to also be available for testing with their TCP/IP stack at the same time. Thinking Machines They have a working subset of Z39.50 on a Mac, talking to a local server. They are working on a TCP/IP connection to a Connection Machine. Their code is available; part is pure Z39.50; part is for Type 3 query. They have not described in ASN.1. They have used the User Information field for their extensions. Virginia Tech In design stage. In 30-45 days working model will be completed at high level. Carnegie-Mellon Running on Unix servers and clients under the Motif interface, using OCLC software and OCLC query. National Library of Canada Doing feasibility study with UTLAS, which will be completed in July. Results will be made available to the Z39.50 Implementers Group. They will be receiving Z39.50 software from Canada in approximately 6 months. UTLAS Will implement Z39.50 as a Target only. They are not planning to implement as an Origin (at least not for now). NOTIS They are planning to implement a "Z39.50 entry way" during the next two years. For now, they will be controlling both sides of the communication (i.e., Origin and Target, a NOTIS-to-NOTIS link) over TCP/IP. Most of the participants stated that they should have something implemented in 12-24 months. Lennie Stovel mentioned that LITA (at ALA) would be discussing Z39.50 implementations and the Internet. (June 25, 1990, 9:00-11:00) 6. Document Numbering Scheme Documents will be numbered as follows: ZIG 90-001 Documents containing descriptions of implemented variances to the protocol will be numbered as follows: ZIGE-1 ("E" as in "experimental"). Documents will be available electronically. (Thinking Machines will set this up.) Instructions on how to access these documents will be posted to Z39.50 email network. Mark Hinnebusch will assign both types of numbers. 4. PICS Proforma Review Dennis MacKinnon had a number of comments (see ZIG 90-008) as did others. The group went through the document page by page. Changes are noted below. The Z39.50 maintenance agency (LC) will act as editor of the proforma. Section 6.1: Note what version 2 is likely to include (changes to make Z39.50 compatible with ISO Search and Retrieve standards). Section 6.2: It was proposed that we drop "Types 2 and 3" and state "Other types". Section 6.4: Suggested listing other syntaxes. Dennis will supply text. Section 6.6: Change "Pro" column to "o". USMARC is not mandatory. Section 6.7: "Appendix D" will be replaced by SR equivalent. Ralph is going to head an interest group on diagnostic codes, and will collect suggested additions to the list. Lennie proposed addin g a number of the "S3" variants as separate diagnostics. Additions will be sent to Ralph and he will edit and send on to Ray. We agreed that we should not form subgroups. Rather we should take care of interest group topics by sharing ideas in the network. The interest groups will each have a discussion "provoker". For example, Ralph will provoke diagnostics discussion, etc. Section 7.1.1: Change "Pro" column to "ns" for both Max-message and Max-record size. Add an item for "Query size" as "ns". Section 7.1.2: The values "F" and "B" are also possible in the "Simple form", therefore, revise text to clarify this point. Section 7.2: Change "Pro" column to "o" for Delete, Access-Cntrl-Resp, and Rsrc-Cntrl-Resp. Ray will fill out PDUs for Delete, Access Control, and Resource Control in PICS. Section 7.3.3: Change "Pro" column to "o" for MedStElemSetNms and PrefSyntax. Section 7.3.3.1: Change "Pro" column to "o" for named resultSetName xmitted. Section 7.3.3.2: In "Pro" column change "n's" to "o's". Section 7.3.5: Change "n" to "o" for PrefRecSyntax. Section 7.3.7: "Pro" column = "o", "m", "o". Section 7.3.13: Section was suggested to contain "constraints on User Info Field". After discussion, it was felt that a better place was in the Init APDU and in the Init Response. In Origin Transmission of Init, this would be expressed as "uses" of user information field. In Origin Receipt of Init Response, this would be expressed as "constraints" on user information field. Section 8.1.1: Change "Pro" column values to "ns". Section 8.1.2: Add "Other" values of ElementSetNames. Section 8.2: Change "Pro" column to "o" for Delete, Access-Cntrl-Resp and Rsrc-Cntrl-Resp. Section 8.3.3.1: Change "Pro" column values to "ns". Section 8.3.3.2: Change "Std" column and "Pro" column as follows: Std Pro named resultSetNames supported o o max concurrent result sets ns ns result sets not unilaterally o o Section 8.3.3.3: Change "Pro" column values from "na" to "o". Section 8.3.3.4: Add "Other". Section 8.3.4.1: Change "Pro" column values to "ns". Add two new sections for Vendor Notes and for Site-specific Information. Ray will make changes and distribute via electronic mail. 5. Profile Document (ZIG 90-001) Review There was some discussion about a possible extension to the standard. There was interest (Dennis MacKinnon) in having Target transmit end-of-session information when session is terminated. This information would include resource control type information. Ralph LeVan stated that he would rather have information at any time, not just at end of session. Section 8.6: Delete. Considered redundant. Document adopted as edited. 3. Management of Profile Changes, Protocol Changes There is a desire to "fix" a "Version 2" in time and distributing it, so that implementations can talk to each other. This would perhaps include some of the changes that the group wants. After discussion it was agreed that some "experimental" implementations would precede the standard. They should be distributed (posted) as ZIGEs. The FTP server would be the holder of the "experimental" description. Consider Option bit assignments (additional "experimental" ones) to be temporary. The maintenance agency will coordinate the bit assignments. There is a desire to add some attributes to the Bib-1 list. 9. Diacritics and Alternate Character Sets 10. a. b. and c. Images, graphics, fonts 11. Z39.50 over SNA LU6.2 Mark requested that these be removed from the agenda (since they were his suggestions, and time was in short supply). Nobody objected. 7. Standing Committees We agreed that we should not form subgroups. Interest group topics would be discussed by sharing ideas in the general Z39.50 network. The interest group s will each have a discussion "provoker". I. Diagnostics (additional) -- Ralph LeVan (OCLC) II. Database Records (e.g., bibliographic, holdings, circulation, text files, etc.) This "interest group" will be in two parts: 1. Every type of record except full-text -- Mark Hinnebusch (Florida) 2. Full-text -- Franklin Davis (Thinking Machines) III. Large data objects -- Clifford Lynch (UC) (ZIG 90-007) IV. Additional search attributes -- Clifford Lynch V. ASN.1 and bib records -- Ralph LeVan 8. Network Layer Addressing Ray distributed ZIG 90-006, an extract from a GOSIP document, and explained the concept. No significant discussion. 10.d. Large Data Objects Carnegie-Mellon must transfer several megabytes of data in one record (graphic data). If transferred as one record, they have been having performance problems on busy network (TCP level errors, e.g., having segments retransmitted, throwing packets away, etc.). They wanted flow control at Applications Layer. Now, they use Maximum-message size as fragment indicator. In addition, they are using element set name in "creative" ways. Clifford Lynch distributed ZIG 90-007. He will be the provoker of ZIG 90-007-related discussion. It was suggested that the Help/Explain could be used to communicate some of this information. Discussion will continue via email with ZIG 90-007 as starting point. 12. Revisit Layering Over TCP/IP Should Transport Class 0 be inserted between Session and TCP/IP (a general proposal--RFC 1006--in Internet)? It was decided that there was no advantage to adding TP0 under OSI Session, Presentation, and Application. Clifford Lynch will reconfirm this on the listserv. 13. Explain/help Service ZIG 90-009, describing extensions to SR to support an Explain service, was distributed by Clifford Lynch. Clifford wants feedback on the document. (Possible expansion, etc.) 14. Document Identifiers This item was suggested by email from Brewster Kahle. We will continue discussion on electronic mail. 15. Add Number of Result Sets in Init Response PDU It will be recommended to maintenance agency that a field be added to the Init PDU that specifies the number of result sets supported by Target. When is a Target considered stateless? When zero result sets are saved and Present is not supported. The majority of those present felt that it should be possible to not support Present Service. Franklin Davis (with some help from his friends) will prepare a case for a protocol change making it possible to have stateless machines and submit it to maintenance agency. It was agreed that the Init Response was the best place for this information. Many want to be able to state that their system cannot guarantee that any result sets will be saved (i.e., a value equal to "indeterminate" or unknown). It was proposed that this could be assumed if the element value was not included . This will require new text on page 9 of Z39.50 (conformance). There are two issues: 1. Protocol mechanism needed for statelessness 2. The conformance issue Franklin will put together a ZIGE (ZIGE 1) and distribute (post) it. 16. Make Present a Separate Service Present is currently defined as a separately-supportable service. 16.a. Various Protocol Enhancements Thinking Machines' Type 3/Relevance Feedback query (related works as searc h terms). Two types: 1. Related works as search term (document ID included in search term) 2. Keywords are weighted terms present in the text of the documents, but are NOT specified in search term. It was suggested that attributes be added, if possible, to type 1 query to cover these requirements. Franklin will prepare ZIGE 2 -- Attribute Extensions needed for Relevance Feedback Query (and other attribute extensions). OCLC would like to include an attribute for browse. It was stated that TC46 folks have been assigned to work on this. When a draft comes out of TC46, it will be distributed (as a ZIG). OCLC would like to be able to request a "human formatted record" (meaning non-USMARC, only understood by a human being, not a machine). This record would be a printable string that would be displayed as received. It was suggested that record could be described as printable string with understanding that each occurrence is displayed as a separate line (a temporary format type). These would be used if Origin system cannot process Target system records. Some present suggested that a value be added to indicate preferred record syntax. Dennis will see whether ODA (Office Document Architecture) documentation provides any help. 17. Union Catalog Records Clifford stated that he was simply interested in hearing what types of records libraries were going to be putting in the result sets. Lennie stated that RLIN was NOT going to send primary cluster record only, but will send the entire cluster. The holdings will be embedded in each record. It was suggested that the database_name be used to select primary cluster members only. Ralph (OCLC) stated that their holdings database was separate. If searching OCLC, holdings for a particular record can be retrieved by sending a search for a particular record "AND OCLC_Holdings_Database" (i.e., include database_name for the holdings file). DLA is considering whether to provide separate records for each holding unit, or a base record, with or without embedded holdings. FCLA will be sending separate records. This discussion will be continued in electronic mail. 18. Search Values for Language Clifford stated that some systems do not use the USMARC language codes. However, after discussion, it was thought that most systems would carry the USMARC language codes in the 008 field. When an external list is used (e.g., USMARC Language Codes), this information should be generally available. Possibly there should be a search attribute that refers to a specific list. 19. New Search Attributes What is the "scope" of Bib-1 attribute list? Is there a statement? What does bibliographic database mean? Can Bib-1 be used to search other database types? See also the Software Kinetics comments in ZIG 90-008. OCLC is interested in doing Patent Database searches using Bib-1 attributes , but they feel that they would need to add "inventor", which they feel is different from "personal name". Thinking Machines' attributes could be added (e.g., "score" or "relevance", i.e., how well did the document match the query). Discussion will continue over electronic mail. Clifford Lynch will be the provoker. There was some discussion on the topic of brief records. Dennis MacKinnon does not want brief records to be transferred in USMARC, but wants to use another transfer syntax. He said that he would be distributing a description of the brief record they wish to use in ILL as an example. 24. Testing OCLC -- Ready during 4th quarter of 1990 (OSI stack) Hope to be ready with TCP/IP by 4th quarter as well. Version = variant of Z39.50. Thinking Machines -- End of summer (TCP/IP). Version = Z39.50 (not using ASN.1 initially). Carnegie-Mellon -- End of year (TCP/IP). Penn State/Univ. of Calif -- 1st quarter of next year (TCP/IP). UC Berkeley -- End of summer (TCP/IP). 25. Code-sharing Thinking Machines' file server will be the repository for sharable source code. NEXT MEETING The next meeting will be in Washington, D.C. The meeting following that will be on the West Coast. The date of the next meeting was tentatively set for September 13 and 14 (Thursday and Friday). ZIG Document List ZIG 90-001 Application Profile for Z39.60. March 1990. (First Draft) ZIG 90-002 Profile PICS Proforma for Z39.50. March 1990. (First Draft) ZIG 90-003 Z39.50 Maintenance Agency Terms of Reference and Procedures. November 1989. ZIG 90-004 Documentation - Search and Retrieve Service Definition. ISO/DIS 10162. ZIG 90-005 Documentation - Search and Retrieve Protocol Specification. ISO/DIS 10163. ZIG 90-006 5. Addressing Requirements. [Extract from GOSIP documentation] ZIG 90-007 Image and Full-text transfer under Z39.50 -- issues for discussion [Clifford A. Lynch, June 1990] ZIG 90-008 Contributions on: Z39.50 PICS Proforma, Agenda Items, Stop Words, Z39.50 Profile, Attributes. D. MacKinnon and J. Zeeman, Software Kinetics Ltd. June 1990. [5 documents] ZIG 90-009 Extensions to ISO DP 10162/10163 to Support an Explain Service. Clifford A. Lynch, December 9, 1989. Attendees: Margaret Baker, UC Berkeley Ralph LeVan, OCLC Jean-Eudes Beriault, NLC Clifford Lynch, UC DLA Franklin Davis, Thinking Machines Dennis MacKinnon, Software Kinetics Ray Denenberg, LC Jim Michael, DRA Larry Dixson, LC Bill Mooney, UTLAS Sean Donelan, DRA Mark Needleman, UC DLA Tom Dopirak, Carnegie-Mellon Sara Randall, NOTIS Eric Ferrin, Penn State Curtis Shields, Virginia Tech Bob Fleischer, DEC Lennie Stovel, RLG Mark Hinnebusch, FCLA Randy Williams, DRA John Kunze, UC Berkeley Notes prepared by Larry Dixson and edited by Lennie Stovel. To: Z3950IW@NERVM.BITNET ======================================================================== 32 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 8866; Thu, 28 Jun 90 12:12:48 EDT Date: Thu, 28 Jun 90 12:11:40 EDT Sender: Distribution List Reply-to: Distribution List From: Franklin Davis To: "MARK HINNEBUSCH, FCLA" Subject: Thinking Machines Z39.50 ftp server You can now ftp Z39.50 documents from 131.239.2.1 think.com Login as "anonymous" with any password (normally your own login.) Change directory to Z3950 (or z3950): cd Z3950 The following subdirectories exist: drwxrwxr-x 2 fad 512 Jun 7 14:25 Code drwxrwxr-x 2 fad 512 Jun 11 19:39 General drwxrwxr-x 2 fad 512 Jun 7 14:35 ZIG -rw-rw-r-- 1 fad 1683 Jun 7 14:30 mail-list.doc -rw-rw-r-- 1 fad 1323 Jun 7 14:28 mail-list.doc~ If you have any problems please contact me. --Franklin franklin a davis Thinking Machines Corp. Cambridge, MA 02142 617-876-1111 {ames, harvard, mit-eddie, uunet}!think!fad ======================================================================== 156 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 7061; Mon, 11 Jun 90 15:17:39 EDT Date: Mon, 11 Jun 90 15:10:10 -0400 Sender: Distribution List Reply-to: Distribution List From: yeongw@spartacus.psi.com To: "MARK HINNEBUSCH, FCLA" Subject: Re: RLG position on Schoffstall & Yeong article Cc: schoff@psi.com, yeongw@psi.com, z3950iw@nervm.BITNET Gentlemen, We thank you for your message on our paper, published in the April issue of Computer Communications Review. It is clear that we differ in our evaluation of the Z39.50 standard and protocol. You point out in your own reply what is perhaps Z39.50's greatest failing: that it is incompletely specified in the service and protocol specification. As you yourselves admit, "it is necessary for a community of implementors and users to agree on how they will utilize a protocol." While this may be acceptable in the OSI community, it is most emphatically not acceptable in the Internet community, which incorporates TCP/IP technology and mature OSI technologies such as the Directory. The Internet Request for Comment mechanism exists in part to provide the definition of Internet standards, not to shore up weak existing standards. In its standards definition role, the RFC process is analagous to the OSI standards process, not the OSI PICS series of documents. There is no analogue to the PICS documents in the Internet community, because in the Internet community, standards documents are supposed to completely specify standards. With regard to the use of ROS, we continue to maintain that its incorporation into the Z39.50 standard would be advantageous. As we argued in the paper, ROS provides a valuable abstraction for transactional services that Z39.50 needs. It is for the purposes of utilizing that abstraction that we suggest the use of ROS, for after all, the very purpose of layering and the OSI stack is to abstract and hide the details of underlying service realizations. You are absolutely right in pointing out that "the Z39.50 standard itself does not constrain the form or content of records in any way". No, that was one of the things left to an implementors agreement, in which it was represented to us by both OCLC, our partners and yourselves (RLG) that the USMARC format was the "accepted" way of representing bibliographic data. We may have erred in our paper by ascribing the MARC format requirement to the standard itself, and not to an implementors agreement, but we continue to stand by our position that the MARC format is haphazard and redundant. To argue which one, the standard or the implementor's agreement required the use of MARC is to quibble over semantics. As you point out yourselves, both are required to completely specify the standard and make it useful to users. To that end, we maintain that the Z39.50 standard incorrectly specified the format for bibliographic data representation during information exchange. Furthermore, we continue to maintain that the Type 1 Query is incorrect. The Reverse Polish Notation is, as you point out, a means for the expression of a grammar. In this case, the grammar in question should have been the arithmetic grammar, which we feel was incorrectly translated. Specifically, the termination transition of the grammar S --> a with 'a' a terminal was incorrectly replaced by two transitions S --> b b --> a with the second transition being an "epsilon transition" (null transition, requiring no input to effect the transition). Note that to avoid being mired down by the semantics of "operand" and "argument", the innocuous literals 'a' and 'b' were used, with 'a' being synonymous to "operand" and 'b' being synonymous with "argument". After due consideration of that portion of your response pertaining to element sets, index terms and diagnostic records, we stand by our criticism that they should have been specified in the standard, and not left to private agreement. We understand, and have always understood that element sets and index terms are dependent on the nature of the database. However, to put it bluntly, bibliographic information is bibliographic information, and it does not seem to be particularly relevant as to whether the information is in an OCLC database, an RLG database or a Library of Congress database: we feel that the sheer range of data found in each database dwarfs any differences there may be between information from different databases. Thus we continue to maintain that it should be possible to define common element sets and common index terms that may be used to search any bibliographic database regardless of which database provider owned the database. The same, we feel is true of diagnostic records. Regardless of which database is being searched, the same protocol is being used to do the retrieving, Z39.50. And again, the same general type of data is being retrieved, bibliographic data. In view of this, we continue to feel that the lack of a common diagnostic record in the protocol is an omission that should be rectified immediately, while its lack of such a common format is a serious enough deficiency to preclude further acceptance of the protocol by the user community. We do not dispute that piggybacking functionality is useful in a protocol, but only that it will be not be useful in a realistic service where users are charged for bibliographic data retrieved. Gentlemen, you state that the piggybacking functionality is used everyday in your production implementation. We believe you. However we are forced to inquire as to exactly how your production implementation is being used, and by whom, to effect the retrieval of bibliographic information. Specifically, are users of your production implementation being charged for each unit of bibliographic data being retrieved? Piggybacking is an excellent, and as we ourselves stated, a time-honored optimization in protocol design. It fails in this case only because of the charging mechanisms that would be in place in a realistic service, when users would be penalized (by being charged) for an implementation guessing wrongly. We concede that the current APDU tags require the same number of bits to be encoded as they would if they started at 0, but that would appear to be only a fortunate accident caused by the existence of a particular number of Z39.50 APDUs: if even one more APDU existed, the tag for that APDU would require an extra 8 bits to encode in the BER, which would not be the case if APDU tags started at 0. After examination of clauses 3.2.3.1.4 and 3.2.3.1.6, we find that diagnostic records and the present status are defined, but continue to fail to see any definition of the action to be performed if an origin attempts to retrieve from a nonexistent result set. Specifically we continue to be unable to find any statement to the effect that under such conditions, the actions you describe in your response should be taken. The state machines defined in clause 4.2.3 of the Z39.50 specification certainly exist, but are unfortunately incomplete. As stated in the protocol specification, second sentence of the second paragraph of clause 4.2.3, "a blank cell represents the combination of an incoming event and a state that is not defined in the IRPM". It is exactly our contention that there are several such "blank cells" in the IRPM, and thus that the protocol machine may transition to unknown state. Perhaps one problem with the specification of Z39.50 lies in the mechanism by which it was specified itself. In your conclusion you describe the process by which Z39.50 was developed, and assert that the procedures assure review by all interested and affected parties. We dispute that assertion: the standards process you describe is often times closed to the eventual end-users of implementations of a protocol, precluding what may be the most important community of all, the community which would be most affected in the end by a protocol specification. For example, examination of the membership of NISO reveals that for the most part, it is made up of libraries, publishers and standards bodies, with no end-user representation whatsoever. In closing we thank you for the continued interest in Z39.50 and for the response to our paper. While we continue to feel that Z39.50 is poorly specified, we are gratified that work continues on the development of the standard. We sincerely hope that eventually bibliographic information retrieval over the network will be a reality, but will endeavor to ensure that the current specification progresses no further in the standards process. ======================================================================== 55 Received: from RLG.BITNET by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 6395; Mon, 11 Jun 90 14:16:44 EDT Date: Mon, 11 Jun 90 11:12:51 PDT To: fclmth@NERVM.BITNET From: "Lennie Stovel" Subject: ZIG document list - Lennie's version Mark, Franklin, Ray, It wasn't clear to me who is supposed to be the assigner of ZIG numbers. Is it the maintenance agency (Ray), the file server maintainer (Franklin) or the chair (Mark)? Whoever it is, here's the list of ZIG documents that I put together. Whoever is in charge, let me know if this is the right list. After that, it's YOURS. Also, who is keeping track of ZIGE numbers? I think that's supposed to be Ray. Do you agree? -- Lennie -------------------------------------------------------------- Z39.50 IMPLEMENTERS GROUP (ZIG) Documents List ZIG 90-1 Application Profile for Z39.60. March 1990. (First Draft) ZIG 90-2 Profile PICS Proforma for Z39.50. March 1990. (First Draft) ZIG 90-3 Z39.50 Maintenance Agency Terms of Reference and Procedures. November 1989. ZIG 90-4 Documentation - Search and Retrieve Service Definition. ISO/DIS 10162. ZIG 90-5 Documentation - Search and Retrieve Protocol Specification. ISO/DIS 10163. ZIG 90-6 5. Addressing Requirements. [Extract from GOSIP documentation] ZIG 90-7 Image and Full-text transfer under Z39.50 -- issues for discussion [Clifford A. Lynch, June 1990] ZIG 90-8 Contributions on: Z39.50 PICS Proforma, Agenda Items, Stop Words, Z39.50 Profile, Attributes. D. MacKinnon and J. Zeeman, Software Kinetics Ltd. June 1990. ZIG 90-9 Extensions to ISO DP 10162/10163 to Support an Explain Service. Clifford A. Lynch, December 9, 1989. To: MARKH(FCLMTH@NERVM), FAD@THINK.COM, BB.RAY ======================================================================== 35 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 0412; Thu, 07 Jun 90 14:39:08 EDT Date: Thu, 7 Jun 90 14:39:22 EDT From: Franklin Davis To: FCLMTH@nervm.nerdc.ufl.edu Subject: ZIG repository I've created the ZIG repository. I'd like someone to try it out before I post it. It's at think.com (131.239.2.1). There is a main directory Z3950. There are subdirectories Code, General, and ZIG. There is also a copy of mailing list documentation. Code now contains a link to our code. General contains the RLG comments on the ACM paper. I may find the other comments and put them there. ZIG has numbers 1 and 2. I don't know if I have any others machine-readable. Take a look at the README file, and send me suggestions. I had a great time! There sure are a lot of things to do on my list... Well, onward. One question: did we make any collective decision about different max-message-lengths for incoming vs. outgoing messages? I think we didn't. 'later, --Franklin ======================================================================== 104 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 0139; Thu, 07 Jun 90 14:15:06 EDT Date: Thu, 7 Jun 90 08:47:58 PDT To: "MARK HINNEBUSCH, FCLA" Sender: Distribution List Reply-to: Distribution List From: "Ray Denenberg - LC" Subject: ASN.1 for Access Control and Resource Control I brought copies of this to distribute at the meeting but we never got to this topic. Please comment by August 1. Proposed changes to ASN.1 description of Z39.50 (based on SR protocol) to incorporate Access Control and Resource Control. 1. CHANGE DEFINITION OF SR-APDU TO: SR-APDU ::= CHOICE {initializeRequest [20] IMPLICIT InitializeRequest, initializeResponse [21] IMPLICIT InitializeResponse, searchRequest [22] IMPLICIT SearchRequest, searchResponse [23] IMPLICIT SearchResponse, presentRequest [24] IMPLICIT PresentRequest, deleteResultSetResponse [27] IMPLICIT DeleteResultSetResponse, accessControlRequest [28] IMPLICIT AccessControlRequest,--new accessControlResponse [29] IMPLICIT AccessControlResponse, --new resourceControlRequest [30] IMPLICIT ResourceControlRequest, --new resourceControlResponse [31] IMPLICIT ResourceControlResponse --new -- new APDUs can be added in the future at the end of this list. 2. WITHIN THE SEARCH RESPONSE APDU, CHANGE DEFINITION OF ResultSetStatus TO: resultSetStatus [26] IMPLICIT PartialResults 3. ADD DEFINITIONS FOR THE FOLLOWING APDUS: -- 8.1.1.5 Access Control APDUs AccessControlRequest ::= SEQUENCE {referenceId ReferenceId OPTIONAL, securityChallenge [37] IMPLICIT OCTET STRING} AccessControlResponse ::= SEQUENCE {referenceId ReferenceId OPTIONAL, securityChallengeResponse [38] IMPLICIT OCTET STRING} -- 8.1.1.6 Resource Control APDUs ResourceControlRequest ::= SEQUENCE {referenceId ReferenceId OPTIONAL, SuspendedFlag [39] IMPLICIT BOOLEAN, -- "true" = suspended ResourceReport [40] IMPLICIT VisibleString OPTIONAL, partialResultsAvailable [41] IMPLICIT PartialResults OPTIONAL} ResourceControlResponse ::= SEQUENCE {referenceId ReferenceId OPTIONAL, ContinueFlag [42] BOOLEAN, -- "true" = "continue" ResultSetWanted [43] BOOLEAN OPTIONAL} -- "true" = "result set wanted" -- required during a search if Continue -- flag is false; otherwises should not occur 4.ADD TO GLOBAL AUXILIARY DEFINITIONS: PartialResults ::= INTEGER {subset (1), interim (2), none (3)} 5. DESCRIBE RESOURCE REPORT (CURRENTLY APPENDIX E OF Z39.50) AS FOLLOWS: ResourceReport ::=SEQUENCE {estimates [1] SEQUENCE OF Estimate, textMessage [2] VisibleString OPTIONAL} Estimate ==: CHOICE { currentSrchRslt [1] IMPLICIT INTEGER -- current result for search finalSrchResult [2] IMPLICIT INTEGER -- result if search completes currentPrsntRslt [3] IMPLICIT INTEGER -- current result for present finalPrsntRslt [4] IMP,ICIT INTEGER -- result IF present completes crntProcessngCmd [5] IMPLICIT INTEGER -- processing time used by command fnlProcessingCmd [6] IMPLICIT INTEGER -- total if command completes crntprocessAsoc [7] IMPLICIT INTEGER -- time used by association so far CurrentCostCmd [8] IMPLICIT INTEGER -- cost for this command so far FnlCostCmd [9] IMPLICIT INTEGER -- cost for command if IT completes currentCostAsso [10] IMPLICIT INTEGER -- cost for this association so far } ======================================================================== 18 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 1046; Fri, 01 Jun 90 10:01:44 EDT Date: Fri, 1 Jun 1990 09:57:28 EDT Sender: Distribution List Reply-to: Distribution List From: FCLMTH@NERVM (Mark Hinnebusch, FCLA) To: "MARK HINNEBUSCH, FCLA" Subject: Additional agenda items Fast revision: 17. Union catalog records 18. Search values for language in the attribute set 19. New attributes 20. Extension to retrieve a set of attribute values 21. stoplists 22. Review attribute and error message sets 23. Resource control request record standardization/registration ======================================================================== 28 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 0941; Fri, 01 Jun 90 09:56:23 EDT Date: Fri, 1 Jun 1990 09:47:55 EDT Sender: Distribution List Reply-to: Distribution List From: FCLMTH@NERVM (Mark Hinnebusch, FCLA) To: "MARK HINNEBUSCH, FCLA" Subject: Final Agenda 1. Name of this group 2. Status reports 3. Management of profile changes, protocol changes 4. PICS Proforma review and adoption 5. Profile review and adoption 6. Document numbering scheme 7. Scheduling/activation of subcommittees 8. Network layer addressing 9. Diacritics and alternate character sets10. Images, graphics, fonts, large data objects 11. Z39.50 over SNA LU6.2 12. Revisit layering over TCP/IP 13. Explain/help 14. Document Identifiers 15. Add number of result sets to init response PDU 16. Make Present a separate service Rather an ambitious agenda, to say the least. See you in St. Louis! ======================================================================== 196 Received: from NERVM by NERVM.NERDC.UFL.EDU (Mailer R2.05) with BSMTP id 5790; Fri, 01 Jun 90 17:55:58 EDT Date: 6/01/90 14.50.22 90.152 PDT Sender: Distribution List Reply-to: Distribution List From: To: "MARK HINNEBUSCH, FCLA" Subject: DLA comments of ACM/SIGCOMM article Comments on the Now-Infamous "A Critique of Z39.50 Based on Implementation Experience" by Schoffstall and Yeong. Here are some additional comments on this article. I have tried not to duplicate the previous comments from Wayne, Mark, and Ray, with which I basically agree. First, a few general points: The article alleges to be a critique based on implementation experience. Neither the implementation nor the experience is described in the article, and there is no discussion of anything likely to have been discovered by implementation. In fact, the article really reads as a critique based on a cursory reading of the Z39.50 protocol standard, without an understanding of the context within which it developed, the process by which it developed (which is quite different from common practice in the Internet world), or what is happening with the protocol today (e.g., ISO 10162/10163). Although the article is dated January 31, 1990, the authors seem unaware of recent events. For example, the bibliography lists the standard as unpublished, and in the introduction it is described a proposed, rather than actual standard. The article views Z39.50 as a static entity (circia about 1988), while critiquing it in the context of developments in the OSI world right up to the present (e.g., ASN.1, which really didn't emerge until around 1988, as I understand it, although the bibliography lists it as December 1987). The research is shoddy: There is no mention of the dozen or so articles written that are relevant to the subject at hand, including others on implementation experience. There is no discussion of other institutions (e.g., CMU) who are using it in much the same context as the NYSERNET project seems to have been cast. There is no discussion of the work of the National Library of Canada on use of ROS with Z39.50. The passive voice of the article frequently cites arguments and information that are unsubstantiated by sources. Next, specific points: 1. Introduction: It confuses what Z39.50 does with access to bibliographic data. The Z39.50 standard covers more than just bibliographic data. The whole premise of the article confuses the authors' dislike of practices for computer-based bibliographic records, issues in the use of the protocol in bibliographic retrieval environments, and problems the authors see in the protocol proper. These issues really must be addressed separately. Because the paper fails to discuss implementation experience, it necessarily fails to discuss what might have been learned from actual deployment of the implementation in an environment with multiple (independent) servers and clients. It is not clear that deployment of the implementation under discussion actually occurred (unlike the situations at RLG, OCLC, LC, and now developing at Project Mercury at CMU, for example). The model (many clients, few servers) has nothing to do with the protocol. Rather, this is the NYSERNet-envisioned implementation environment. I agree there are likely to be many more clients than servers, which generates questions about registry management, system design, etc.; however, it is not really a protocol-related issue. The authors confuse index names and index terms (e.g., search keys); they do not demonstrate an understanding of attribute sets and their registry. It would have been interesting to hear how they handled that in the actual implementation (which is not discussed). Side issue: has anyone on the list actually seen the NYSERNET implementation in operation? 2. ROS: I believe that the problems with ROS have nothing to do with "flaws based on implementation experience." This is a much more general issue. I believe there is a mythology afoot (perhaps in part coming from Marshal Rose's work on ISO-DE) that OSI applications = ROS. I can't find much basis to support this theory. Had the authors done their homework, they might have looked at the rather extensive work that the National Library of Canada has done in this area, as well as at the relationships between Remote Database Access, ROS, and Z39.50. There are substantial problems with child processes, and it seems likely that use of ROS would have been incompatible with at least some versions of the protocol while it was under development since it was not until a rather late draft that it became neatly transactional (as those on Committee D will recall). 3. Protocol Transfer Syntax/ASN.1: This has been dealt with by others. 4. MARC Format: This section is sufficiently confused and misinformed that I am not going to consider it point by point, other than to mention that ASN.1 encoding will not fix the problem about which the authors of the paper seem most disturbed: the large number of possible data elements in the description of a work under MARC. They are confusing encoding schemes and data element names. 5. Z39.50 Query Formats: I think these problems have been discussed already. Type 0 isn't a flaw; it is intended to allow the query submission, management, and record transfer part of the protocol to be used separately from the search format. The paper confuses conformance issues here with protocol design. 6. Element Sets and Index Terms: Most of the errors in this section have been mentioned. However, there is confusion between bibliographic searching and Z39.50, which is more general. Here is a particular example of the undocumented passive voice: "it is often argued that bibliographic data is inherently unstructured" -- Who argues? What of the MARC format, which provides just such structure? 7. Diagnostic Records: This section is just plain wrong, when compared to the standard. 8. Result Set Naming: In fact, a client can discover if a server supports result set naming by trying to create a named one and seeing if the client gets an error. It is true that it would be much better to pass this information along in the initiating process. 9. Cramming Functionality: It is true that DELETE isn't pretty, and might be tidier as two different APDUs. I don't understand the criticism that SEARCH offers two distinct functions. Can anybody else shed light on this? 10. Piggybacking: The main arguments against piggybacking seem to be that (1) information providers charge (which is not uniformly true, by any means), and (2) it is possible to write bad applications using this feature when the applications are used with databases that do recharge. This makes no sense. (As a side comment, piggybacking is even more useful than was originally thought since it opens the door to the development of "stateless servers," such as the work that Thinking Machines is doing.) 11. Nits: I think that most of the assorted errors in this section have already been addressed by others. I do think that comments such as "The Z39.50 protocol machine tends to transition into unknown states in general" seem inappropriate for a paper of this type, particularly when the state machine is defined by a state table. More useful would have been some discussion of real implementation issues, such as handling of illegal input (an issue that is always outside the realm of standards documents, but key in developing robust working implementations of standards). 12. Conclusions: These are beyond criticism. I think, however, it is worth trying to summarize conclusions one might draw from the paper. The ROS issue really, to my mind, is not of great significance. The use of ROS-type services is a mechanical choice that does not greatly alter the semantics of the protocol. I think that we all agree that ASN.1 encoding for the APDUs is the right thing to do, and we are doing it. The authors seem to be of the opinion that the MARC transfer format should be replaced by ASN.1, and that we need to start over from scratch in cataloging with a new (and much simpler) set of data elements. This, of course, has nothing whatsoever to do with the Z39.50 protocol. The "new improved" protocol should allow clients to get lists of supported element sets and attribute sets from servers. I agree that this would be a useful protocol extension and it is on the list of things to do in the published protocol. But this suggestion misses the point: The client needs to find out what index names the server supports with reference to standardized (registered) attribute sets, so that the client can associate semantics with these index names. It is not sufficient that the client learn only that the server supports an index name called XYZZY; the client needs to know if the server supports AUTHOR, PERSONAL AUTHOR, or CORPORATE AUTHOR, for example, so that it can attempt to map end-user searches into appropriate queries for a given server. This is a more difficult problem, but one that I think we need to address. It is not at all clear what the authors of this paper are really proposing, aside from redefining cataloging. Do we need to clean up ambiguities in the protocol definition? Do we need to add protocol extensions? (I think we all agree that we do, and the protocol clearly suggests that this is the intention.) Or do the authors think we need to start over again ("It is concluded that Z39.50 is inadequate as the basis for an end-user bibliographic retrieval service" does imply such a philosophy). One wonders what they feel the correct basis would be. Clifford Lynch.