



Network Working Group                                       M. Kühlewind
Internet-Draft                                                  Ericsson
Intended status: Informational                                   M. Duke
Expires: 18 June 2023                                             Google
                                                        15 December 2022


        Guidelines for the Organization of Fully Online Meetings
                   draft-ietf-shmoo-online-meeting-05

Abstract

   This document provides guidelines for the planning and organization
   of fully online meetings, regarding the number, length, and
   composition of sessions on the meeting agenda.  These guidelines are
   based on the experience with online meetings during the COVID-19
   pandemic in 2020 and 2021.

Discussion Venues

   This note is to be removed before publishing as an RFC.

   Discussion of this document takes place on the Stay Home Meet Only
   Online Working Group mailing list (manycouches@ietf.org), which is
   archived at https://mailarchive.ietf.org/arch/browse/manycouches/.

   Source for this draft and an issue tracker can be found at
   https://github.com/mirjak/draft-shmoo-online-meeting.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 18 June 2023.






Kühlewind & Duke          Expires 18 June 2023                  [Page 1]

Internet-Draft        Organization Online Meetings         December 2022


Copyright Notice

   Copyright (c) 2022 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
   2.  Some History  . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Guidelines for Online Meeting Planning  . . . . . . . . . . .   4
     3.1.  Time Zone Selection . . . . . . . . . . . . . . . . . . .   4
       3.1.1.  Guidelines for selection  . . . . . . . . . . . . . .   5
     3.2.  Number of Days and Total Hours per Day  . . . . . . . . .   6
     3.3.  Session/Break Length  . . . . . . . . . . . . . . . . . .   6
     3.4.  Number of Parallel Tracks . . . . . . . . . . . . . . . .   7
   4.  Additional Considerations and Recommendations . . . . . . . .   7
     4.1.  Full vs. Limited Agenda (and interim meetings)  . . . . .   7
     4.2.  Flexibility of Time Usage . . . . . . . . . . . . . . . .   8
     4.3.  Inclusivity and Socializing . . . . . . . . . . . . . . .   8
     4.4.  Experiments . . . . . . . . . . . . . . . . . . . . . . .   9
   5.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .   9
   6.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     6.1.  Normative References  . . . . . . . . . . . . . . . . . .   9
     6.2.  Informative References  . . . . . . . . . . . . . . . . .   9
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

1.  Introduction

   In 2020, the COVID-19 pandemic forced the IETF to convert all its
   plenary meetings to online-only events.  This document records the
   experience gained by holding plenary meetings fully online and
   proposes guidelines based on this experience.  In general,
   participant surveys indicated satisfaction with the organization of
   these meetings.

   Although these guidelines reflect lessons learned in 2020 and 2021,
   the IETF is encouraged to continue to experiment with the format and
   agenda of fully online meetings, using this document as a baseline.



Kühlewind & Duke          Expires 18 June 2023                  [Page 2]

Internet-Draft        Organization Online Meetings         December 2022


   Hybrid meetings (meaning meetings that have large remote
   participation but also onsite participation) are out of scope.
   However, some of the experience gained from fully online meetings
   might also provide input for decisions regarding the organization of
   hybrid meetings.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   This document uses the term "plenary meeting" for the whole IETF
   meeting that covers the IETF meeting week; this term is used to
   distinguish the plenary meeting from other IETF meetings like
   "interim meetings".  The term "administrative plenary" is used for
   the respective session during the IETF meeting week that is usually
   hosted on Wednesday.

2.  Some History

   When the World Health Organization (WHO) declared a world-wide
   pandemic in March 2020, the IETF canceled its plenary meeting and
   organized an online replacement in less than two weeks.  For this
   first online-only meeting, the agenda was reduced to a set of
   sessions that benefitted most from cross-area participation, like
   BoFs, first-time meetings of new working groups, and dispatch
   sessions.  It also included the administrative plenary to preserve
   the official hand-over procedures that occur at the March meeting, as
   described in [RFC8713].

   With a reduced agenda, the meeting format was 2 sessions (about 4
   hours) per day with a maximum of two parallel tracks.  Other working
   group meetings were scheduled as interims over the following six
   weeks.  The IESG published a purely advisory recommended schedule
   [INTERIM-SCHEDULE] to reduce conflicts among those interims.

   While satisfaction was high right after the meetinng [_107-FEEDBACK],
   some participants later indicated in mailing list discussion that the
   period of intensive interims had a greater impact on their calendar
   than a single plenary meeting week, and in some meetings
   participation was reduced.  Those interims tended to occur at times
   convenient for the bulk of participants, which was convenient for
   most but could exclude those in less common time zones.





Kühlewind & Duke          Expires 18 June 2023                  [Page 3]

Internet-Draft        Organization Online Meetings         December 2022


   For the remainder of 2020 and 2021, the online schedule was switched
   back to be similar to an in-person meeting (1-2 hour slots and 8-9
   parallel tracks).  However, each day was limited to 5-6 hours in
   recognition that remote participation is more tiring.

   All fully online meetings followed the time zone of the planned in-
   person meeting location.  As a six-hour agenda has some flexibility
   regarding the start time while still fitting within a previously used
   8-hour in-person agenda, the start time was approximately noon, with
   adjustments of an hour or so to mitigate the impact of early morning
   hours in time zones with many participants.  As selection of in-
   person meeting sites was consistent with the 1-1-1 guideline as
   documented in [RFC8719], this approach was intended to share the
   burden across all common geographies roughly equally.

3.  Guidelines for Online Meeting Planning

3.1.  Time Zone Selection

   The following algorithm was not used in 2020 or 2021, but enables
   most participants to avoid late-night sessions in 2 out of every 3
   fully online IETF plenary meetings.  Basically, every fully online
   meeting is for two regions of the three regions described in
   [RFC8179], with one being roughly after sunrise and the other around
   sundown.  This has the tradeoff that the third region is in the
   middle of night.

   The times are also seasonally adjusted to leverage differentials in
   Daylight Saving Time.  These time slots are as follows, in UTC, based
   on the Daylight Saving Practices at the time of publication:

   +===============+=========================+=========================+
   | Name          | Times (Northern Summer) | Times (Northern         |
   |               |                         | Winter)                 |
   +===============+=========================+=========================+
   | North America | 0500-1100 UTC           | 0600-1200 UTC           |
   | Night         |                         |                         |
   +---------------+-------------------------+-------------------------+
   | Asia Night    | 1300-1900 UTC           | 1400-2000 UTC           |
   +---------------+-------------------------+-------------------------+
   | Europe Night  | 2200-0400 UTC           | 2200-0400 UTC           |
   +---------------+-------------------------+-------------------------+

                                  Table 1

   Note that the "European Night" slot covers the "Early Morning" slot
   for Asia where most countries do not have Daylight Saving Time.




Kühlewind & Duke          Expires 18 June 2023                  [Page 4]

Internet-Draft        Organization Online Meetings         December 2022


   If Daylight Saving Practices change, which is under consideration in
   multiple countries at the time of publication, this table may need
   adjustment.

   The intent of rotating between these three slots is to scatter
   meetings throughout the course of the global day, to maximize the
   ease of participants so that no attendee has to be consistently
   inconvenienced, regardless of their location and what time of day is
   optimal for their schedule.  However, as participation is distributed
   globally, it needs to be acknowledged that restricting the scheme to
   three regions observes the intent of [RFC8179] but does not achieve
   the goal of 2 non-late-night sessions for all participants equally.

3.1.1.  Guidelines for selection

   The IETF SHOULD select a start time from these three choices based on
   the prior three meetings.  The following table covers all
   permutations of previous meetings held in-person in Region A, B, or
   C; or remotely in the nights of one of those regions.

   +================+================+==============+==================+
   | 3 meetings ago | 2 meetings ago | Last Meeting | Online           |
   |                |                |              | Selection        |
   +================+================+==============+==================+
   | Any            | Any            | In-Person A  | A Night          |
   +----------------+----------------+--------------+------------------+
   | Any            | Online A Night | Online B     | C Night          |
   |                |                | Night        |                  |
   +----------------+----------------+--------------+------------------+
   | Online A Night | In-Person B    | Online B     | C Night          |
   |                |                | Night        |                  |
   +----------------+----------------+--------------+------------------+
   | In-Person A    | In-Person B    | Online B     | A Night          |
   |                |                | Night        |                  |
   +----------------+----------------+--------------+------------------+
   | In-Person A    | In-Person A    | Online A     | see below        |
   |                |                | Night        |                  |
   +----------------+----------------+--------------+------------------+
   | Online A Night | Online B Night | Online C     | A Night          |
   |                |                | Night        |                  |
   +----------------+----------------+--------------+------------------+

                                  Table 2








Kühlewind & Duke          Expires 18 June 2023                  [Page 5]

Internet-Draft        Organization Online Meetings         December 2022


   This table follows two basic guidelines: 1) Whenever a fully online
   meeting follows an in-person meeting, the online meeting time is used
   that most disadvantages most the participants in the time zone where
   the in-person meeting was held. 2) If multiple fully online meetings
   follow each other, the time zone selection should be rotated based on
   the most recent time zones that the in-person meetings were held in.

   The final case occurs in the rare event that back-to-back in-person
   plenary meetings occur in the same region.  In this case, find the
   most recent meeting that was neither in 'A' (if in-person) nor in 'A'
   night (if fully online).  If this meeting was in-person in region
   'B', then the next meeting should be in 'B' Night.  If it was remote
   in 'B' Night, the next meeting should be in 'C' Night.

3.2.  Number of Days and Total Hours per Day

   By 2021, fully online meetings were consistently held over 5 days
   with roughly 6-hour meeting days.  The day with the administrative
   plenary, which concludes with multiple open mic sessions, sometimes
   exceeded this limit.

   Six hours of online meetings, with two 30-minute breaks, was a
   compromise between the physical limits of attending an online meeting
   in an inconvenient time zone, and the demand for many sessions with a
   manageable number of conflicts.  The IETF 109 feedback [_109-SURVEY]
   indicated broad satisfaction with a 5-day meeting but only medium
   satisfaction with the overall length of each day.

   The IETF did not seriously consider extending sessions into the
   weekend before or after the main meeting week, although the Hackathon
   occupied the entire week before (see [RFC9311]).

3.3.  Session/Break Length

   For fully online meetings there are typically fewer sessions per day
   than for in-person meetings, to keep the overall meeting day to
   roughly 6 hours.  With fewer sessions, chairs were offered only two
   options for session length (instead of three).

   IETF-108, based on an indicated preference of the community,
   scheduled 50- and 100-minute slots, with 10-minute breaks, in order
   to keep the overall day length at 5 hours.  This resulted in many
   sessions going over time, which indicated that 10 minutes for breaks
   is not practical.

   The survey after IETF-109 [_109-SURVEY] showed high satisfaction with
   60/120-minute session lengths and 30-minute breaks, and a significant
   improvement in satisfaction over IETF-108.



Kühlewind & Duke          Expires 18 June 2023                  [Page 6]

Internet-Draft        Organization Online Meetings         December 2022


   The longer breaks, while extending the day, provided adequate time
   for meals, exercise, and "hallway" conversations using online tools.

3.4.  Number of Parallel Tracks

   In-person meetings are limited in the number of parallel tracks by
   the number of meeting rooms, but online meetings are not.  However,
   more parallel tracks increases the number of possible agenda
   conflicts.

   If the total number of requested sessions exceeds the capacity of the
   usual 8 parallel tracks, it is possible for a fully online meeting to
   simply use more tracks.  If the number and length of meeting days is
   seen as fixed, this decision is implicitly made by the working group
   chairs requesting a certain number of sessions and length.

   IETF-111 used 9 parallel tracks for some of the sessions, and
   experienced slightly more conflicts in the agenda scheduling process,
   though there was no statistically significant increase in
   dissatisfaction about conflicts in the survey [_111-SURVEY].

   The IESG encouraged working group chairs to limit their session
   requests and use interim meetings aggressively for focused work.

4.  Additional Considerations and Recommendations

4.1.  Full vs. Limited Agenda (and interim meetings)

   The IETF-108 meeting survey [_108-SURVEY] asked about the structure
   of that meeting (full meeting) compared to that of IETF 107, which
   hosted only a limited set of sessions followed by interims in the
   weeks after.  The structure of IETF 108 was preferred by 82%.
   Respondents valued cross-participation and an intensive meeting week
   for maintaining project momentum.

   Furthermore, a well-defined meeting time, rather than spreading many
   interims over the whole year, can make deconflicting with other non-
   IETF meetings easier.

   However, interim meetings can also help to reduce scheduling
   conflicts during an IETF week and allow for a more optimal time slot
   for the key participants.  While interim meetings are less likely to
   attract people with casual interest, they provide a good opportunity
   for the most active participants of a group to have detailed
   technical discussions and solve recorded issues efficiently.






Kühlewind & Duke          Expires 18 June 2023                  [Page 7]

Internet-Draft        Organization Online Meetings         December 2022


4.2.  Flexibility of Time Usage

   This document recommends further experiments with reducing conflicts
   by leveraging the increased flexibility of the online format.

   An in-person meeting must fit all sessions into an acceptable length
   for international travel (usually roughly a week), but online
   meetings do not have that constraint.

   Therefore, it would be possible to keep most regular working group
   sessions within the usual five main meeting days but have some of the
   more conflicted sessions in other dedicated time slots.  As the
   Hackathon for fully online meetings is usually held in the week
   before the online plenary meeting [RFC9311], that week is already a
   highly active week for many IETF participants and might provide an
   opportunity to schedule a few selected sessions.

   This might work especially well for sessions that are of high
   interest to a large part of community, such as BoFs and dispatch
   meetings, and therefore hard to schedule during the main IETF week.

   At IETF 112, the IESG ran an experiment where the administrative
   plenary was scheduled on the Wednesday before the official session
   week.  The experiment report [_112-EXPERIMENT] found that it led to a
   reduction in scheduling conflicts but also a slight drop in
   attendance of the administrative plenary, partly due to insufficient
   awareness.

4.3.  Inclusivity and Socializing

   Participation in the fully online meetings in 2021 was high and had a
   stable per-country distribution, even though time zones were rotated.
   This indicates that online meetings support a more consistent
   geographic distribution of participants than in-person meetings,
   where participation often fluctuates based on the location.

   However, online meetings do not provide an equivalent opportunity to
   socialize.  Despite significant investment in tools to foster hallway
   conversations, many did not use those tools, whether due to ignorance
   of them, dislike of the tools, or a preference for the other
   activities at home (including sleep and food) over hallway
   interactions.

   There was a decrease in submission of new (-00) drafts during 2020
   and 2021, although the overall number of draft submissions remained
   stable, which might result from the loss of these interactions.
   Informal conversations might be important to inspire new work.




Kühlewind & Duke          Expires 18 June 2023                  [Page 8]

Internet-Draft        Organization Online Meetings         December 2022


4.4.  Experiments

   This document recommends further experiments with the meeting
   structure.  Often, only practical experience can answer open
   questions.  A given meeting SHOULD only experiment with one major
   change at a time in order to be able to assess the outcome correctly.
   Furthermore, the IESG SHOULD announce any such experiment well in
   advance, so people can adjust to changes and potentially provide
   feedback.

5.  Acknowledgments

   Thanks to Brian Carpenter, Lars Eggert, Toerless Eckert, Charles
   Eckel, Jason Livingood, Sanjeev Gupta, Dale Worley, and Mark
   Nottingham for their reviews and many from more for their input and
   suggestions on the time zone discussion!

6.  References

6.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/rfc/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

   [RFC8179]  Bradner, S. and J. Contreras, "Intellectual Property
              Rights in IETF Technology", BCP 79, RFC 8179,
              DOI 10.17487/RFC8179, May 2017,
              <https://www.rfc-editor.org/rfc/rfc8179>.

6.2.  Informative References

   [INTERIM-SCHEDULE]
              Cooper, A., "Post-IETF-107 Recommended Virtual Interim
              Schedule", 13 March 2020,
              <https://mailarchive.ietf.org/arch/msg/wgchairs/
              l382SqKVVHoTzFw9kIYl2boM6_c/>.









Kühlewind & Duke          Expires 18 June 2023                  [Page 9]

Internet-Draft        Organization Online Meetings         December 2022


   [RFC8713]  Kucherawy, M., Ed., Hinden, R., Ed., and J. Livingood,
              Ed., "IAB, IESG, IETF Trust, and IETF LLC Selection,
              Confirmation, and Recall Process: Operation of the IETF
              Nominating and Recall Committees", BCP 10, RFC 8713,
              DOI 10.17487/RFC8713, February 2020,
              <https://www.rfc-editor.org/rfc/rfc8713>.

   [RFC8719]  Krishnan, S., "High-Level Guidance for the Meeting Policy
              of the IETF", BCP 226, RFC 8719, DOI 10.17487/RFC8719,
              February 2020, <https://www.rfc-editor.org/rfc/rfc8719>.

   [RFC9311]  Eckel, C., "Running an IETF Hackathon", RFC 9311,
              DOI 10.17487/RFC9311, September 2022,
              <https://www.rfc-editor.org/rfc/rfc9311>.

   [_107-FEEDBACK]
              Daley, J., "IETF 107 Virtual Meeting Survey Report", 17
              April 2020, <https://www.ietf.org/media/documents/ietf-
              107-survey-results.pdf>.

   [_108-SURVEY]
              Daley, J., "IETF 108 Meeting Survey", 13 August 2020,
              <https://www.ietf.org/blog/ietf-108-meeting-survey>.

   [_109-SURVEY]
              Daley, J., "IETF 109 Post-Meeting Survey", 7 December
              2020,
              <https://www.ietf.org/blog/ietf-109-post-meeting-survey/>.

   [_111-SURVEY]
              Daley, J., "IETF 111 Post-Meeting Survey", 23 August 2021,
              <https://www.ietf.org/blog/ietf-111-post-meeting-survey/>.

   [_112-EXPERIMENT]
              IESG, "IETF 112 Plenary Experiment Evaluation", 4 February
              2022, <https://www.ietf.org/blog/ietf112-plenary-
              experiment-evaluation/>.

Authors' Addresses

   Mirja Kühlewind
   Ericsson
   Email: mirja.kuehlewind@ericsson.com


   Martin Duke
   Google
   Email: martin.h.duke@gmail.com



Kühlewind & Duke          Expires 18 June 2023                 [Page 10]
