diff options
| author | Eduardo Chappa <chappa@washington.edu> | 2013-06-03 10:30:56 -0600 |
|---|---|---|
| committer | Eduardo Chappa <chappa@washington.edu> | 2013-06-03 10:30:56 -0600 |
| commit | e4b35478c8b3ce7352a74b2fea0e067f068daf18 (patch) | |
| tree | 0b8a97debddc8210c6696c252c26f03703b606fa /imap/docs/rfc/rfc4549.txt | |
| parent | a46157ba61f2c65f88b42abb31db60c4a714f87b (diff) | |
| download | alpine-e4b35478c8b3ce7352a74b2fea0e067f068daf18.tar.xz | |
* Changes to configure.ac to add -lkrb5 to the link
* Changes to avoud errors in compilation when -Wformat-security is used
* Remove RFC files from source code
Diffstat (limited to 'imap/docs/rfc/rfc4549.txt')
| -rw-r--r-- | imap/docs/rfc/rfc4549.txt | 1963 |
1 files changed, 0 insertions, 1963 deletions
diff --git a/imap/docs/rfc/rfc4549.txt b/imap/docs/rfc/rfc4549.txt deleted file mode 100644 index 8430ee10..00000000 --- a/imap/docs/rfc/rfc4549.txt +++ /dev/null @@ -1,1963 +0,0 @@ - - - - - - -Network Working Group A. Melnikov, Ed. -Request for Comments: 4549 Isode Ltd. -Category: Informational June 2006 - - - Synchronization Operations for Disconnected IMAP4 Clients - -Status of This Memo - - This memo provides information for the Internet community. It does - not specify an Internet standard of any kind. Distribution of this - memo is unlimited. - -Copyright Notice - - Copyright (C) The Internet Society (2006). - -Abstract - - This document attempts to address some of the issues involved in - building a disconnected IMAP4 client. In particular, it deals with - the issues of what might be called the "driver" portion of the - synchronization tool: the portion of the code responsible for issuing - the correct set of IMAP4 commands to synchronize the disconnected - client in the way that is most likely to make the human who uses the - disconnected client happy. - - This note describes different strategies that can be used by - disconnected clients and shows how to use IMAP protocol in order to - minimize the time of the synchronization process. - - This note also lists IMAP extensions that a server should implement - in order to provide better synchronization facilities to disconnected - clients. - - - - - - - - - - - - - - - - - -Melnikov Informational [Page 1] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - -Table of Contents - - 1. Introduction ....................................................3 - 1.1. Conventions Used in This Document ..........................3 - 2. Design Principles ...............................................3 - 3. Overall Picture of Synchronization ..............................4 - 4. Mailbox Synchronization Steps and Strategies ....................7 - 4.1. Checking UID Validity ......................................7 - 4.2. Synchronizing Local Changes with the Server ................8 - 4.2.1. Uploading Messages to the Mailbox ...................8 - 4.2.2. Optimizing "move" and "copy" Operations .............9 - 4.2.3. Replaying Local Flag Changes .......................14 - 4.2.4. Processing Mailbox Compression (EXPUNGE) Requests ..15 - 4.2.5. Closing a Mailbox ..................................17 - 4.3. Details of "Normal" Synchronization of a Single Mailbox ...18 - 4.3.1. Discovering New Messages and Changes to Old - Messages ...........................................18 - 4.3.2. Searching for "Interesting" Messages. ..............20 - 4.3.3. Populating Cache with "Interesting" Messages. ......21 - 4.3.4. User-Initiated Synchronization .....................22 - 4.4. Special Case: Descriptor-Only Synchronization .............22 - 4.5. Special Case: Fast New-Only Synchronization ...............23 - 4.6. Special Case: Blind FETCH .................................23 - 5. Implementation Considerations ..................................24 - 5.1. Error Recovery during Playback ............................26 - 5.2. Quality of Implementation Issues ..........................28 - 5.3. Optimizations .............................................28 - 6. IMAP Extensions That May Help ..................................30 - 6.1. CONDSTORE Extension .......................................30 - 7. Security Considerations ........................................33 - 8. References .....................................................33 - 8.1. Normative References ......................................33 - 8.2. Informative References ....................................34 - 9. Acknowledgements ...............................................34 - - - - - - - - - - - - - - - - - -Melnikov Informational [Page 2] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - -1. Introduction - - Several recommendations presented in this document are generally - applicable to all types of IMAP clients. However, this document - tries to concentrate on disconnected mail clients [IMAP-MODEL]. It - also suggests some IMAP extensions* that should be implemented by - IMAP servers in order to make the life of disconnected clients - easier. In particular, the [UIDPLUS] extension was specifically - designed to streamline certain disconnected operations, like - expunging, uploading, and copying messages (see Sections 4.2.1, - 4.2.2.1, and 4.2.4). - - Readers of this document are also strongly advised to read RFC 2683 - [RFC2683]. - - * Note that the functionality provided by the base IMAP protocol - [IMAP4] is sufficient to perform basic synchronization. - -1.1. Conventions Used in This Document - - In examples, "C:" and "S:" indicate lines sent by the client and - server, respectively. Long lines in examples are broken for - editorial clarity. - - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", - "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this - document are to be interpreted as described in RFC 2119 [KEYWORDS]. - - Let's call an IMAP command idempotent if the result of executing the - command twice sequentially is the same as the result of executing the - command just once. - -2. Design Principles - - All mailbox state or content information stored on the disconnected - client should be viewed strictly as a cache of the state of the - server. The "master" state remains on the server, just as it would - with an interactive IMAP4 client. The one exception to this rule is - that information about the state of the disconnected client's cache - (the state includes flag changes while offline and during scheduled - message uploads) remains on the disconnected client: that is, the - IMAP4 server is not responsible for remembering the state of the - disconnected IMAP4 client. - - We assume that a disconnected client is a client that, for whatever - reason, wants to minimize the length of time that it is "on the - phone" to the IMAP4 server. Often this will be because the client is - using a dialup connection, possibly with very low bandwidth, but - - - -Melnikov Informational [Page 3] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - sometimes it might just be that the human is in a hurry to catch an - airplane, or some other event beyond our control. Whatever the - reason, we assume that we must make efficient use of the network - connection, both in the usual sense (not generating spurious traffic) - and in the sense that we would prefer not to have the connection - sitting idle while the client and/or the server is performing - strictly local computation or I/O. Another, perhaps simpler way of - stating this is that we assume that network connections are - "expensive". - - Practical experience with disconnected mail systems has shown that - there is no single synchronization strategy that is appropriate for - all cases. Different humans have different preferences, and the same - human's preference will vary depending both on external circumstance - (how much of a hurry the human is in today) and on the value that the - human places on the messages being transferred. The point here is - that there is no way that the synchronization program can guess - exactly what the human wants to do, so the human will have to provide - some guidance. - - Taken together, the preceding two principles lead to the conclusion - that the synchronization program must make its decisions based on - some kind of guidance provided by the human, by selecting the - appropriate options in the user interface or through some sort of - configuration file. Almost certainly, it should not pause for I/O - with the human in the middle of the synchronization process. The - human will almost certainly have several different configurations for - the synchronization program, for different circumstances. - - Since a disconnected client has no way of knowing what changes might - have occurred to the mailbox while it was disconnected, message - numbers are not useful to a disconnected client. All disconnected - client operations should be performed using UIDs, so that the client - can be sure that it and the server are talking about the same - messages during the synchronization process. - -3. Overall Picture of Synchronization - - The basic strategy for synchronization is outlined below. Note that - the real strategy may vary from one application to another or may - depend on a synchronization mode. - - a) Process any "actions" that were pending on the client that were - not associated with any mailbox. (In particular sending messages - composed offline with SMTP. This is not part of IMAP - synchronization, but it is mentioned here for completeness.) - - - - - -Melnikov Informational [Page 4] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - b) Fetch the current list of "interesting" mailboxes. (The - disconnected client should allow the user to skip this step - completely.) - - c) "Client-to-server synchronization": for each IMAP "action" that - was pending on the client, do the following: - - 1) If the action implies opening a new mailbox (any operation that - operates on messages), open the mailbox. Check its UID - validity value (see Section 4.1 for more details) returned in - the UIDVALIDITY response code. If the UIDVALIDITY value - returned by the server differs, the client MUST empty the local - cache of the mailbox and remove any pending "actions" that - refer to UIDs in that mailbox (and consider them failed). Note - that this doesn't affect actions performed on client-generated - fake UIDs (see Section 5). - - 2) Perform the action. If the action is to delete a mailbox - (DELETE), make sure that the mailbox is closed first (see also - Section 3.4.12 of [RFC2683]). - - d) "Server-to-client synchronization": for each mailbox that requires - synchronization, do the following: - - 1) Check the mailbox UIDVALIDITY (see Section 4.1 for more - details) with SELECT/EXAMINE/STATUS. - - If UIDVALIDITY value returned by the server differs, the client - MUST - - * empty the local cache of that mailbox; - * remove any pending "actions" that refer to UIDs in that - mailbox and consider them failed; and - * skip step 2-II. - - 2) Fetch the current "descriptors"; - - I) Discover new messages. - - II) Discover changes to old messages. - - 3) Fetch the bodies of any "interesting" messages that the client - doesn't already have. - - e) Close all open mailboxes not required for further operations (if - staying online) or disconnect all open connections (if going - offline). - - - - -Melnikov Informational [Page 5] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - Terms used: - - "Actions" are queued requests that were made by the human to the - client's Mail User Agent (MUA) software while the client was - disconnected. - - We define "descriptors" as a set of IMAP4 FETCH data items. - Conceptually, a message's descriptor is that set of information that - allows the synchronization program to decide what protocol actions - are necessary to bring the local cache to the desired state for this - message; since this decision is really up to the human, this - information probably includes at least a few header fields intended - for human consumption. Exactly what will constitute a descriptor - depends on the client implementation. At a minimum, the descriptor - contains the message's UID and FLAGS. Other likely candidates are - the RFC822.SIZE, RFC822.HEADER, BODYSTRUCTURE, or ENVELOPE data - items. - - Comments: - - 1) The list of actions should be ordered. For example, if the human - deletes message A1 in mailbox A, then expunges mailbox A, and then - deletes message A2 in mailbox A, the human will expect that - message A1 is gone and that message A2 is still present but is now - deleted. - - By processing all the actions before proceeding with - synchronization, we avoid having to compensate for the local MUA's - changes to the server's state. That is, once we have processed - all the pending actions, the steps that the client must take to - synchronize itself will be the same no matter where the changes to - the server's state originated. - - 2) Steps a and b can be performed in parallel. Alternatively, step a - can be performed after d. - - 3) On step b, the set of "interesting" mailboxes pretty much has to - be determined by the human. What mailboxes belong to this set may - vary between different IMAP4 sessions with the same server, - client, and human. An interesting mailbox can be a mailbox - returned by LSUB command (see Section 6.3.9 of [IMAP4]). The - special mailbox "INBOX" SHOULD be in the default set of mailboxes - that the client considers interesting. However, providing the - ability to ignore INBOX for a particular session or client may be - valuable for some mail filtering strategies. - - - - - - -Melnikov Informational [Page 6] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - 4) On step d-2-II, the client also finds out about changes to the - flags of messages that the client already has in its local cache, - and about messages in the local cache that no longer exist on the - server (i.e., messages that have been expunged). - - 5) "Interesting" messages are those messages that the synchronization - program thinks the human wants to have cached locally, based on - the configuration and the data retrieved in step b. - - 6) A disconnected IMAP client is a special case of an IMAP client, so - it MUST be able to handle any "unexpected" unsolicited responses, - like EXISTS and EXPUNGE, at any time. The disconnected client MAY - ignore EXPUNGE response during "client-to-server" synchronization - phase (step c). - - The rest of this discussion will focus primarily on the - synchronization issues for a single mailbox. - -4. Mailbox Synchronization Steps and Strategies - -4.1. Checking UID Validity - - The "UID validity" of a mailbox is a number returned in an - UIDVALIDITY response code in an OK untagged response at mailbox - selection time. The UID validity value changes between sessions when - UIDs fail to persist between sessions. - - Whenever the client selects a mailbox, the client must compare the - returned UID validity value with the value stored in the local cache. - If the UID validity values differ, the UIDs in the client's cache are - no longer valid. The client MUST then empty the local cache of that - mailbox and remove any pending "actions" that refer to UIDs in that - mailbox. The client MAY also issue a warning to the human. The - client MUST NOT cancel any scheduled uploads (i.e., APPENDs) for the - mailbox. - - Note that UIDVALIDITY is not only returned on a mailbox selection. - The COPYUID and APPENDUID response codes defined in the [UIDPLUS] - extension (see also 4.2.2) and the UIDVALIDITY STATUS response data - item also contain a UIDVALIDITY value for some other mailbox. The - client SHOULD behave as described in the previous paragraph (but it - should act on the other mailbox's cache), no matter how it obtained - the UIDVALIDITY value. - - - - - - - - -Melnikov Informational [Page 7] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - -4.2. Synchronizing Local Changes with the Server - -4.2.1. Uploading Messages to the Mailbox - - Two of the most common examples of operations resulting in message - uploads are: - - 1) Saving a draft message - - 2) Copying a message between remote mailboxes on two different IMAP - servers or a local mailbox and a remote mailbox. - - Message upload is performed with the APPEND command. A message - scheduled to be uploaded has no UID associated with it, as all UIDs - are assigned by the server. The APPEND command will effectively - associate a UID with the uploaded message that can be stored in the - local cache for future reference. However, [IMAP4] doesn't describe - a simple mechanism to discover the message UID by just performing the - APPEND command. In order to discover the UID, the client can do one - of the following: - - 1) Remove the uploaded message from cache. Then, use the mechanism - described in 4.3 to fetch the information about the uploaded - message as if it had been uploaded by some other client. - - 2) Try to fetch header information as described in 4.2.2 in order to - find a message that corresponds to the uploaded message. One - strategy for doing this is described in 4.2.2. - - Case 1 describes a not particularly smart client. - - C: A003 APPEND Drafts (\Seen $MDNSent) {310} - S: + Ready for literal data - C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) - C: From: Fred Foobar <foobar@blt.example.COM> - C: Subject: afternoon meeting - C: To: mooch@owatagu.siam.edu - C: Message-Id: <B27397-0100000@blt.example.COM> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: Hello Joe, do you think we can meet at 3:30 tomorrow? - C: - S: A003 OK APPEND Completed - - Fortunately, there is a simpler way to discover the message UID in - the presence of the [UIDPLUS] extension: - - - - -Melnikov Informational [Page 8] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - C: A003 APPEND Drafts (\Seen $MDNSent) {310} - S: + Ready for literal data - C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) - C: From: Fred Foobar <foobar@blt.example.COM> - C: Subject: afternoon meeting - C: To: mooch@owatagu.siam.edu - C: Message-Id: <B27397-0100000@blt.example.COM> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: Hello Joe, do you think we can meet at 3:30 tomorrow? - C: - S: A003 OK [APPENDUID 1022843275 77712] APPEND completed - - The UID of the appended message is the second parameter of APPENDUID - response code. - -4.2.2. Optimizing "move" and "copy" Operations - - Practical experience with IMAP and other mailbox access protocols - that support multiple mailboxes suggests that moving a message from - one mailbox to another is an extremely common operation. - -4.2.2.1. Moving a Message between Two Mailboxes on the Same Server - - In IMAP4, a "move" operation between two mailboxes on the same server - is really a combination of a COPY operation and a STORE +FLAGS - (\Deleted) operation. This makes good protocol sense for IMAP, but - it leaves a simple-minded disconnected client in the silly position - of deleting and possibly expunging its cached copy of a message, then - fetching an identical copy via the network. - - However, the presence of the UIDPLUS extension in the server can - help: - - C: A001 UID COPY 567,414 "Interesting Messages" - S: A001 OK [COPYUID 1022843275 414,567 5:6] Completed - - This tells the client that the message with UID 414 in the current - mailbox was successfully copied to the mailbox "Interesting Messages" - and was given the UID 5, and that the message with UID 567 was given - the UID 6. - - In the absence of UIDPLUS extension support in the server, the - following trick can be used. By including the Message-ID: header and - the INTERNALDATE data item as part of the descriptor, the client can - check the descriptor of a "new" message against messages that are - already in its cache and avoid fetching the extra copy. Of course, - - - -Melnikov Informational [Page 9] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - it's possible that the cost of checking to see if the message is - already in the local cache may exceed the cost of just fetching it, - so this technique should not be used blindly. If the MUA implements - a "move" command, it makes special provisions to use this technique - when it knows that a copy/delete sequence is the result of a "move" - command. - - Note that servers are not required (although they are strongly - encouraged with "SHOULD language") to preserve INTERNALDATE when - copying messages. - - Also note that since it's theoretically possible for this algorithm - to find the wrong message (given sufficiently malignant Message-ID - headers), implementers should provide a way to disable this - optimization, both permanently and on a message-by-message basis. - - Example 1: Copying a message in the absence of UIDPLUS extension. - - At some point in time the client has fetched the source message and - some information was cached: - - C: C021 UID FETCH <uids> (BODY.PEEK[] INTERNALDATE FLAGS) - ... - S: * 27 FETCH (UID 123 INTERNALDATE "31-May-2002 05:26:59 -0600" - FLAGS (\Draft $MDNSent) BODY[] {1036} - S: ... - S: Message-Id: <20040903110856.22a127cd@chardonnay> - S: ... - S: ...message body... - S: ) - ... - S: C021 OK fetch completed - - Later on, the client decides to copy the message: - - C: C035 UID COPY 123 "Interesting Messages" - S: C035 OK Completed - - As the server hasn't provided the COPYUID response code, the client - tries the optimization described above: - - C: C036 SELECT "Interesting Messages" - ... - C: C037 UID SEARCH ON 31-May-2002 HEADER - "Message-Id" "20040903110856.22a127cd@chardonnay" - S: SEARCH 12368 - S: C037 OK completed - - - - -Melnikov Informational [Page 10] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - Note that if the server has returned multiple UIDs in the SEARCH - response, the client MUST NOT use any of the returned UID. - -4.2.2.2. Moving a Message from a Remote Mailbox to a Local - - Moving a message from a remote mailbox to a local is done with FETCH - (that includes FLAGS and INTERNALDATE) followed by UID STORE <uid> - +FLAGS.SILENT (\Deleted): - - C: A003 UID FETCH 123 (BODY.PEEK[] INTERNALDATE FLAGS) - S: * 27 FETCH (UID 123 INTERNALDATE "31-May-2002 05:26:59 -0600" - FLAGS (\Seen $MDNSent) BODY[] - S: ...message body... - S: ) - S: A003 OK UID FETCH completed - C: A004 UID STORE <uid> +FLAGS.SILENT (\Deleted) - S: A004 STORE completed - - Note that there is no reason to fetch the message during - synchronization if it's already in the client's cache. Also, the - client SHOULD preserve delivery date in the local cache. - -4.2.2.3. Moving a Message from a Local Mailbox to a Remote - - Moving a message from a local mailbox to a remote is done with - APPEND: - - C: A003 APPEND Drafts (\Seen $MDNSent) "31-May-2002 05:26:59 -0600" - {310} - S: + Ready for literal data - C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) - C: From: Fred Foobar <foobar@blt.example.COM> - C: Subject: afternoon meeting - C: To: mooch@owatagu.siam.edu - C: Message-Id: <B27397-0100000@blt.example.COM> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: Hello Joe, do you think we can meet at 3:30 tomorrow? - C: - S: A003 OK [APPENDUID 1022843275 77712] completed - - The client SHOULD specify the delivery date from the local cache in - the APPEND. - - If the [LITERAL+] extension is available, the client can save a - round-trip*: - - - - -Melnikov Informational [Page 11] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - C: A003 APPEND Drafts (\Seen $MDNSent) "31-May-2002 05:26:59 -0600" - {310+} - C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) - C: From: Fred Foobar <foobar@blt.example.COM> - C: Subject: afternoon meeting - C: To: mooch@owatagu.siam.edu - C: Message-Id: <B27397-0100000@blt.example.COM> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: Hello Joe, do you think we can meet at 3:30 tomorrow? - C: - S: A003 OK [APPENDUID 1022843275 77712] completed - - * Note that there is a risk that the server will reject the message - due to its size. If this happens, the client will waste bandwidth - transferring the whole message. If the client wouldn't have used - the LITERAL+, this could have been avoided: - - C: A003 APPEND Drafts (\Seen $MDNSent) "31-May-2004 05:26:59 -0600" - {16777215} - S: A003 NO Sorry, message is too big - -4.2.2.4. Moving a Message between Two Mailboxes on Different Servers - - Moving a message between two mailbox on two different servers is a - combination of the operations described in 4.2.2.2 followed by the - operations described in 4.2.2.3. - -4.2.2.5. Uploading Multiple Messages to a Remote Mailbox with - MULTIAPPEND - - When there is a need to upload multiple messages to a remote mailbox - (e.g., as per 4.2.2.3), the presence of certain IMAP extensions may - significantly improve performance. One of them is [MULTIAPPEND]. - - For some mail stores, opening a mailbox for appending might be - expensive. [MULTIAPPEND] tells the server to open the mailbox once - (instead of opening and closing it "n" times per "n" messages to be - uploaded) and to keep it open while a group of messages is being - uploaded to the server. - - Also, if the server supports both [MULTIAPPEND] and [LITERAL+] - extensions, the entire upload is accomplished in a single - command/response round-trip. - - - - - - -Melnikov Informational [Page 12] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - Note: Client implementers should be aware that [MULTIAPPEND] performs - append of multiple messages atomically. This means, for example, if - there is not enough space to save "n"-th message (or the message has - invalid structure and is rejected by the server) after successful - upload of "n-1" messages, the whole upload operation fails, and no - message will be saved in the mailbox. Although this behavior might - be desirable in certain situations, it might not be what you want. - Otherwise, the client should use the regular APPEND command (Section - 4.2.2.3), possibly utilizing the [LITERAL+] extension. See also - Section 5.1 for discussions about error recovery. - - Note: MULTIAPPEND can be used together with the UIDPLUS extension in - a way similar to what was described in Section 4.2.1. [MULTIAPPEND] - extends the syntax of the APPENDUID response code to allow for - multiple message UIDs in the second parameter. - - Example 2: - - This example demonstrates the use of MULTIAPPEND together with - UIDPLUS (synchronization points where the client waits for - confirmations from the server are marked with "<--->"): - - C: A003 APPEND Jan-2002 (\Seen $MDNSent) "31-May-2002 05:26:59 -0600" - {310} - <---> - S: + Ready for literal data - C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) - C: From: Fred Foobar <foobar@blt.example.COM> - C: Subject: afternoon meeting - C: To: mooch@owatagu.siam.edu - C: Message-Id: <B27397-0100000@blt.example.COM> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: Hello Joe, do you think we can meet at 3:30 tomorrow? - C: (\Seen) " 1-Jun-2002 22:43:04 -0800" {286} - <---> - S: + Ready for literal data - C: Date: Mon, 7 Feb 1994 22:43:04 -0800 (PST) - C: From: Joe Mooch <mooch@OWaTaGu.siam.EDU> - C: Subject: Re: afternoon meeting - C: To: foobar@blt.example.com - C: Message-Id: <a0434793874930@OWaTaGu.siam.EDU> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: 3:30 is fine with me. - C: - - - -Melnikov Informational [Page 13] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - S: A003 OK [APPENDUID 1022843275 77712,77713] completed - - The upload takes 3 round-trips. - - Example 3: - - In this example, Example 2 was modified for the case when the server - supports MULTIAPPEND, LITERAL+, and UIDPLUS. The upload takes only 1 - round-trip. - - C: A003 APPEND Jan-2002 (\Seen $MDNSent) "31-May-2002 05:26:59 -0600" - {310+} - C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST) - C: From: Fred Foobar <foobar@blt.example.COM> - C: Subject: afternoon meeting - C: To: mooch@owatagu.siam.edu - C: Message-Id: <B27397-0100000@blt.example.COM> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: Hello Joe, do you think we can meet at 3:30 tomorrow? - C: (\Seen) " 1-Jun-2002 22:43:04 -0800" {286+} - C: Date: Mon, 7 Feb 1994 22:43:04 -0800 (PST) - C: From: Joe Mooch <mooch@OWaTaGu.siam.EDU> - C: Subject: Re: afternoon meeting - C: To: foobar@blt.example.com - C: Message-Id: <a0434793874930@OWaTaGu.siam.EDU> - C: MIME-Version: 1.0 - C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII - C: - C: 3:30 is fine with me. - C: - S: A003 OK [APPENDUID 1022843275 77712,77713] completed - -4.2.3. Replaying Local Flag Changes - - The disconnected client uses the STORE command to synchronize local - flag state with the server. The disconnected client SHOULD use - +FLAGS.SILENT or -FLAGS.SILENT in order to set or unset flags - modified by the user while offline. The FLAGS form MUST NOT be used, - as there is a risk that this will overwrite flags on the server that - have been changed by some other client. - - Example 4: - - For the message with UID 15, the disconnected client stores the - following flags \Seen and $Highest. The flags were modified on the - server by some other client: \Seen, \Answered, and $Highest. While - - - -Melnikov Informational [Page 14] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - offline, the user requested that the $Highest flags be removed and - that the \Deleted flag be added. The flag synchronization sequence - for the message should look like: - - C: A001 UID STORE 15 +FLAGS.SILENT (\Deleted) - S: A001 STORE completed - C: A002 UID STORE 15 -FLAGS.SILENT ($Highest) - S: A002 STORE completed - - If the disconnected client is able to store an additional binary - state information (or a piece of information that can take a value - from a predefined set of values) in the local cache of an IMAP - mailbox or in a local mailbox (e.g., message priority), and if the - server supports storing of arbitrary keywords, the client MUST use - keywords to store this state on the server. - - Example 5: - - Imagine a speculative mail client that can mark a message as one of - work-related ($Work), personal ($Personal), or spam ($Spam). In - order to mark a message as personal, the client issues: - - C: A001 UID STORE 15 +FLAGS.SILENT ($Personal) - S: A001 STORE completed - C: A002 UID STORE 15 -FLAGS.SILENT ($Work $Spam) - S: A002 STORE completed - - In order to mark the message as not work, not personal and not spam, - the client issues: - - C: A003 UID STORE 15 -FLAGS.SILENT ($Personal $Work $Spam) - S: A003 STORE completed - -4.2.4. Processing Mailbox Compression (EXPUNGE) Requests - - A naive disconnected client implementation that supports compressing - a mailbox while offline may decide to issue an EXPUNGE command to the - server in order to expunge messages marked \Deleted. The problem - with this command during synchronization is that it permanently - erases all messages with the \Deleted flag set, i.e., even those - messages that were marked as \Deleted on the server while the user - was offline. Doing this might result in an unpleasant surprise for - the user. - - Fortunately the [UIDPLUS] extension can help in this case as well. - The extension introduces UID EXPUNGE command, that, unlike EXPUNGE, - takes a UID set parameter, that lists UIDs of all messages that can - be expunged. When processing this command the server erases only - - - -Melnikov Informational [Page 15] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - messages with \Deleted flag listed in the UID list. Thus, messages - not listed in the UID set will not be expunged even if they have the - \Deleted flag set. - - Example 6: - - While the user was offline, 3 messages with UIDs 7, 27, and 65 were - marked \Deleted when the user requested to compress the open mailbox. - Another client marked a message \Deleted on the server (UID 34). - During synchronization, the disconnected client issues: - - C: A001 UID EXPUNGE 7,27,65 - S: * ... EXPUNGE - S: * ... EXPUNGE - S: * ... EXPUNGE - S: A001 UID EXPUNGE completed - - If another client issues UID SEARCH DELETED command (to find all - messages with the \Deleted flag) before and after the UID EXPUNGE, it - will get: - - Before: - - C: B001 UID SEARCH DELETED - S: * SEARCH 65 34 27 7 - S: B001 UID SEARCH completed - - After: - - C: B002 UID SEARCH DELETED - S: * SEARCH 34 - S: B002 UID SEARCH completed - - In the absence of the [UIDPLUS] extension, the following sequence of - commands can be used as an approximation. Note: It's possible for - another client to mark additional messages as deleted while this - sequence is being performed. In this case, these additional messages - will be expunged as well. - - 1) Find all messages marked \Deleted on the server. - - C: A001 UID SEARCH DELETED - S: * SEARCH 65 34 27 7 - S: A001 UID SEARCH completed - - 2) Find all messages that must not be erased (for the previous - example the list will consist of the message with UID 34). - - - - -Melnikov Informational [Page 16] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - 3) Temporarily remove \Deleted flag on all messages found in step 2. - - C: A002 UID STORE 34 -FLAGS.SILENT (\Deleted) - S: A002 UID STORE completed - - 4) Expunge the mailbox. - - C: A003 EXPUNGE - S: * 20 EXPUNGE - S: * 7 EXPUNGE - S: * 1 EXPUNGE - S: A003 EXPUNGE completed - - Here, the message with UID 7 has message number 1, with UID 27 has - message number 7, and with UID 65 has message number 20. - - 5) Restore \Deleted flag on all messages found when performing step - 2. - - C: A004 UID STORE 34 +FLAGS.SILENT (\Deleted) - S: A004 UID STORE completed - -4.2.5. Closing a Mailbox - - When the disconnected client has to close a mailbox, it should not - use the CLOSE command, because CLOSE does a silent EXPUNGE. (Section - 4.2.4 explains why EXPUNGE should not be used by a disconnected - client.) It is safe to use CLOSE only if the mailbox was opened with - EXAMINE. - - If the mailbox was opened with SELECT, the client can use one of the - following commands to implicitly close the mailbox and prevent the - silent expunge: - - 1) UNSELECT - This is a command described in [UNSELECT] that works as - CLOSE, but doesn't cause the silent EXPUNGE. This command is - supported by the server if it reports UNSELECT in its CAPABILITY - list. - - 2) SELECT <another_mailbox> - SELECT causes implicit CLOSE without - EXPUNGE. - - 3) If the client intends to issue LOGOUT after closing the mailbox, - it may just issue LOGOUT, because LOGOUT causes implicit CLOSE - without EXPUNGE as well. - - 4) SELECT <non_existing_mailbox> - If the client knows a mailbox that - doesn't exist or can't be selected, it MAY SELECT it. - - - -Melnikov Informational [Page 17] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - If the client opened the mailbox with SELECT and just wants to avoid - implicit EXPUNGE without closing the mailbox, it may also use the - following: - - 5) EXAMINE <mailbox> - Reselect the same mailbox in read-only mode. - -4.3. Details of "Normal" Synchronization of a Single Mailbox - - The most common form of synchronization is where the human trusts the - integrity of the client's copy of the state of a particular mailbox - and simply wants to bring the client's cache up to date so that it - accurately reflects the mailbox's current state on the server. - -4.3.1. Discovering New Messages and Changes to Old Messages - - Let <lastseenuid> represent the highest UID that the client knows - about in this mailbox. Since UIDs are allocated in strictly - ascending order, this is simply the UID of the last message in the - mailbox that the client knows about. Let <lastseenuid+1> represent - <lastseenuid>'s UID plus one. Let <descriptors> represent a list - consisting of all the FETCH data item items that the implementation - considers part of the descriptor; at a minimum this is just the FLAGS - data item, but it usually also includes BODYSTRUCTURE and - RFC822.SIZE. At this step, <descriptors> SHOULD NOT include RFC822. - - With no further information, the client can issue the following two - commands: - - tag1 UID FETCH <lastseenuid+1>:* <descriptors> - tag2 UID FETCH 1:<lastseenuid> FLAGS - - The first command will request some information about "new" messages - (i.e., messages received by the server since the last - synchronization). It will also allow the client to build a message - number to UID map (only for new messages). The second command allows - the client to - - 1) update cached flags for old messages; - - 2) find out which old messages got expunged; and - - 3) build a mapping between message numbers and UIDs (for old - messages). - - The order here is significant. We want the server to start returning - the list of new message descriptors as fast as it can, so that the - client can start issuing more FETCH commands, so we start out by - asking for the descriptors of all the messages we know the client - - - -Melnikov Informational [Page 18] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - cannot possibly have cached yet. The second command fetches the - information we need to determine what changes may have occurred to - messages that the client already has cached. Note that the former - command should only be issued if the UIDNEXT value cached by the - client differs from the one returned by the server. Once the client - has issued these two commands, there's nothing more the client can do - with this mailbox until the responses to the first command start - arriving. A clever synchronization program might use this time to - fetch its local cache state from disk or to start the process of - synchronizing another mailbox. - - The following is an example of the first FETCH: - - C: A011 UID fetch 131:* (FLAGS BODYSTRUCTURE INTERNALDATE - RFC822.SIZE) - - Note 1: The first FETCH may result in the server's sending a huge - volume of data. A smart disconnected client should use message - ranges (see also Section 3.2.1.2 of [RFC2683]), so that the user is - able to execute a different operation between fetching information - for a group of new messages. - - Example 7: - - Knowing the new UIDNEXT returned by the server on SELECT or EXAMINE - (<uidnext>), the client can split the UID range - <lastseenuid+1>:<uidnext> into groups, e.g., 100 messages. After - that, the client can issue: - - C: A011 UID fetch <lastseenuid+1>:<lastseenuid+100> - (FLAGS BODYSTRUCTURE INTERNALDATE RFC822.SIZE) - ... - C: A012 UID fetch <lastseenuid+101>:<lastseenuid+200> - (FLAGS BODYSTRUCTURE INTERNALDATE RFC822.SIZE) - ... - ... - C: A0FF UID fetch <lastseenuid+901>:<uidnext> - (FLAGS BODYSTRUCTURE INTERNALDATE RFC822.SIZE) - - Note that unless a SEARCH command is issued, it is impossible to - determine how many messages will fall into a subrange, as UIDs are - not necessarily contiguous. - - Note 2: The client SHOULD ignore any unsolicited EXPUNGE responses - received during the first FETCH command. EXPUNGE responses contain - message numbers that are useless to a client that doesn't have the - message-number-to-UID translation table. - - - - -Melnikov Informational [Page 19] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - The second FETCH command will result in zero or more untagged fetch - responses. Each response will have a corresponding UID FETCH data - item. All messages that didn't have a matching untagged FETCH - response MUST be removed from the local cache. - - For example, if the <lastseenuid> had a value 15000 and the local - cache contained 3 messages with the UIDs 12, 777, and 14999, - respectively, then after receiving the following responses from the - server, the client must remove the message with UID 14999 from its - local cache. - - S: * 1 FETCH (UID 12 FLAGS (\Seen)) - S: * 2 FETCH (UID 777 FLAGS (\Answered \Deleted)) - - Note 3: If the client is not interested in flag changes (i.e., the - client only wants to know which old messages are still on the - server), the second FETCH command can be substituted with: - - tag2 UID SEARCH UID 1:<lastseenuid> - - This command will generate less traffic. However, an implementor - should be aware that in order to build the mapping table from message - numbers to UIDs, the output of the SEARCH command MUST be sorted - first, because there is no requirement for a server to return UIDs in - SEARCH response in any particular order. - -4.3.2. Searching for "Interesting" Messages. - - This step is performed entirely on the client (from the information - received in the step described in 4.3.1), entirely on the server, or - on some combination of both. The decision on what is an - "interesting" message is up to the client software and the human. - One easy criterion that should probably be implemented in any client - is whether the message is "too big" for automatic retrieval, where - "too big" is a parameter defined in the client's configuration. - - Another commonly used criterion is the age of a message. For - example, the client may choose to download only messages received in - the last week (in this case, <date> would be today's date minus 7 - days): - - tag3 UID SEARCH UID <uidset> SINCE <date> - - Keep in mind that a date search disregards time and time zone. The - client can avoid doing this search if it specified INTERNALDATE in - <descriptors> on the step described in 4.3.1. If the client did, it - can perform the local search on its message cache. - - - - -Melnikov Informational [Page 20] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - At this step, the client also decides what kind of information about - a particular message to fetch from the server. In particular, even - for a message that is considered "too big", the client MAY choose to - fetch some part(s) of it. For example, if the message is a - multipart/mixed containing a text part and a MPEG attachment, there - is no reason for the client not to fetch the text part. The decision - of which part should or should not be fetched can be based on the - information received in the BODYSTRUCTURE FETCH response data item - (i.e., if BODYSTRUCTURE was included in <descriptors> on the step - described in 4.3.1). - -4.3.3. Populating Cache with "Interesting" Messages. - - Once the client has found out which messages are "interesting", it - can start issuing appropriate FETCH commands for "interesting" - messages or parts thereof. - - Note that fetching a message into the disconnected client's local - cache does NOT imply that the human has (or even will) read the - message. Thus, the synchronization program for a disconnected client - should always be careful to use the .PEEK variants of the FETCH data - items that implicitly set the \Seen flag. - - Once the last descriptor has arrived and the last FETCH command has - been issued, the client simply needs to process the incoming fetch - items and use them to update the local message cache. - - In order to avoid deadlock problems, the client must give processing - of received messages priority over issuing new FETCH commands during - this synchronization process. This may necessitate temporary local - queuing of FETCH requests that cannot be issued without causing a - deadlock. In order to achieve the best use of the "expensive" - network connection, the client will almost certainly need to pay - careful attention to any flow-control information that it can obtain - from the underlying transport connection (usually a TCP connection). - - Note: The requirement stated in the previous paragraph might result - in an unpleasant user experience, if followed blindly. For example, - the user might be unwilling to wait for the client to finish - synchronization before starting to process the user's requests. A - smart disconnected client should allow the user to perform requested - operations in between IMAP commands that are part of the - synchronization process. See also Note 1 in Section 4.3.1. - - - - - - - - -Melnikov Informational [Page 21] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - Example 8: - - After fetching a message BODYSTRUCTURE, the client discovers a - complex MIME message. Then, it decides to fetch MIME headers of the - nested MIME messages and some body parts. - - C: A011 UID fetch 11 (BODYSTRUCTURE) - S: ... - C: A012 UID fetch 11 (BODY[HEADER] BODY[1.MIME] BODY[1.1.MIME] - BODY[1.2.MIME] BODY[2.MIME] BODY[3.MIME] BODY[4.MIME] - BODY[5.MIME] BODY[6.MIME] BODY[7.MIME] BODY[8.MIME] BODY[9.MIME] - BODY[10.MIME] BODY[11.MIME] BODY[12.MIME] BODY[13.MIME] - BODY[14.MIME] BODY[15.MIME] BODY[16.MIME] BODY[17.MIME] - BODY[18.MIME] BODY[19.MIME] BODY[20.MIME] BODY[21.MIME]) - S: ... - C: A013 UID fetch 11 (BODY[1.1] BODY[1.2]) - S: ... - C: A014 UID fetch 11 (BODY[3] BODY[4] BODY[5] BODY[6] BODY[7] BODY[8] - BODY[9] BODY[10] BODY[11] BODY[13] BODY[14] BODY[15] BODY[16] - BODY[21]) - S: ... - -4.3.4. User-Initiated Synchronization - - After the client has finished the main synchronization process as - described in Sections 4.3.1-4.3.3, the user may optionally request - additional synchronization steps while the client is still online. - This is not any different from the process described in Sections - 4.3.2 and 4.3.3. - - Typical examples are: - - 1) fetch all messages selected in UI. - 2) fetch all messages marked as \Flagged on the server. - -4.4. Special Case: Descriptor-Only Synchronization - - For some mailboxes, fetching the descriptors might be the entire - synchronization step. Practical experience with IMAP has shown that - a certain class of mailboxes (e.g., "archival" mailboxes) are used - primarily for long-term storage of important messages that the human - wants to have instantly available on demand but does not want - cluttering up the disconnected client's cache at any other time. - Messages in this kind of mailbox would be fetched exclusively by - explicit actions queued by the local MUA. Thus, the only - synchronization desirable on this kind of mailbox is fetching enough - descriptor information for the user to be able to identify messages - for subsequent download. - - - -Melnikov Informational [Page 22] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - Special mailboxes that receive messages from a high volume, low - priority mailing list might also be in this category, at least when - the human is in a hurry. - -4.5. Special Case: Fast New-Only Synchronization - - In some cases, the human might be in such a hurry that he or she - doesn't care about changes to old messages, just about new messages. - In this case, the client can skip the UID FETCH command that obtains - the flags and UIDs for old messages (1:<lastseenuid>). - -4.6. Special Case: Blind FETCH - - In some cases, the human may know (for whatever reason) that he or - she always wants to fetch any new messages in a particular mailbox, - unconditionally. In this case, the client can just fetch the - messages themselves, rather than just the descriptors, by using a - command like: - - tag1 UID FETCH <lastseenuid+1>:* (FLAGS BODY.PEEK[]) - - Note that this example ignores the fact that the messages can be - arbitrary long. The disconnected client MUST always check for - message size before downloading, unless explicitly told otherwise. A - well-behaved client should instead use something like the following: - - 1) Issue "tag1 UID FETCH <lastseenuid+1>:* (FLAGS RFC822.SIZE)". - - 2) From the message sizes returned in step 1, construct UID set - <required_messages>. - - 3) Issue "tag2 UID FETCH <required_messages> (BODY.PEEK[])". - - or - - 1) Issue "tag1 UID FETCH <lastseenuid+1>:* (FLAGS)". - - 2) Construct UID set <old_uids> from the responses of step 1. - - 3) Issue "tag2 SEARCH UID <old_uids> SMALLER <message_limit>". - Construct UID set <required_messages> from the result of the - SEARCH command. - - 4) Issue "tag3 UID FETCH <required_messages> (BODY.PEEK[])". - - - - - - - -Melnikov Informational [Page 23] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - or - - 1) Issue "tag1 UID FETCH <lastseenuid+1>:* (FLAGS - BODY.PEEK[]<0.<length>>)", where <length> should be replaced with - the maximal message size the client is willing to download. - - Note: In response to such a command, the server will only return - partial data if the message is longer than <length>. It will - return the full message data for any message whose size is smaller - than or equal to <length>. In the former case, the client will - not be able to extract the full MIME structure of the message from - the truncated data, so the client should include BODYSTRUCTURE in - the UID FETCH command as well. - -5. Implementation Considerations - - Below are listed some common implementation pitfalls that should be - considered when implementing a disconnected client. - - 1) Implementing fake UIDs on the client. - - A message scheduled to be uploaded has no UID, as UIDs are - selected by the server. The client may implement fake UIDs - internally in order to reference not-yet-uploaded messages in - further operations. (For example, a message could be scheduled to - be uploaded, but subsequently marked as deleted or copied to - another mailbox). Here, the client MUST NOT under any - circumstances send these fake UIDs to the server. Also, client - implementers should be reminded that according to [IMAP4] a UID is - a 32-bit unsigned integer excluding 0. So, both 4294967295 and - 2147483648 are valid UIDs, and 0 and -1 are both invalid. Some - disconnected mail clients have been known to send negative numbers - (e.g., "-1") as message UIDs to servers during synchronization. - - Situation 1: The user starts composing a new message, edits it, - saves it, continues to edit it, and saves it again. - - A disconnected client may record in its replay log (log of - operations to be replayed on the server during synchronization) - the sequence of operations as shown below. For the purpose of - this situation, we assume that all draft messages are stored in - the mailbox called Drafts on an IMAP server. We will also use the - following conventions: <old_uid> is the UID of the intermediate - version of the draft when it was saved for the first time. This - is a fake UID generated on the client. <new_uid> is the UID of - the final version of the draft. This is another fake UID - generated on the client. - - - - -Melnikov Informational [Page 24] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - 1) APPEND Drafts (\Seen $MDNSent \Drafts) {<nnn>} - ...first version of the message follows... - - 2) APPEND Drafts (\Seen $MDNSent \Drafts) {<mmm>} - ...final version of the message follows... - - 3) STORE <old_uid> +FLAGS (\Deleted) - - Step 1 corresponds to the first attempt to save the draft message, - step 2 corresponds to the second attempt to save the draft - message, and step 3 deletes the first version of the draft message - saved in step 1. - - A naive disconnected client may send the command in step 3 without - replacing the fake client generated <old_uid> with the value - returned by the server in step 1. A server will probably reject - this command, which will make the client believe that the - synchronization sequence has failed. - - 2) Section 5.1 discusses common implementation errors related to - error recovery during playback. - - 3) Don't assume that the disconnected client is the only client used - by the user. - - Situation 2: Some clients may use the \Deleted flag as an - indicator that the message should not appear in the user's view. - Usage of the \Deleted flag for this purpose is not safe, as other - clients (e.g., online clients) might EXPUNGE the mailbox at any - time. - - 4) Beware of data dependencies between synchronization operations. - - It might be very tempting for a client writer to perform some - optimizations on the playback log. Such optimizations might - include removing redundant operations (for example, see - optimization 2 in Section 5.3), or their reordering. - - It is not always safe to reorder or remove redundant operations - during synchronization because some operations may have - dependencies (as Situation 3 demonstrates). So, if in doubt, - don't do this. - - Situation 3: The user copies a message out of a mailbox and then - deletes the mailbox. - - - - - - -Melnikov Informational [Page 25] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - C: A001 SELECT Old-Mail - S: ... - C: A002 UID COPY 111 ToDo - S: A002 OK [COPYUID 1022843345 111 94] Copy completed - ... - C: A015 CLOSE - S: A015 OK Completed - C: A016 DELETE Old-Mail - S: A016 OK Mailbox deletion completed successfully - - If the client performs DELETE (tag A016) first and COPY (tag A002) - second, then the COPY fails. Also, the message that the user so - carefully copied into another mailbox has been lost. - -5.1. Error Recovery during Playback - - Error recovery during synchronization is one of the trickiest parts - to get right. Below, we will discuss certain error conditions and - suggest possible choices for handling them. - - 1) Lost connection to the server. - - The client MUST remember the current position in the playback - (replay) log and replay it starting from the interrupted operation - (the last command issued by the client, but not acknowledged by - the server) the next time it successfully connects to the same - server. If the connection was lost while executing a non- - idempotent IMAP command (see the definition in Section 1), then - when the client is reconnected, it MUST make sure that the - interrupted command was indeed not executed. If it wasn't - executed, the client must restart playback from the interrupted - command, otherwise from the following command. - - Upon reconnect, care must be taken in order to properly reapply - logical operations that are represented by multiple IMAP commands, - e.g., UID EXPUNGE emulation when UID EXPUNGE is not supported by - the server (see Section 4.2.4). - - Once the client detects that the connection to the server was - lost, it MUST stop replaying its log. There are existing - disconnected clients that, to the great annoyance of users, pop up - an error dialog for each and every playback operation that fails. - - 2) Copying/appending messages to a mailbox that doesn't exist. (The - server advertises this condition by sending the TRYCREATE response - code in the tagged NO response to the APPEND or COPY command.) - - - - - -Melnikov Informational [Page 26] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - The user should be advised about the situation and be given one of - the following choices: - - a) Try to recreate a mailbox. - b) Copy/upload messages to another mailbox. - c) Skip copy/upload. - d) Abort replay. - - 3) Copying messages from a mailbox that doesn't exist, or renaming or - getting/changing ACLs [ACL] on a mailbox that doesn't exist: - - a) Skip operation. - b) Abort replay. - - 4) Deleting mailboxes or deleting/expunging messages that no longer - exist. - - This is actually is not an error and should be ignored by the - client. - - 5) Performing operations on messages that no longer exist. - - a) Skip operation. - b) Abort replay. - - In the case of changing flags on an expunged message, the client - should silently ignore the error. - - Note 1: Several synchronization operations map to multiple IMAP - commands (for example, "move" described in 4.2.2). The client must - guarantee atomicity of each such multistep operation. For example, - when performing a "move" between two mailboxes on the same server, if - the server is unable to copy messages, the client MUST NOT attempt to - set the \Deleted flag on the messages being copied, let alone expunge - them. However, the client MAY consider that move operation to have - succeeded even if the server was unable to set the \Deleted flag on - copied messages. - - Note 2: Many synchronization operations have data dependencies. A - failed operation must cause all dependent operations to fail as well. - The client should check this and MUST NOT try to perform all - dependent operations blindly (unless the user corrected the original - problem). For example, a message may be scheduled to be appended to - a mailbox on the server and later on the appended message may be - copied to another mailbox. If the APPEND operation fails, the client - must not attempt to COPY the failed message later on. (See also - Section 5, Situation 3). - - - - -Melnikov Informational [Page 27] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - -5.2. Quality of Implementation Issues - - Below, some quality of implementation issues are listed for - disconnected clients. They will help to write a disconnected client - that works correctly, performs synchronization as quickly as possible - (and thus can make the user happier as well as save her some money), - and minimizes the server load: - - 1) Don't lose information. - - No matter how smart your client is in other areas, if it loses - information, users will get very upset. - - 2) Don't do work unless explicitly asked. Be flexible. Ask all - questions BEFORE starting synchronization, if possible. - - 3) Minimize traffic. - - The client MUST NOT issue a command if the client already received - the required information from the server. - - The client MUST make use of UIDPLUS extension if it is supported - by the server. - - See also optimization 1 in Section 5.3. - - 4) Minimize the number of round-trips. - - Round-trips kill performance, especially on links with high - latency. Sections 4.2.2.5 and 5.2 give some advice on how to - minimize the number of round-trips. - - See also optimization 1 in Section 5.3. - -5.3. Optimizations - - Some useful optimizations are described in this section. A - disconnected client that supports the recommendations listed below - will give the user a more pleasant experience. - - 1) The initial OK or PREAUTH responses may contain the CAPABILITY - response code as described in Section 7.1 of [IMAP4]. This - response code gives the same information as returned by the - CAPABILITY command*. A disconnected client that pays attention to - this response code can avoid sending CAPABILITY command and will - save a round-trip. - - - - - -Melnikov Informational [Page 28] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - * Note: Some servers report in the CAPABILITY response code - extensions that are only relevant in unauthenticated state or in - all states. Such servers usually send another CAPABILITY - response code upon successful authentication using LOGIN or - AUTHENTICATE command (that negotiates no security layer; see - Section 6.2.2 of [IMAP4]). The CAPABILITY response code sent - upon successful LOGIN/AUTHENTICATE might be different from the - CAPABILITY response code in the initial OK response, as - extensions only relevant for unauthenticated state will not be - advertised, and some additional extensions available only in - authenticated and/or selected state will be. - - Example 9: - - S: * OK [CAPABILITY IMAP4REV1 LOGIN-REFERRALS STARTTLS - AUTH=DIGEST-MD5 AUTH=SRP] imap.example.com ready - C: 2 authenticate DIGEST-MD5 - S: 2 OK [CAPABILITY IMAP4REV1 IDLE NAMESPACE MAILBOX-REFERRALS SCAN - SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND] - User authenticated (no layer) - - 2) An advanced disconnected client may choose to optimize its replay - log. For example, there might be some operations that are - redundant (the list is not complete): - - a) an EXPUNGE followed by another EXPUNGE or CLOSE; - b) changing flags (other than the \Deleted flag) on a message that - gets immediately expunged; - c) opening and closing the same mailbox. - - When optimizing, be careful about data dependencies between commands. - For example, if the client is wishing to optimize (see case b, above) - - tag1 UID STORE <uid1> +FLAGS (\Deleted) - ... - tag2 UID STORE <uid1> +FLAGS (\Flagged) - ... - tag3 UID COPY <uid1> "Backup" - ... - tag4 UID EXPUNGE <uid1> - - it can't remove the second UID STORE command because the message is - being copied before it gets expunged. - - In general, it might be a good idea to keep mailboxes open during - synchronization (see case c above), if possible. This can be more - easily achieved in conjunction with optimization 3 described below. - - - - -Melnikov Informational [Page 29] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - 3) Perform some synchronization steps in parallel, if possible. - - Several synchronization steps don't depend on each other and thus - can be performed in parallel. Because the server machine is - usually more powerful than the client machine and can perform some - operations in parallel, this may speed up the total time of - synchronization. - - In order to achieve such parallelization, the client will have to - open more than one connection to the same server. Client writers - should not forget about non-trivial cost associated with - establishing a TCP connection and performing an authentication. - The disconnected client MUST NOT use one connection per mailbox. - In most cases, it is sufficient to have two connections. The - disconnected client SHOULD avoid selecting the same mailbox in - more than one connection; see Section 3.1.1 of [RFC2683] for more - details. - - Any mailbox synchronization MUST start with checking the - UIDVALIDITY as described in Section 4.1 of this document. The - client MAY use STATUS command to check UID Validity of a non- - selected mailbox. This is preferable to opening many connections - to the same server to perform synchronization of multiple - mailboxes simultaneously. As described in Section 5.3.10 of - [IMAP4], this SHOULD NOT be used on the selected mailbox. - -6. IMAP Extensions That May Help - - The following extensions can save traffic and/or the number of - round-trips: - - 1) The use of [UIDPLUS] is discussed in Sections 4.1, 4.2.1, 4.2.2.1 - and 4.2.4. - - 2) The use of the MULTIAPPEND and LITERAL+ extensions for uploading - messages is discussed in Section 4.2.2.5. - - 3) Use the CONDSTORE extension (see Section 6.1) for quick flag - resynchronization. - -6.1. CONDSTORE Extension - - An advanced disconnected mail client should use the [CONDSTORE] - extension when it is supported by the server. The client must cache - the value from HIGHESTMODSEQ OK response code received on mailbox - opening and update it whenever the server sends MODSEQ FETCH data - items. - - - - -Melnikov Informational [Page 30] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - If the client receives NOMODSEQ OK untagged response instead of - HIGHESTMODSEQ, it MUST remove the last known HIGHESTMODSEQ value from - its cache and follow the more general instructions in Section 3. - - When the client opens the mailbox for synchronization, it first - compares UIDVALIDITY as described in step d-1 in Section 3. If the - cached UIDVALIDITY value matches the one returned by the server, the - client MUST compare the cached value of HIGHESTMODSEQ with the one - returned by the server. If the cached HIGHESTMODSEQ value also - matches the one returned by the server, then the client MUST NOT - fetch flags for cached messages, as they hasn't changed. If the - value on the server is higher than the cached one, the client MAY use - "SEARCH MODSEQ <cached-value>" to find all messages with flags - changed since the last time the client was online and had the mailbox - opened. Alternatively, the client MAY use "FETCH 1:* (FLAGS) - (CHANGEDSINCE <cached-value>)". The latter operation combines - searching for changed messages and fetching new information. - - In all cases, the client still needs to fetch information about new - messages (if requested by the user) as well as discover which - messages have been expunged. - - Step d ("Server-to-client synchronization") in Section 4 in the - presence of the CONDSTORE extension is amended as follows: - - d) "Server-to-client synchronization" - For each mailbox that - requires synchronization, do the following: - - 1a) Check the mailbox UIDVALIDITY (see section 4.1 for more - details) with SELECT/EXAMINE/STATUS. - - If the UIDVALIDITY value returned by the server differs, the - client MUST - - * empty the local cache of that mailbox; - * "forget" the cached HIGHESTMODSEQ value for the mailbox; - * remove any pending "actions" that refer to UIDs in that - mailbox (note that this doesn't affect actions performed on - client-generated fake UIDs; see Section 5); and - * skip steps 1b and 2-II; - - 1b) Check the mailbox HIGHESTMODSEQ. If the cached value is the - same as the one returned by the server, skip fetching message - flags on step 2-II, i.e., the client only has to find out - which messages got expunged. - - - - - - -Melnikov Informational [Page 31] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - 2) Fetch the current "descriptors". - - I) Discover new messages. - - II) Discover changes to old messages and flags for new messages - using - "FETCH 1:* (FLAGS) (CHANGEDSINCE <cached-value>)" or - "SEARCH MODSEQ <cached-value>". - - Discover expunged messages; for example, using - "UID SEARCH 1:<lastseenuid>". (All messages not returned - in this command are expunged.) - - 3) Fetch the bodies of any "interesting" messages that the client - doesn't already have. - - Example 10: - - The UIDVALIDITY value is the same, but the HIGHESTMODSEQ value - has changed on the server while the client was offline. - - C: A142 SELECT INBOX - S: * 172 EXISTS - S: * 1 RECENT - S: * OK [UNSEEN 12] Message 12 is first unseen - S: * OK [UIDVALIDITY 3857529045] UIDs valid - S: * OK [UIDNEXT 201] Predicted next UID - S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) - S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited - S: * OK [HIGHESTMODSEQ 20010715194045007] - S: A142 OK [READ-WRITE] SELECT completed - - After that, either: - - C: A143 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 20010715194032001) - S: * 2 FETCH (UID 6 MODSEQ (20010715205008000) FLAGS (\Deleted)) - S: * 5 FETCH (UID 9 MODSEQ (20010715195517000) FLAGS ($NoJunk - $AutoJunk $MDNSent)) - ... - S: A143 OK FETCH completed - - or: - - - - - - - - - -Melnikov Informational [Page 32] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - - C: A143 UID SEARCH MODSEQ 20010715194032001 UID 1:20 - S: * SEARCH 6 9 11 12 18 19 20 23 (MODSEQ 20010917162500) - S: A143 OK Search complete - C: A144 UID SEARCH 1:20 - S: * SEARCH 6 9 ... - S: A144 OK FETCH completed - -7. Security Considerations - - It is believed that this document does not raise any new security - concerns that are not already present in the base [IMAP4] protocol, - and these issues are discussed in [IMAP4]. Additional security - considerations may be found in different extensions mentioned in this - document; in particular, in [UIDPLUS], [LITERAL+], [CONDSTORE], - [MULTIAPPEND], and [UNSELECT]. - - Implementers are also reminded about the importance of thorough - testing. - -8. References - -8.1. Normative References - - [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate - Requirement Levels", BCP 14, RFC 2119, March 1997. - - [IMAP4] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - - VERSION 4rev1", RFC 3501, March 2003. - - [UIDPLUS] Crispin, M., "Internet Message Access Protocol (IMAP) - - UIDPLUS extension", RFC 4315, December 2005. - - [LITERAL+] Myers, J., "IMAP4 non-synchronizing literals", RFC - 2088, January 1997. - - [CONDSTORE] Melnikov, A. and S. Hole, "IMAP Extension for - Conditional STORE Operation or Quick Flag Changes - Resynchronization", RFC 4551, June 2006. - - [MULTIAPPEND] Crispin, M., "Internet Message Access Protocol (IMAP) - - MULTIAPPEND Extension", RFC 3502, March 2003. - - [UNSELECT] Melnikov, A., "Internet Message Access Protocol (IMAP) - UNSELECT command", RFC 3691, February 2004. - - [RFC2683] Leiba, B., "IMAP4 Implementation Recommendations", RFC - 2683, September 1999. - - - - -Melnikov Informational [Page 33] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - -8.2. Informative References - - [ACL] Melnikov, A., "IMAP4 Access Control List (ACL) - Extension", RFC 4314, December 2005. - - [IMAP-MODEL] Crispin, M., "Distributed Electronic Mail Models in - IMAP4", RFC 1733, December 1994. - -9. Acknowledgements - - This document is based on version 01 of the text written by Rob - Austein in November 1994. - - The editor appreciates comments posted by Mark Crispin to the IMAP - mailing list and the comments/corrections/ideas received from Grant - Baillie, Cyrus Daboo, John G. Myers, Chris Newman, and Timo Sirainen. - - The editor would also like to thank the developers of Netscape - Messenger and Mozilla mail clients for providing examples of - disconnected mail clients that served as a base for many - recommendations in this document. - -Editor's Address - - Alexey Melnikov - Isode Limited - 5 Castle Business Village - 36 Station Road - Hampton, Middlesex - TW12 2BX - United Kingdom - - Phone: +44 77 53759732 - EMail: alexey.melnikov@isode.com - - - - - - - - - - - - - - - - - -Melnikov Informational [Page 34] - -RFC 4549 Synch Ops for Disconnected IMAP4 Clients June 2006 - - -Full Copyright Statement - - Copyright (C) The Internet Society (2006). - - This document is subject to the rights, licenses and restrictions - contained in BCP 78, and except as set forth therein, the authors - retain all their rights. - - This document and the information contained herein are provided on an - "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS - OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET - ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, - INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE - INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED - WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. - -Intellectual Property - - The IETF takes no position regarding the validity or scope of any - Intellectual Property Rights or other rights that might be claimed to - pertain to the implementation or use of the technology described in - this document or the extent to which any license under such rights - might or might not be available; nor does it represent that it has - made any independent effort to identify any such rights. Information - on the procedures with respect to rights in RFC documents can be - found in BCP 78 and BCP 79. - - Copies of IPR disclosures made to the IETF Secretariat and any - assurances of licenses to be made available, or the result of an - attempt made to obtain a general license or permission for the use of - such proprietary rights by implementers or users of this - specification can be obtained from the IETF on-line IPR repository at - http://www.ietf.org/ipr. - - The IETF invites any interested party to bring to its attention any - copyrights, patents or patent applications, or other proprietary - rights that may cover technology that may be required to implement - this standard. Please address the information to the IETF at - ietf-ipr@ietf.org. - -Acknowledgement - - Funding for the RFC Editor function is provided by the IETF - Administrative Support Activity (IASA). - - - - - - - -Melnikov Informational [Page 35] - |