Differences between revisions 16 and 105 (spanning 89 versions)
Revision 16 as of 2008-03-03 21:58:08
Size: 3486
Editor: TimoSirainen
Comment:
Revision 105 as of 2019-05-29 06:55:08
Size: 5168
Editor: EllieTimoney
Comment: partially update info for cyrus (URL, version, and scripted tests only)
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= IMAP Server Compliance =
= IMAP Server Compliancy Status =
Line 6: Line 5:
 * Search: SEARCH command correctness.
 * Expunges: How messages expunged by another session are handled.
  * A: Expunging a message from one session keeps it available for other sessions until they can be notified that it's expunged. Flag updates to expunged messages may not be visible to other sessions though.
  * B: Some cached fields may be returned for expunged messages, but message bodies can't be retrieved anymore. Flags can still be updated within the session (but updates may not be visible to other sessions).
  * C: Even message flags can't be updated for expunged messages.
 * Failures: Number of failures using [:ImapTest/Tests:command/reply tests] (failures / number of tests). These numbers may not be exact all the time, because the tests are still changing.
 * Expunge fetch: If message is expunged in one session, can another session that still sees the message fetch its contents?
  * Yes: Yes, everything can be fetched
  * Cached: Only cached data can be fetched. Message header or body can't be.
  * No: Nothing but flags can be fetched.
 * Expunge store: If message is expunged in one session, can another session update its flags?
  * Yes: Yes, and a 3rd session also sees the change
  * Session: Yes, but a 3rd session won't see the change
  * Delayed: Yes, but a 3rd session won't see '''any''' changes until EXPUNGEs are reported
  * Ignored: STORE command replies OK, but no change is made
  * No: STORE command replies NO
 * Failures (failures/total): Number of failures using [[ImapTest/ScriptedTests|scripted tests]]. These numbers may not be exact all the time, because the tests are still changing.
  * Failure groups: Each test belongs to a wider group of tests, typically testing a command or part of a command. If this count is low but individual command failure count is high, it probably means that the server has failed to implement wrong only a couple of commands.
  * Base fails: Number of individual base IMAP4rev1 protocol commands that failed.
  * Ext fails: Number of individual IMAP extension commands that failed. Extensions not supported aren't included in the numbers.
Line 14: Line 21:
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' ||
||[[http://www.dovecot.org/|Dovecot]] ||Yes ||Yes ||Yes ||Yes / Cached (depends on storage) ||Yes ||0/40 ||0/403 ||0/100 ||
||[[https://github.com/jonabbey/panda-imap/|Panda IMAP]] 2008, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||0/34 ||0/328 ||0/97 ||
||[[http://netwinsite.com/surgemail/|SurgeMail]] 5.0h3 ||Yes ||Yes ||Yes ||Yes ||Yes ||0/35 ||0/342 ||0/26 ||
Line 15: Line 26:
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Search''' || '''Expunges''' || '''Failures''' ||
|| [http://www.dovecot.org/ Dovecot] 1.1.rc2+ || Yes || Yes || Yes || OK || B || 0/21 ||

Line 19: Line 30:
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' ||
||[[http://www.washington.edu/imap/|UW-IMAP]] 2007b, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||2/34 ||0/328 ||6/53 ||
||[[http://www.isode.com/products/m-box.html|Isode M-Box]] 14.3a0 ||No ||Unreliable ||Yes ||Cached ||Ignored ||4/40 ||1/408 ||8/112 ||
||[[http://www.communigate.com/community/|CommuniGate Pro]] 5.2.1 ||Yes ||Yes ||Yes ||Cached (some) ||No ||8/34 ||8/328 ||0/0 ||
||[[https://www.cyrusimap.org/|Cyrus]] 3.0.10 ||No ||Unreliable ||Bugs ||Cached ||Delayed ||0/35 ||0/366 ||0/100 ||
||[[http://www.sun.com/software/products/messaging_srvr/index.xml|Sun Java Messenging Server]] 6.3-0.15 ||No ||Unreliable ||Bugs ||Yes ||Delayed ||9/34 ||17/328 ||9/21 ||
||[[http://www.archiveopteryx.org/|Archiveopteryx]] 3.0.3 ||No ||Unreliable ||No ||No ||No ||13/38 ||25/346 ||6/26 ||
||[[http://www.courier-mta.org/imap/|Courier]] 4.3.1 ||Yes ||Unreliable ||Yes ||No ||No ||18/34 ||33/328 ||20/53 ||
||[[http://www.gmail.com/|GMail]] 2012-08-12 ||No ||Not implemented ||Bugs ||Cached (some) ||Ignored ||12/49 ||66/376 ||0/0 ||
||[[http://www.zarafa.com/|Zarafa Collaboration Platform]] 7.1.10 ||No ||Unreliable ||Yes ||No ||No ||6/35 ||14/340 ||0/0 ||
||[[http://www.zimbra.com/|Zimbra]] 5.0.5 ||Yes ||Yes ||Yes ||Cached ||No ||8/34 ||50/328 ||2/33 ||
||[[http://www.microsoft.com/exchange/default.mspx|MS Exchange]] ||No ||Unreliable ||? ||No ||No ||16/40 ||52/287 ||0/0 ||
||[[http://www.citadel.org/|Citadel]] 7.36 ||No ||Unreliable ||Yes ||Yes ||Session (flag changes are never seen by other sessions) ||19/34 ||98/328 ||0/0 ||
||[[http://www.hmailserver.com/|hMailServer]] 5.3.3 b1879 ||? ||? ||? ||? ||? ||||||<style="text-align: center;">Fails hardcoded OK response format test. Other tests don't work, if OK response test fails. ||
Line 20: Line 45:
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Search''' || '''Expunges''' || '''Failures''' ||
|| [http://www.archiveopteryx.org/ Archiveopteryx] 2.06 || No |||| ? (connections hang too easily) || Bugs || C ||
|| [http://www.courier-mta.org/imap/ Courier] 4.3.0 || Yes || Unreliable || Yes || SMALLER/LARGER wrong || C || 10/21 ||
|| [http://cyrusimap.web.cmu.edu/ Cyrus] 2.3.11 || No || Unreliable || Bugs || Int.dates wrong || B? Buggy || 6/21 ||
|| [http://www.dovecot.org/ Dovecot] 1.0 || Yes || Unreliable || Bugs || Int.dates wrong || B || 1/21 ||
|| [http://www.isode.com/products/m-box.html Isode M-Box] 14.2a0 || No || Unreliable || Yes || () bugs || C || 6/20 ||
|| [http://www.sun.com/software/products/messaging_srvr/index.xml Sun Java Messenging Server] 6.3-0.15 || No || Unreliable || Bugs || Bugs || A || 6/20 ||
|| [http://www.washington.edu/imap/ UW-IMAP] 2007, mix format || Yes || Yes || Bugs || OK || A || 3/21 ||
|| [http://www.washington.edu/imap/ UW-IMAP] 2007a.DEV, mix format || Yes || Yes || Yes || OK || A || 1/21 ||
|| [http://www.zimbra.com/ Zimbra] 5.0.1 || Yes || Yes || Yes || Bugs || C || 8/20 ||

Line 32: Line 49:

|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Search''' || '''Expunges''' || '''Failures''' ||
|| [http://www.citadel.org/ Citadel] 7.32 || No || ? || ? || Bugs || EXPUNGEs sent too often ||
|| [http://www.communigate.com/community/ CommuniGate Pro] 5.2.0 || ? || ? || ? || () bugs || C - EXPUNGEs lost sometimes || 10/19 ||
|| [http://www.ibm.com/software/lotus/products/notes/ IBM Domino] 8.0 || No || Unreliable || ? || SMALLER/LARGER not supported || C - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors || 16/20 ||
|| [http://www.kerio.eu/kms_home.html Kerio Mail Server] 6.5.0 || ? || ? || ? || Bugs || EXPUNGEs not sent || 16/20 ||
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failures''' ||
||[[http://www.dbmail.org/|dbmail]] 3.0.2 ||? ||Unreliable ||? ||Yes ||Yes || 9/361 - UID/sequence mapping becomes wrong ||
||[[http://www.ibm.com/software/lotus/products/notes/|IBM Domino]] 8.0 ||No ||Unreliable ||? ||? ||No ||16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors ||
||[[http://www.kerio.eu/kms_home.html|Kerio Mail Server]] 6.5.1 ||? ||? ||? ||? ||? ||18/34 - EXPUNGEs are sent wrong ||
||[[http://www.axigen.com/|Axigen]] 7.1.2 ||? ||Unreliable ||Yes ||Broken ||Broken ||19/312 - FETCH/STORE sends EXPUNGEs immediately ||

IMAP Server Compliancy Status

  • Checkpoint: checkpoint parameter works. When issuing a CHECK command in all sessions, their state looks identical.
  • \Recent: Exactly one session sees a new message as \Recent - no more and no less.
  • Atomic flags: Flags and keywords can be added/removed in multiple sessions without one session dropping changes made by other (own_flags test)
  • Expunge fetch: If message is expunged in one session, can another session that still sees the message fetch its contents?
    • Yes: Yes, everything can be fetched
    • Cached: Only cached data can be fetched. Message header or body can't be.
    • No: Nothing but flags can be fetched.
  • Expunge store: If message is expunged in one session, can another session update its flags?
    • Yes: Yes, and a 3rd session also sees the change
    • Session: Yes, but a 3rd session won't see the change
    • Delayed: Yes, but a 3rd session won't see any changes until EXPUNGEs are reported

    • Ignored: STORE command replies OK, but no change is made
    • No: STORE command replies NO
  • Failures (failures/total): Number of failures using scripted tests. These numbers may not be exact all the time, because the tests are still changing.

    • Failure groups: Each test belongs to a wider group of tests, typically testing a command or part of a command. If this count is low but individual command failure count is high, it probably means that the server has failed to implement wrong only a couple of commands.
    • Base fails: Number of individual base IMAP4rev1 protocol commands that failed.
    • Ext fails: Number of individual IMAP extension commands that failed. Extensions not supported aren't included in the numbers.

Fully compliant servers:

Server

Checkpoint

\Recent

Atomic flags

Expunge fetch

Expunge store

Failure groups

Base fails

Ext fails

Dovecot

Yes

Yes

Yes

Yes / Cached (depends on storage)

Yes

0/40

0/403

0/100

Panda IMAP 2008, mix format

Yes

Yes

Yes

Yes

Session

0/34

0/328

0/97

SurgeMail 5.0h3

Yes

Yes

Yes

Yes

Yes

0/35

0/342

0/26

Non-compliant servers:

Server

Checkpoint

\Recent

Atomic flags

Expunge fetch

Expunge store

Failure groups

Base fails

Ext fails

UW-IMAP 2007b, mix format

Yes

Yes

Yes

Yes

Session

2/34

0/328

6/53

Isode M-Box 14.3a0

No

Unreliable

Yes

Cached

Ignored

4/40

1/408

8/112

CommuniGate Pro 5.2.1

Yes

Yes

Yes

Cached (some)

No

8/34

8/328

0/0

Cyrus 3.0.10

No

Unreliable

Bugs

Cached

Delayed

0/35

0/366

0/100

Sun Java Messenging Server 6.3-0.15

No

Unreliable

Bugs

Yes

Delayed

9/34

17/328

9/21

Archiveopteryx 3.0.3

No

Unreliable

No

No

No

13/38

25/346

6/26

Courier 4.3.1

Yes

Unreliable

Yes

No

No

18/34

33/328

20/53

GMail 2012-08-12

No

Not implemented

Bugs

Cached (some)

Ignored

12/49

66/376

0/0

Zarafa Collaboration Platform 7.1.10

No

Unreliable

Yes

No

No

6/35

14/340

0/0

Zimbra 5.0.5

Yes

Yes

Yes

Cached

No

8/34

50/328

2/33

MS Exchange

No

Unreliable

?

No

No

16/40

52/287

0/0

Citadel 7.36

No

Unreliable

Yes

Yes

Session (flag changes are never seen by other sessions)

19/34

98/328

0/0

hMailServer 5.3.3 b1879

?

?

?

?

?

Fails hardcoded OK response format test. Other tests don't work, if OK response test fails.

Major problems with multiple connections (makes further testing difficult and MAY CAUSE ACCIDENTAL MAIL LOSS!):

Server

Checkpoint

\Recent

Atomic flags

Expunge fetch

Expunge store

Failures

dbmail 3.0.2

?

Unreliable

?

Yes

Yes

9/361 - UID/sequence mapping becomes wrong

IBM Domino 8.0

No

Unreliable

?

?

No

16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors

Kerio Mail Server 6.5.1

?

?

?

?

?

18/34 - EXPUNGEs are sent wrong

Axigen 7.1.2

?

Unreliable

Yes

Broken

Broken

19/312 - FETCH/STORE sends EXPUNGEs immediately

None: ImapTest/ServerStatus (last edited 2023-11-07 21:14:12 by TimoSirainen)