Differences between revisions 72 and 105 (spanning 33 versions)
Revision 72 as of 2008-12-17 18:22:38
Size: 4857
Editor: TimoSirainen
Comment:
Revision 105 as of 2019-05-29 06:55:08
Size: 5168
Editor: EllieTimoney
Comment: partially update info for cyrus (URL, version, and scripted tests only)
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= IMAP Server Compliance =
= IMAP Server Compliancy Status =
Line 16: Line 15:
 * Failures (failures/total): Number of failures using [:ImapTest/ScriptedTests:scripted tests]. These numbers may not be exact all the time, because the tests are still changing.  * Failures (failures/total): Number of failures using [[ImapTest/ScriptedTests|scripted tests]]. These numbers may not be exact all the time, because the tests are still changing.
Line 18: Line 17:
  * Base fails: Number of individual base IMAP4rev1 protocol commands that failed.    * Base fails: Number of individual base IMAP4rev1 protocol commands that failed.
Line 22: Line 21:
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' ||
||[[http://www.dovecot.org/|Dovecot]] ||Yes ||Yes ||Yes ||Yes / Cached (depends on storage) ||Yes ||0/40 ||0/403 ||0/100 ||
||[[https://github.com/jonabbey/panda-imap/|Panda IMAP]] 2008, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||0/34 ||0/328 ||0/97 ||
||[[http://netwinsite.com/surgemail/|SurgeMail]] 5.0h3 ||Yes ||Yes ||Yes ||Yes ||Yes ||0/35 ||0/342 ||0/26 ||
Line 23: Line 26:
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Expunge fetch''' || '''Expunge store''' || '''Failure groups''' || '''Base fails''' || '''Ext fails''' ||
|| [http://www.dovecot.org/ Dovecot] 1.1.5+ || Yes || Yes || Yes || Cached || Yes || 0/40 || 0/403 || 0/100 ||
|| [http://panda.com/imap/ Panda IMAP] 2008, mix format || Yes || Yes || Yes || Yes || Session || 0/34 || 0/328 || 0/97 ||

Line 28: Line 30:
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failure groups''' ||'''Base fails''' ||'''Ext fails''' ||
||[[http://www.washington.edu/imap/|UW-IMAP]] 2007b, mix format ||Yes ||Yes ||Yes ||Yes ||Session ||2/34 ||0/328 ||6/53 ||
||[[http://www.isode.com/products/m-box.html|Isode M-Box]] 14.3a0 ||No ||Unreliable ||Yes ||Cached ||Ignored ||4/40 ||1/408 ||8/112 ||
||[[http://www.communigate.com/community/|CommuniGate Pro]] 5.2.1 ||Yes ||Yes ||Yes ||Cached (some) ||No ||8/34 ||8/328 ||0/0 ||
||[[https://www.cyrusimap.org/|Cyrus]] 3.0.10 ||No ||Unreliable ||Bugs ||Cached ||Delayed ||0/35 ||0/366 ||0/100 ||
||[[http://www.sun.com/software/products/messaging_srvr/index.xml|Sun Java Messenging Server]] 6.3-0.15 ||No ||Unreliable ||Bugs ||Yes ||Delayed ||9/34 ||17/328 ||9/21 ||
||[[http://www.archiveopteryx.org/|Archiveopteryx]] 3.0.3 ||No ||Unreliable ||No ||No ||No ||13/38 ||25/346 ||6/26 ||
||[[http://www.courier-mta.org/imap/|Courier]] 4.3.1 ||Yes ||Unreliable ||Yes ||No ||No ||18/34 ||33/328 ||20/53 ||
||[[http://www.gmail.com/|GMail]] 2012-08-12 ||No ||Not implemented ||Bugs ||Cached (some) ||Ignored ||12/49 ||66/376 ||0/0 ||
||[[http://www.zarafa.com/|Zarafa Collaboration Platform]] 7.1.10 ||No ||Unreliable ||Yes ||No ||No ||6/35 ||14/340 ||0/0 ||
||[[http://www.zimbra.com/|Zimbra]] 5.0.5 ||Yes ||Yes ||Yes ||Cached ||No ||8/34 ||50/328 ||2/33 ||
||[[http://www.microsoft.com/exchange/default.mspx|MS Exchange]] ||No ||Unreliable ||? ||No ||No ||16/40 ||52/287 ||0/0 ||
||[[http://www.citadel.org/|Citadel]] 7.36 ||No ||Unreliable ||Yes ||Yes ||Session (flag changes are never seen by other sessions) ||19/34 ||98/328 ||0/0 ||
||[[http://www.hmailserver.com/|hMailServer]] 5.3.3 b1879 ||? ||? ||? ||? ||? ||||||<style="text-align: center;">Fails hardcoded OK response format test. Other tests don't work, if OK response test fails. ||
Line 29: Line 45:
|| '''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Expunge fetch''' || '''Expunge store''' || '''Failure groups''' || '''Base fails''' || '''Ext fails''' ||
|| [http://www.washington.edu/imap/ UW-IMAP] 2007b, mix format || Yes || Yes || Yes || Yes || Session || 2/34 || 0/328 || 6/53 ||
|| [http://www.isode.com/products/m-box.html Isode M-Box] 14.3a0 || No || Unreliable || Yes || Cached || Ignored || 4/40 || 0/328 || 8/112 ||
|| [http://www.dovecot.org/ Dovecot] 1.0.15 || Yes || Unreliable || Bugs || Cached || Yes || 2/34 || 4/328 || 2/53 ||
|| [http://www.communigate.com/community/ CommuniGate Pro] 5.2.1 || Yes || Yes || Yes || Cached (some) || No || 8/34 || 8/328 || 0/0 ||
|| [http://cyrusimap.web.cmu.edu/ Cyrus] 2.3.12p2 || No || Unreliable || Bugs || Cached || Delayed || 9/34 || 11/328 || 6/53 ||
|| [http://www.sun.com/software/products/messaging_srvr/index.xml Sun Java Messenging Server] 6.3-0.15 || No || Unreliable || Bugs || Yes || Delayed || 9/34 || 17/328 || 9/21 ||
|| [http://www.archiveopteryx.org/ Archiveopteryx] 3.0.3 || No || Unreliable || No || No || No || 13/38 || 25/346 || 6/26 ||
|| [http://www.courier-mta.org/imap/ Courier] 4.3.1 || Yes || Unreliable || Yes || No || No || 18/34 || 33/328 || 20/53 ||
|| [http://www.gmail.com/ GMail] 2008-09-20 || No || Not implemented || Bugs || Yes (but FETCH FLAGS dies) || No || 15/34 || 36/328 || 0/0 ||
|| [http://drupal.zarafa.com/download-community Zarafa] 6.20 || No || Unreliable || Yes || No || No || 15/40 || 41/403 || 0/0 ||
|| [http://www.zimbra.com/ Zimbra] 5.0.5 || Yes || Yes || Yes || Cached || No || 8/34 || 50/328 || 2/33 ||
|| [http://www.citadel.org/ Citadel] 7.36 || No || Unreliable || Yes || Yes || Session (flag changes are never seen by other sessions) || 19/34 || 98/328 || 0/0 ||
|| [http://www.dbmail.org/ dbmail] 2.3.4 (2.3.x = unstable/dev version - 2.2.x is even worse though) || No || Unreliable || No || Yes || Delayed (always) |||||| Crashes in the middle. ||

Line 45: Line 49:

||
'''Server''' || '''Checkpoint''' || '''\Recent''' || '''Atomic flags''' || '''Expunge fetch''' || '''Expunge store''' || '''Failures''' ||
|| [http://www.ibm.com/software/lotus/products/notes/ IBM Domino] 8.0 || No || Unreliable || ? || ? || No || 16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors ||
|| [http://www.kerio.eu/kms_home.html Kerio Mail Server] 6.5.1 || ? || ? || ? || ? || ? || 18/34 - EXPUNGEs are sent wrong ||
||'''Server''' ||'''Checkpoint''' ||'''\Recent''' ||'''Atomic flags''' ||'''Expunge fetch''' ||'''Expunge store''' ||'''Failures''' ||
||[[http://www.dbmail.org/|dbmail]] 3.0.2 ||? ||Unreliable ||? ||Yes ||Yes || 9/361 - UID/sequence mapping becomes wrong ||
||[[http://www.ibm
.com/software/lotus/products/notes/|IBM Domino]] 8.0 ||No ||Unreliable ||? ||? ||No ||16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors ||
||[[http://www.kerio.eu/kms_home.html|Kerio Mail Server]] 6.5.1 ||? ||? ||? ||? ||? ||18/34 - EXPUNGEs are sent wrong ||
||[[http://www.axigen.com/|Axigen]] 7.1.2 ||? ||Unreliable ||Yes ||Broken ||Broken ||19/312 - FETCH/STORE sends EXPUNGEs immediately ||

IMAP Server Compliancy Status

  • Checkpoint: checkpoint parameter works. When issuing a CHECK command in all sessions, their state looks identical.
  • \Recent: Exactly one session sees a new message as \Recent - no more and no less.
  • Atomic flags: Flags and keywords can be added/removed in multiple sessions without one session dropping changes made by other (own_flags test)
  • Expunge fetch: If message is expunged in one session, can another session that still sees the message fetch its contents?
    • Yes: Yes, everything can be fetched
    • Cached: Only cached data can be fetched. Message header or body can't be.
    • No: Nothing but flags can be fetched.
  • Expunge store: If message is expunged in one session, can another session update its flags?
    • Yes: Yes, and a 3rd session also sees the change
    • Session: Yes, but a 3rd session won't see the change
    • Delayed: Yes, but a 3rd session won't see any changes until EXPUNGEs are reported

    • Ignored: STORE command replies OK, but no change is made
    • No: STORE command replies NO
  • Failures (failures/total): Number of failures using scripted tests. These numbers may not be exact all the time, because the tests are still changing.

    • Failure groups: Each test belongs to a wider group of tests, typically testing a command or part of a command. If this count is low but individual command failure count is high, it probably means that the server has failed to implement wrong only a couple of commands.
    • Base fails: Number of individual base IMAP4rev1 protocol commands that failed.
    • Ext fails: Number of individual IMAP extension commands that failed. Extensions not supported aren't included in the numbers.

Fully compliant servers:

Server

Checkpoint

\Recent

Atomic flags

Expunge fetch

Expunge store

Failure groups

Base fails

Ext fails

Dovecot

Yes

Yes

Yes

Yes / Cached (depends on storage)

Yes

0/40

0/403

0/100

Panda IMAP 2008, mix format

Yes

Yes

Yes

Yes

Session

0/34

0/328

0/97

SurgeMail 5.0h3

Yes

Yes

Yes

Yes

Yes

0/35

0/342

0/26

Non-compliant servers:

Server

Checkpoint

\Recent

Atomic flags

Expunge fetch

Expunge store

Failure groups

Base fails

Ext fails

UW-IMAP 2007b, mix format

Yes

Yes

Yes

Yes

Session

2/34

0/328

6/53

Isode M-Box 14.3a0

No

Unreliable

Yes

Cached

Ignored

4/40

1/408

8/112

CommuniGate Pro 5.2.1

Yes

Yes

Yes

Cached (some)

No

8/34

8/328

0/0

Cyrus 3.0.10

No

Unreliable

Bugs

Cached

Delayed

0/35

0/366

0/100

Sun Java Messenging Server 6.3-0.15

No

Unreliable

Bugs

Yes

Delayed

9/34

17/328

9/21

Archiveopteryx 3.0.3

No

Unreliable

No

No

No

13/38

25/346

6/26

Courier 4.3.1

Yes

Unreliable

Yes

No

No

18/34

33/328

20/53

GMail 2012-08-12

No

Not implemented

Bugs

Cached (some)

Ignored

12/49

66/376

0/0

Zarafa Collaboration Platform 7.1.10

No

Unreliable

Yes

No

No

6/35

14/340

0/0

Zimbra 5.0.5

Yes

Yes

Yes

Cached

No

8/34

50/328

2/33

MS Exchange

No

Unreliable

?

No

No

16/40

52/287

0/0

Citadel 7.36

No

Unreliable

Yes

Yes

Session (flag changes are never seen by other sessions)

19/34

98/328

0/0

hMailServer 5.3.3 b1879

?

?

?

?

?

Fails hardcoded OK response format test. Other tests don't work, if OK response test fails.

Major problems with multiple connections (makes further testing difficult and MAY CAUSE ACCIDENTAL MAIL LOSS!):

Server

Checkpoint

\Recent

Atomic flags

Expunge fetch

Expunge store

Failures

dbmail 3.0.2

?

Unreliable

?

Yes

Yes

9/361 - UID/sequence mapping becomes wrong

IBM Domino 8.0

No

Unreliable

?

?

No

16/34 - Too many EXPUNGEs are sent, EXISTS is dropped before sending EXPUNGEs, FETCHing with valid messagesets produce errors

Kerio Mail Server 6.5.1

?

?

?

?

?

18/34 - EXPUNGEs are sent wrong

Axigen 7.1.2

?

Unreliable

Yes

Broken

Broken

19/312 - FETCH/STORE sends EXPUNGEs immediately

None: ImapTest/ServerStatus (last edited 2019-05-29 06:55:08 by EllieTimoney)