Differences between revisions 24 and 25
Revision 24 as of 2021-10-13 12:51:31
Size: 6896
Editor: TimoSirainen
Comment:
Revision 25 as of 2021-10-13 12:56:39
Size: 6892
Editor: TimoSirainen
Comment:
Deletions are marked like this. Additions are marked like this.
Line 73: Line 73:
 * "? reply" - The reply may or may not exist. This is useful only if the "ignore_extra_untagged" header is enabled. Normally unexpected untagged replies are ignored anyway.  * "? reply" - The reply may or may not exist. This is useful only if the "ignore_extra_untagged" header is "no". By default all unexpected untagged replies are ignored.

ImapTest Scripted Testing Configuration

The imaptest scripts can be used to test command/response pairs. It's especially helpful for matching untagged replies as responses to commands in a way that should work with all IMAP standards compliant servers. This means that the order of untagged replies isn't important, and for many responses the order of the parameters isn't important.

Test files

Each test consists of one or two files in the test directory:

  • <name>

  • <name>.mbox

Messages are appended to the test mailbox from the test.mbox file. If the test-specific mbox file doesn't exist, default.mbox is used instead.

The test file begins with a header, followed by an empty line and then a list of commands.

The header contains "key: value" pairs. Allowed keys are:

  • capabilities: Space-separated list of capabilities required from the server for this test. If server doesn't have these capabilities, the test is skipped. (FIXME: Checking not done currently)
  • connections: Number of connections to use for executing this test. If using 2 or more connections, each command must begin with the connection number which is used for the command (1..n). Default is 1.
  • state: nonauth, auth, created, appended or selected. "auth" makes sure all test mailboxes are deleted before starting the test. "created" then creates the test mailbox. "appended" also appends all mails from the test mbox. "selected" finally selects the mailbox. Default is "selected".
  • messages: How many messages to append to mailbox, or "all". If there are more messages than exist in the mbox file, the reading is wrapped to continue from the beginning of the file. Default used to be "all", but 2021-09-30 it was changed to 0.
  • ignore_extra_untagged: If "no", require that all the untagged replies are explicitly listed in the script. By default these untagged replies are ignored.

For example:

capabilities: CHILDREN LIST-EXTENDED
connections: 2
state: auth

Commands

There are two ways to configure commands:

1)

[<connection #>] OK|NO|BAD|"" <command>
[untagged replies]

2)

[<connection #> <command>
[untagged replies]
OK|NO|BAD|"" [<prefix>]

Connection number is used if the test uses more than one concurrent connection ("connections: n" header). The first way is faster to write, while the second allows matching reply's <prefix>.

For example:

# way 1)
ok select $mailbox
* 0 exists

# way 2)
select $mailbox
* 0 exists
1 ok [read-write]

Untagged replies

Each command can be expected to have zero or more untagged replies. The order of them doesn't matter. The untagged replies can be specified in 3 ways:

  • "* reply" - The reply must exist, or the test fails
  • "! reply" - The reply must not exist, or the test fails
  • "? reply" - The reply may or may not exist. This is useful only if the "ignore_extra_untagged" header is "no". By default all unexpected untagged replies are ignored.

For example to make sure LIST is correctly returning "foo/bar", but not "foo":

ok create foo/bar
ok list "" foo/*
* list () "/" foo/bar
! list () "/" foo

Variables

Commands and replies can have $variables. It can also be written as ${variable}. If a variable doesn't have any value yet when it's matched against server input, the variable is initialized from the server input. This allows doing this like:

ok fetch 1,2 uid
* 1 fetch (uid $uid1)
* 2 fetch (uid $uid2)

ok uid store $uid1,$uid2 flags \seen
* 1 fetch (uid $uid1 flags (\seen))
* 2 fetch (uid $uid2 flags (\seen))

If you want to match the imap argument against anything, use "$". This also works for IMAP (lists), unlike named variables. For example:

ok fetch 1 (uid flags)
# flags $ succeeds against any flags-list, while ($) would match any parameter inside the list.
* 1 fetch (uid $ flags $)

Using $n where n is a number maps to sequences at the beginning of a command. These are useful when receiving EXPUNGEs from another session. For example:

1 ok expunge
2 ok uid fetch 3 flags
# server may send expunge before or after fetch - both match this test
* $2 expunge
* $3 fetch (uid 3 (flags ()))

There are also some predefined variables:

  • $user: user@domain
  • $username: user without @domain
  • $domain: domain
  • $password: Password
  • If there are multiple connections with different usernames, $user2, $user3, $username2, $domain2, etc. are also supported
  • $mailbox: Mailbox used for testing. box command line parameter specifies this. The default is "imaptest".

  • $mailbox_url: IMAP URL for the mailbox

Pipelining

Multiple commands can be pipelined, for example:

tag1 status ${mailbox} (messages)
tag2 status ${mailbox}2 (messages)
* status ${mailbox} (messages 0)
* status ${mailbox}2 (messages 0)
tag1 ok
tag2 ok

Directives

$!directives can be used to alter list matching by placing them at the beginning of a list:

  • $!ordered : The element order in the list must match (default for most lists).
  • $!unordered : The element order in the list doesn't matter. Setting this also allows extra elements to be present.
  • $!unordered=n : Like $!unordered, but list consists of a chain of elements where each chain consists of n elements. For example with FETCH (uid 1 flags (\seen)) the FETCH list would use $!unordered=2 while the flags list would use $!unordered.
  • $!noextra : If $!unordered[=n] directive was used, matching ignores extra elements by default. This requires that all elements must be matched.
  • $!extra : Reverse of $!noextra.
  • $!ignore=e : If $!noextra is used, allow an extra element e to exist in the list.
  • $!ban=e : If $!extra is used, don't allow an extra element e to exist in the list.

If a list has no explicit directives, defaults are used (separately for each list within same command):

  • n FETCH ($!unordered=2)
  • n FETCH (FLAGS ($!unordered $!noextra $!ignore=\recent))
  • LIST ($!unordered)
  • LSUB ($!unordered)
  • STATUS mailbox ($!unordered=2)

NOTE: These defaults within the list aren't used at all if any $! directives are used. For example:

* 1 FETCH (FLAGS ($!extra))

is fully expanded as:

* 1 FETCH ($!unordered=2 FLAGS ($!extra))

So the FLAGS won't have $!unordered or $!ignored=\recent, but the parent FETCH list will have the default $!unordered=2.

Preprocessing

You can use !ifenv, !ifnenv, !else and !endif to run tests only if specified environment variables exist. For example:

!ifenv HAVE_FOO
ok foo
* foo stuff
!endif

The "foo" command is run only have HAVE_FOO environment exists.

Similarly !ifnenv HAVE_FOO block is run only if HAVE_FOO environment doesn't exist.

None: ImapTest/ScriptedTests (last edited 2021-10-13 12:56:39 by TimoSirainen)