Differences between revisions 9 and 22 (spanning 13 versions)
Revision 9 as of 2008-02-29 03:30:52
Size: 3490
Editor: TimoSirainen
Comment:
Revision 22 as of 2018-05-30 07:29:49
Size: 5618
Editor: TimoSirainen
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= ImapTest Command/Reply Testing Configuration = = ImapTest Scripted Testing Configuration =
Line 16: Line 16:
 * state: nonauth, auth or selected. If selected is used, the mailbox is recreated and test messages are appended to it automatically. Default is selected.  * state: nonauth, auth, created, appended or selected. "auth" makes sure all test mailboxes are deleted before starting the test. "created" then creates the test mailbox. "appended" also appends all mails from the test mbox. "selected" finally selects the mailbox. Default is "selected".
 * messages: How many messages to append to mailbox. If there are more messages than exist in the mbox file, the reading is wrapped to continue from the beginning of the file. Default is all.
 * ignore_extra_untagged: If "no", require that all the untagged replies are explicitly listed in the script. By default these untagged replies are ignored.
Line 73: Line 75:
$n where n is a number maps to sequences at the beginning of a command. These are useful when receiving EXPUNGEs from another session. For example: If you want to match the imap argument against anything, use "$". This also works for lists, unlike named variables. For example:

{{{
ok fetch 1 (uid flags)
* 1 fetch (uid $ flags $)
}}}

Using
$n where n is a number maps to sequences at the beginning of a command. These are useful when receiving EXPUNGEs from another session. For example:
Line 85: Line 94:
 * $user: user@domain
 * $username: user without @domain
 * $domain: domain
 * $password: Password
 * If there are multiple connections with different usernames, $user2, $user3, $username2, $domain2, etc. are also supported
Line 86: Line 100:
 * $mailbox_url: IMAP URL for the mailbox

== Pipelining ==

Multiple commands can be pipelined, for example:

{{{
tag1 status ${mailbox} (messages)
tag2 status ${mailbox}2 (messages)
* status ${mailbox} (messages 0)
* status ${mailbox}2 (messages 0)
tag1 ok
tag2 ok
}}}
Line 94: Line 122:
 * $!noextra : If $!unordered[=n] directive was used by default matching ignores extra elements. This requires that all elements must be matched.  * $!noextra : If $!unordered[=n] directive was used, matching ignores extra elements by default. This requires that all elements must be matched.
Line 96: Line 124:
 * $!ignore=e : If $!noextra is used, allow an extra element e to exist in the list.
 * $!ban=e : If $!extra is used, don't allow an extra element e to exist in the list.
Line 100: Line 130:
 * n FETCH (FLAGS ($!unordered $!noextra))  * n FETCH (FLAGS ($!unordered $!noextra $!ignore=\recent))
Line 102: Line 132:
 * LSUB ($!unordered)
Line 103: Line 134:

NOTE: These defaults within the list aren't used at all if any $! directives are used. For example:

{{{
* 1 FETCH (FLAGS ($!extra))
}}}

is fully expanded as:

{{{
* 1 FETCH ($!unordered=2 FLAGS ($!extra))
}}}

So the FLAGS won't have $!unordered or $!ignored=\recent, but the parent FETCH list will have the default $!unordered=2.

== Preprocessing ==

You can use `!ifenv`, `!ifnenv`, `!else` and `!endif` to run tests only if specified environment variables exist. For example:

{{{
!ifenv HAVE_FOO
ok foo
* foo stuff
!endif
}}}

The "foo" command is run only have `HAVE_FOO` environment exists.

Similarly `!ifnenv HAVE_FOO` block is run only if `HAVE_FOO` environment doesn't exist.

ImapTest Scripted Testing Configuration

The tests consist of two files in the test directory:

  • <name>

  • <name>.mbox

The test file begins with a header, followed by an empty line and then a list of commands. Messages are appended to the test mailbox from the test.mbox file.

The header contains "key: value" pairs. Allowed keys are:

  • capabilities: Space-separated list of capabilities required from the server for this test. If server doesn't have these capabilities, the test is skipped. (FIXME: Checking not done currently)
  • connections: Number of connections to use for executing this test. If using 2 or more connections, each command must begin with the connection number which is used for the command (1..n). Default is 1.
  • state: nonauth, auth, created, appended or selected. "auth" makes sure all test mailboxes are deleted before starting the test. "created" then creates the test mailbox. "appended" also appends all mails from the test mbox. "selected" finally selects the mailbox. Default is "selected".
  • messages: How many messages to append to mailbox. If there are more messages than exist in the mbox file, the reading is wrapped to continue from the beginning of the file. Default is all.
  • ignore_extra_untagged: If "no", require that all the untagged replies are explicitly listed in the script. By default these untagged replies are ignored.

For example:

capabilities: CHILDREN LIST-EXTENDED
connections: 2
state: auth

Commands

There are two ways to configure commands:

1)

[<connection #>] OK|NO|BAD|"" <command>
[* <tagged reply>] (0 or more)

2)

[<connection #> <command>
[* <tagged reply>] (0 or more)
OK|NO|BAD|"" [<prefix>]

Connection number is used if there are more than one connection. The order of untagged replies doesn't matter. The first way is faster to write, while the second allows matching reply's <prefix>.

For example:

# way 1)
ok select $mailbox
* 0 exists

# way 2)
select $mailbox
* 0 exists
1 ok [read-write]

Variables

Commands and replies can have $variables. If a variable doesn't have value when it's matched against server input, the variable is initialized from the server input. This allows doing this like:

ok fetch 1,2 uid
* 1 fetch (uid $uid1)
* 2 fetch (uid $uid2)

ok uid store $uid1,$uid2 flags \seen
* 1 fetch (uid $uid1 flags (\seen))
* 2 fetch (uid $uid2 flags (\seen))

If you want to match the imap argument against anything, use "$". This also works for lists, unlike named variables. For example:

ok fetch 1 (uid flags)
* 1 fetch (uid $ flags $)

Using $n where n is a number maps to sequences at the beginning of a command. These are useful when receiving EXPUNGEs from another session. For example:

1 ok expunge
2 ok uid fetch 3 flags
# server may send expunge before or after fetch - both match this test
* $2 expunge
* $3 fetch (uid 3 (flags ()))

There are also some predefined variables:

  • $user: user@domain
  • $username: user without @domain
  • $domain: domain
  • $password: Password
  • If there are multiple connections with different usernames, $user2, $user3, $username2, $domain2, etc. are also supported
  • $mailbox: Mailbox used for testing. box command line parameter specifies this. The default is "imaptest".

  • $mailbox_url: IMAP URL for the mailbox

Pipelining

Multiple commands can be pipelined, for example:

tag1 status ${mailbox} (messages)
tag2 status ${mailbox}2 (messages)
* status ${mailbox} (messages 0)
* status ${mailbox}2 (messages 0)
tag1 ok
tag2 ok

Directives

$!directives can be used to alter list matching by placing them at the beginning of a list:

  • $!ordered : The element order in the list must match (default for most lists).
  • $!unordered : The element order in the list doesn't matter. Setting this also allows extra elements to be present.
  • $!unordered=n : Like $!unordered, but list consists of a chain of elements where each chain consists of n elements. For example with FETCH (uid 1 flags (\seen)) the FETCH list would use $!unordered=2 while the flags list would use $!unordered.
  • $!noextra : If $!unordered[=n] directive was used, matching ignores extra elements by default. This requires that all elements must be matched.
  • $!extra : Reverse of $!noextra.
  • $!ignore=e : If $!noextra is used, allow an extra element e to exist in the list.
  • $!ban=e : If $!extra is used, don't allow an extra element e to exist in the list.

If a list has no explicit directives, defaults are used (separately for each list within same command):

  • n FETCH ($!unordered=2)
  • n FETCH (FLAGS ($!unordered $!noextra $!ignore=\recent))
  • LIST ($!unordered)
  • LSUB ($!unordered)
  • STATUS mailbox ($!unordered=2)

NOTE: These defaults within the list aren't used at all if any $! directives are used. For example:

* 1 FETCH (FLAGS ($!extra))

is fully expanded as:

* 1 FETCH ($!unordered=2 FLAGS ($!extra))

So the FLAGS won't have $!unordered or $!ignored=\recent, but the parent FETCH list will have the default $!unordered=2.

Preprocessing

You can use !ifenv, !ifnenv, !else and !endif to run tests only if specified environment variables exist. For example:

!ifenv HAVE_FOO
ok foo
* foo stuff
!endif

The "foo" command is run only have HAVE_FOO environment exists.

Similarly !ifnenv HAVE_FOO block is run only if HAVE_FOO environment doesn't exist.

None: ImapTest/ScriptedTests (last edited 2023-11-07 21:12:13 by TimoSirainen)