Imaptest and Archiveopteryx

Imaptest is an IMAP test suite written by Timo Sirainen, better known for writing the IMAP server Dovecot.

Imaptest can perform two kinds of tests: Scripted and random load.

In both modes, you absolutely must use a single user with imaptest. If you use e.g. user=imaptest%d, you will get a large number of false alarms (99% of its alarms came from this single problem when we first tried using imaptest).

When using its test scripts, Imaptest finds no fault with Archiveopteryx 3.0.1.

When using random load, there will be some false alarms. However, as of Archiveopteryx 3.0.2 there are no real alarms, and the number of false alarms is manageable.

The false alarms fall into these categories:

Size changes

Imaptest often reports that the size of something changed. This is because imaptest tests RFC822 format, while Archiveopteryx stores normalized data. For example, if imaptest stores To: "Nirmala" <nirmala@example.com>, then Archiveopteryx will remove the redundant quote marks as part of storing the address, so its size will change by two bytes.

If you import a mailbox into Archiveopteryx and then export it again, the resulting mailbox will not cause false size alarms, at least not with the same version of Archiveopteryx.

Unexpected logouts and NO/BAD errors

Imaptest often sends several commands at once, and expects the IMAP server to carry them out in sequence. If Archiveopteryx decides to log out imaptest while it's in the middle of processing a batch of commands, then the rest of the commands usually fail with a NO message.

This usually happens when imaptest deletes a message in one connection and then fetches it twice in another. Archiveopteryx responds to the first fetch by saying that message is gone, my friend, you can't retrieve it. Imaptest sometimes doesn't take the hint (it is a test program, after all), and when it tries again, Archiveopteryx logs Imaptest out.

It can also happen if a mailbox is renamed or deleted while imaptest is using it.

Instead of NO, some commands can also fail with a parse error such as BAD Need mailbox. This is a rather misleading error message, but difficult to fix and the problem is not important.

Locking problems

Imaptest occasionally causes deadlocks. When that happens, PostgreSQL aborts one of the queries, Archiveopteryx aborts one of the IMAP commands, and Imaptest usually says something about NO Database problem.

We could avoid this by changing locking order, but the order we use is faster for the kinds of things human users do, and we prefer to optimise for humans.

Flag announcements

IMAP servers must report which flags are used in a mailbox when an IMAP client opens that mailbox. They also report whether flags can be written to disk, and whether new flags can be created. Archiveopteryx does so.

This is not the list of flags for each message, it is the grand list of all flags for all messages.

Imaptest expects the IMAP server to send an update for that grand list whenever a message uses a formerly unknown flag. Archiveopteryx does not send any such update. (Archiveopteryx does report the new flag for the message, of course.)

We believe that Archiveopteryx complies with RFC 3501 (specifically section 7.2.6), which requires that the list is sent as part of executing the SELECT and EXAMINE commands, but does not require sending it at any other time.

In case of questions, please write to info@aox.org.

Relevant links

About this page

Last modified: 2010-11-19
Location: aox.org/clients/imaptest