Postfix Overview - Queue Management

Postfix mail queues

Postfix has four different queues: maildrop, incoming, active and deferred (click the upper left-hand icon for the big picture). Locally-posted mail is deposited into the maildrop, and is copied to the incoming queue after some cleaning up. The incoming queue is for mail that is still arriving or that the queue manager hasn't looked at yet. The active queue is a limited-size queue for mail that the queue manager has opened for delivery. Mail that can't be delivered goes to the deferred queue, so that it does not get in the way of other deliveries.

The queue manager keeps information in memory about the active queue only. The active queue size is limited on purpose: the queue manager should never run out of working memory because of a peak message workload. Whenever there is space in the active queue, the queue manager lets in one message from the incoming queue and one from the deferred queue. This guarantees that new mail will get through even when there is a large backlog.

No thundering herd

Implementing a high-performance mail system is one thing. However, no-one would be pleased when Postfix connects to their site and overwhelms it with lots of simultaneous deliveries. This is an issue especially when a site has been down and mail is backed up elsewhere in the network.

Postfix tries to be a good network neighbor. When delivering mail to a site, Postfix will initially make no more than two simultaneous connections. As long as deliveries succeed, the concurrency slowly increases up to some configurable limit (or until the host or network is unable to handle the load); concurrency is decreased in the face of trouble. For those familiar with TCP/IP implementation details, Postfix implements its own analog of the TCP slow start algorithm

Fairness

Apart from the thundering herd controls, the Postfix delivery strategy is based on round-robin selection. The queue manager sorts message recipients in the active queue by destination, and makes round-robin walks along all destination queues.

On the average, Postfix will do simultaneous deliveries to the same domain only when there is not enough work to keep all outbound SMTP channels busy. So, when AOL goes off-line and comes back, it should not stop the system from delivering to other sites.

When mail arrives faster than Postfix can deliver it, Postfix will favor new mail over delayed mail. The idea is that new mail should be delivered with as little delay as possible; delayed mail can be delivered while the system would otherwise be idle.

Exponential backoff

Postfix implements per-message exponential backoff. When a message cannot be delivered upon the first attempt, the queue manager gives the queue file a time stamp that is offset into the future by some configurable amount of time. Queue files with future time stamps are normally ignored by the queue manager.

Whenever a repeat delivery attempt fails, the queue file time stamp is moved into the future by an amount of time equal to the age of the message. Thus, the time between delivery attempts doubles each time. This strategy effectively implements exponential backoff.

Destination status cache

The Postfix queue manager maintains a limited, short-term list of unreachable destinations. This list helps it to avoid unnecessary delivery attempts, especially with destinations that have a large mail backlog.