Next Previous Contents

16. Optimizing list performance.

Ezmlm-idx is designed to make it as easy as possible to set up mailing lists. The default setup works well for small and medium-sized lists. For large lists, the lists can be made more efficient with a few simple changes.

16.1 Crond-generated digests for better performance.

With the default setup, ezmlm-tstdig(1) in DIR/editor tests if a digest should be sent out. On lists with a lot of traffic this is inefficient. Also, you may want digests to be delivered as a specific time. To do this, use crond(8) to execute ezmlm-get(1) directly, as described elsewhere.

16.2 Optimizing execution of ezmlm-warn(1).

ezmlm-idx>=0.32 comes with much improved bounce handling. Modification as described below should be considered only when you expect thousands of bouncing addresses (virtually never). The description remains, for users of ezmlm-0.53 or earlier versions of ezmlm-idx. For users of ezmlm-0.53 alone, we recommend a patch ( ftp://ftp.id.wustl.edu/pub/patches/ezmlm-return.diff which fixes a bug in ezmlm-0.53 bounce handling. The patch is superseded by ezmlm-idx.

To redistribute the load of bounce warning and probe addresses to off-peak hours, you may want to set up the list without ezmlm-warn(1) by using the ezmlm-make ``-w'' switch, and instead execute ``ezmlm-warn DIR'' via crond(8). You also need to run ``ezmlm-warn -d DIR'' for digest bounces if your list is configured with digests. Normal ezmlm list with ezmlm-idx>=0.32 will have an insignificant bounce load, except if you bulk add addresses, e.g. from a MLM without bounce handling. In the latter case, the load will be higher for the first 2-4 weeks, then decrease drastically. If you feel you need to run ezmlm-warn(1) from crond(8), you should seriously consider sublisting your lists.

Note: the ezmlm-make(1) ``-w'' switch has a special meaning if used at the same time as enabling SQL-support (``-6''; see man pages).

16.3 Decreasing ezmlm-warn time out to increase performance.

With ezmlm-idx, you may alter the ezmlm-warn(1) timeout to a number of seconds with the ``-t seconds'' switch. The default is 1,000,000 seconds or about 11.6 days. This is the time from the first bounce until ezmlm-warn(1) sends a warning message and the time from the warning message bounce until ezmlm-warn(1) sends a probe (which if bounced leads to removal of the address from the subscriber list). If you have a digest list, remember to execute ezmlm-warn(1) with the ``-d'' switch as well.

Decreasing the default to e.g. 5 days will cut in half the average number of files in the bounce directory and the number of messages sent at each crond(8)-directed invocation of ezmlm-warn(1). The trade-off is that worst case, a subscriber may be unsubscribed if his/her mail path is defective for more than twice the timeout. Removing a subscriber after 10 days seems reasonable on a busy list. Do this by adding the ``-t'' switch to all the ezmlm-warn(1) invocations. This timeout should be larger than the interval between ezmlm-warn(1) invocation.

To be aggressive, use ``ezmlm-warn -t0''. This will minimize the time your lists spends servicing bounces, but will for some errors lead to subscribers to be also lead to subscribers being removed if messages to them bounce for two consecutive ezmlm-warn(1) runs. This is useful to rapidly clean up a low quality address collection.

16.4 Use ezmlm without ezmlm-idx for maximum performance.

ezmlm-idx adds a number of functions to ezmlm. It indexes the archive, and adds an index entry for each message, it can remove MIME parts, it can add a subject prefix and message trailer, decode rfc2047-encoded subjects, etc. Although designed to impact minimally on performance, these options when used take time. Even when they are not used, time is spent looking for e.g. the prefix. However, the performance penalty is small, as the absolutely dominating cost of a mailing list is the work qmail does to deliver the messages to subscribers.

In bench marking, we have not found a significant difference in performance between ezmlm-0.53 and ezmlm-0.53+ezmlm-idx-0.32 when ezmlm-idx features are not used. Thus, a non-indexed list with ezmlm-idx-0.32 performs the same as the corresponding ezmlm-0.53 list. Adding an index adds the overhead of another safe write (the index file). Use of other features adds very marginally to execution time. For virtually all lists, the ezmlm execution time is negligible compared to the resources needed by qmail to disseminate the message to the subscribers.

16.5 Not archiving to maximize performance.

An archived list needs to write the message to the archive. If you don't need an archive, don't archive. However, the archive is very useful to allow users to catch up on messages that they didn't receive due to delivery problems.

16.6 Sublists to maximize performance.

Consider splitting your list into sublists, ideally geographically. The main list deals only with a subset of subscribers (or only the sublists), and each sublist deals with a subset of subscribers, bounces, etc. This is the most rational way to scale ezmlm to large lists (see How sublists work for more info on how sublists work and Sublists on how to set up sublists).


Next Previous Contents