qmail delivery statistics

The following graphs show the qmail-1.03 delivery behaviour, sending about 93,000 messages of a weekly newsletter list, injected by ezmlm-0.53 + ezmlm-idx-0.40.

The comparison is done for a delivery cycle with concurrencyremote set to 500, 250 and 150. The data is taken from three successive delivery cycles (each 1 week later, list has lost about 1% (i.e. 1000) of subscribers with each successive cycle).

All the graphs start at the time the message was injected by ezmlm.

Number of concurrent remote deliveries per second
Number of finishing successful deliveries at that second
(click images to enlarge)
This graph shows a snapshot of about 4000 seconds.
The peaks following the first delivery cycle most probably result from retransmits of the messages that couldn't be delivered during the first cycle.
This graph is a zoom in to the first 1500 seconds of the same data as on the left side.

comparison of failed/deferred to successful deliveries
cut at 200 (y-axis), zoomed in to the first 1500 seconds (x-axis)
(click images to enlarge)
concurrencyremote 150 concurrencyremote 250 concurrencyremote 500
Numbers accounted to the second the delivery started
Numbers accounted to the second the delivery ended.

duration of successful deliveries minimum/median/maximum
zoomed in to the first 1500 seconds (x-axis)
numbers accounted to the second the delivery started
(click images to enlarge)
concurrencyremote 150 concurrencyremote 250 concurrencyremote 500
cut at 1000 seconds (y-axis)
cut at 500 seconds (y-axis)
cut at 200 seconds (y-axis)

Some bandwidth information for the 150 data

This graph shows bandwidth consumption during the delivery. The graph is generated from 3-minute probes taken from the machine with the command

netstat -biI <eth_if>
The x-axis is the time of the day (i.e. 12:00 p.m. to 15:00 p.m.), the y-axis is median bandwidth consumption per 3 minute intervals calculated to KBytes/second.
Thus one can see from the graph that the maximum bandwidth consumption has been at around 6 Mbps. The machine has a 100 Mbps 3Com card and is well connected to a 100 Mbps switch. Our backbone is multihomed with lines in the range of 34 Mbps - 150 Mbps. Network congestion should not be a limiting factor here.
(Sorry, I don't have bandwidth info for the 250 and 500 data).

Bandwidth consumption (KBytes/s)
(click images to enlarge)
concurrencyremote 150

Some additional information from the 500 data

  • qmail has finished the first delivery cycle after about 2500 seconds (40 minutes).
  • at that time 90800 messages have been successfully sent.
    • messages have been sent to 5400 unique ip addresses
    • there is one IP address that had received 20303 messages
    • 90% of all messages are sent to only 300 unique IP addresses
    • that means that 90% of all messages got delivered to only 5.5% of all hosts.

To generate the following graphs I have counted the number of deliveries per each unique IP address and sorted them descending by this number. For the graphs I have used these data series and successivly summed up the number of IP addresses and the number of deliveries.

x-axis: no of unique IP addresses (linear)
y-axis: percentage of deliveries (linear)
x-axis: no of unique IP addresses (logarithmic)
y-axis: percentage of deliveries (linear)
 
x-axis: percentage of unique IP addresses (linear)
y-axis: percentage of deliveries (linear)
 

System information

FreeBSD 3.4-RELEASE
CPU: Pentium III (551.25-MHz 686-class CPU)
real memory = 268435456 (262144K bytes)
ahc0: rev 0x00 int a irq 10 on pci0.11.0
ahc0: aic7890/91 Wide Channel A, SCSI Id=7, 16/255 SCBs
da0 at ahc0 bus 0 target 0 lun 0
da0: Fixed Direct Access SCSI-2 device
da0: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled
da0: 30000MB (61440000 512 byte sectors: 255H 63S/T 3824C)
xl0: <3Com 3c905B-TX Fast Etherlink XL> rev 0x30 int a irq 12 on pci0.9.0
xl0: autoneg complete, link status good (full-duplex, 100Mbps)

qmail-1.03 + bigconcurrency modifications (concurrencyremote=500)

ezmlm-0.53 + ezmlm-idx-0.40

Thanks to

  • Peter van Dijk, for his thoughts and comments
  • Steff Hoehne, for her patience and understanding ;-)
Copyright © 1997-2007 Markus Stumpf (Maex) · created: Tue Apr 10 22:23:14 CEST 2001 · last modified: Sat Jun 2 23:34:02 CEST 2007