Log file checking
in [Postfix]
|
Prev: problem sending outside of local machine
Next: routing all mails to customized processor, without individual mail directory
From: Sahil Tandon on 31 Jul 2010 20:32 On Sun, 2010-08-01 at 01:53:42 +0200, Mark Scholten wrote: > I want the following information (per day or per hour, it should be possible > to exclude email addresses or to only get information for certain email > addresses): > - Number of email attempts made by other systems > - Number of messages blocked based on the HELO requirements (I have a few > regexp lines with blocked HELOs (botnets/spammers)) > - Number of connections greylisted (we use postgrey) > - Number of attempts for an invalid recipient > - Number of messages blocked based on blacklists > - Number of messages blocked by content filter (not really important) > - Number of messages accepted (not blocked at any stage) I use postfix-logwatch. See: http://logreporters.sourceforge.net/ If it doesn't meet your exact needs, then hack it to do so. If your changes would benefit a wider audience, then share them with Mike. And unless you have a Postfix problem, we are veering off-topic. -- Sahil Tandon <sahil(a)FreeBSD.org>
From: Stan Hoeppner on 31 Jul 2010 21:49 Mark Scholten put forth on 7/31/2010 6:53 PM: > I want the following information (per day or per hour, it should be possible > to exclude email addresses or to only get information for certain email > addresses): /usr/sbin/pflogsumm.pl --smtpd_stats /var/log/mail.log /var/log/mail.log.1 Grand Totals ------------ messages 3658 received 5323 delivered 0 forwarded 480 deferred (2631 deferrals) 1 bounced 1740 rejected (24%) 0 reject warnings 0 held 0 discarded (0%) 25387k bytes received 49655k bytes delivered 825 senders 728 sending hosts/domains 19 recipients 18 recipient hosts/domains > - Number of email attempts made by other systems smtpd 5304 connections 1399 hosts/domains 10 avg. connect time (seconds) 14:54:24 total connect time > - Number of messages blocked based on the HELO requirements (I have a few > regexp lines with blocked HELOs (botnets/spammers)) If these are done with something like "check_helo_access regexp:/etc/postfix/helo.regexp" then you'd see something like this, but with "Helo command rejected: ". I don't do any custom HELO checks, only client checks, but the output is otherwise the same in pflogsumm. Client host rejected: Dynamic - Please relay via ISP (chello.nl) (total: 1) 1 dhcp-077-248-074-059.chello.nl Client host rejected: Dynamic - Please relay via ISP (embarqhsd.net) (total: 1) 1 embarqhsd.net Client host rejected: Dynamic - Please relay via ISP (eunet.rs) (total: 1) 1 dynamic-78-30-138-239.adsl.eunet.rs ** I have separate rejection messages for each expression in my regexp table. Pflogsumm counts each one as distinct, and gives a total for each one, instead of a total for all "custom HELO checks" If you want a singular total for yours, you probably don't want to specify rejection text for each, but use the Postfix default. Doing so should give you the total you want. > - Number of connections greylisted (we use postgrey) Recipient address rejected: Greylisted (total: 30) 30 stan(a)hardwarefreak.com ** greylisting here is used as a last ditch bot blocker. Some call this "very selective greylisting". > - Number of attempts for an invalid recipient Recipient address rejected: User unknown in local recipient table (total: 24) 21 4050505(a)hardwarefreak.com 1 4C4F0705.2050005(a)hardwarefreak.com 1 4c4f17db.7010101(a)hardwarefreak.com 1 4c20361c.7090309(a)hardwarefreak.com > - Number of messages blocked based on blacklists message reject detail --------------------- RCPT Client host rejected: Access denied (total: 262) 22 annaeyes.com ... Client host rejected: Email not accepted from Africa (total: 34) 3 41.140.254.160 ... Client host rejected: Mail not accepted from Belarus (total: 4) 3 93.85.201.97 ... Client host rejected: Mail not accepted from China (total: 23) 6 60.190.77.242 ... Client host rejected: Mail not accepted from Hungary (total: 1) 1 www.imac.hu Client host rejected: Mail not accepted from Indonesia (total: 14) 6 118.96.252.201 ... Client host rejected: Mail not accepted from Korea (total: 32) 3 61.105.220.135 ... Client host rejected: Mail not accepted from Malaysia (total: 1) 1 110.74.129.155 ... Client host rejected: Mail not accepted from Romania (total: 10) 3 81.181.221.62 ... Client host rejected: Mail not accepted from Russia (total: 34) 3 77.34.255.9 ... Client host rejected: Mail not accepted from Thailand (total: 6) 3 113.53.213.186 ... Client host rejected: Mail not accepted from Ukraine (total: 11) 3 79.135.202.145 > - Number of messages blocked by content filter (not really important) Here neither. I don't use content filters. If you saw my entire A/S Postfix config and my user base you'd understand why. > - Number of messages accepted (not blocked at any stage) This is a gripe of my own. Once you get an accurate method for counting this via the mail log, please share it with the pflogsumm dev. My guess is that it's not at all straightforward, due to the multiple delivery methods available. > I did check pflogsumm, however most information isn't provided by pflogsumm > (same for awstats). At least not with the package debian provides. All of the above snippets are from Version: 1.1.0-3 (Lenny) It appears pflogsumm meets all of your requirements but one. Maybe not in the exact mode of operation you'd like, but this is open source code. Change it as you see fit to meet your needs. Just share your patches. :) -- Stan
From: "Mark Scholten" on 1 Aug 2010 06:46 > -----Original Message----- > From: owner-postfix-users(a)postfix.org [mailto:owner-postfix- > users(a)postfix.org] On Behalf Of Stan Hoeppner > Sent: Sunday, August 01, 2010 3:50 AM > To: postfix-users(a)postfix.org > Subject: Re: Log file checking > > Mark Scholten put forth on 7/31/2010 6:53 PM: > > > I want the following information (per day or per hour, it should be > possible > > to exclude email addresses or to only get information for certain > > addresses): > > /usr/sbin/pflogsumm.pl --smtpd_stats /var/log/mail.log > /var/log/mail.log.1 > > Grand Totals > ------------ > messages > > 3658 received > 5323 delivered > 0 forwarded > 480 deferred (2631 deferrals) > 1 bounced > 1740 rejected (24%) > 0 reject warnings > 0 held > 0 discarded (0%) > > 25387k bytes received > 49655k bytes delivered > 825 senders > 728 sending hosts/domains > 19 recipients > 18 recipient hosts/domains > > > - Number of email attempts made by other systems > > smtpd > > 5304 connections > 1399 hosts/domains > 10 avg. connect time (seconds) > 14:54:24 total connect time > > > > - Number of messages blocked based on the HELO requirements (I have a > few > > regexp lines with blocked HELOs (botnets/spammers)) > > If these are done with something like "check_helo_access > regexp:/etc/postfix/helo.regexp" then you'd see something like this, > but with > "Helo command rejected: ". I don't do any custom HELO checks, only > client > checks, but the output is otherwise the same in pflogsumm. > > Client host rejected: Dynamic - Please relay via ISP (chello.nl) > (total: 1) > 1 dhcp-077-248-074-059.chello.nl > Client host rejected: Dynamic - Please relay via ISP > (embarqhsd.net) > (total: 1) > 1 embarqhsd.net > Client host rejected: Dynamic - Please relay via ISP (eunet.rs) > (total: 1) > 1 dynamic-78-30-138-239.adsl.eunet.rs > > ** I have separate rejection messages for each expression in my regexp > table. > Pflogsumm counts each one as distinct, and gives a total for each one, > instead of a total for all "custom HELO checks" If you want a singular > total > for yours, you probably don't want to specify rejection text for each, > but use > the Postfix default. Doing so should give you the total you want. > > > - Number of connections greylisted (we use postgrey) > > Recipient address rejected: Greylisted (total: 30) > 30 stan(a)hardwarefreak.com > > ** greylisting here is used as a last ditch bot blocker. Some call > this "very > selective greylisting". > > > - Number of attempts for an invalid recipient > > Recipient address rejected: User unknown in local recipient table > (total: 24) > 21 4050505(a)hardwarefreak.com > 1 4C4F0705.2050005(a)hardwarefreak.com > 1 4c4f17db.7010101(a)hardwarefreak.com > 1 4c20361c.7090309(a)hardwarefreak.com > > > - Number of messages blocked based on blacklists > > message reject detail > --------------------- > RCPT > Client host rejected: Access denied (total: 262) > 22 annaeyes.com > ... > Client host rejected: Email not accepted from Africa (total: 34) > 3 41.140.254.160 > ... > Client host rejected: Mail not accepted from Belarus (total: 4) > 3 93.85.201.97 > ... > Client host rejected: Mail not accepted from China (total: 23) > 6 60.190.77.242 > ... > Client host rejected: Mail not accepted from Hungary (total: 1) > 1 www.imac.hu > Client host rejected: Mail not accepted from Indonesia (total: 14) > 6 118.96.252.201 > ... > Client host rejected: Mail not accepted from Korea (total: 32) > 3 61.105.220.135 > ... > Client host rejected: Mail not accepted from Malaysia (total: 1) > 1 110.74.129.155 > ... > Client host rejected: Mail not accepted from Romania (total: 10) > 3 81.181.221.62 > ... > Client host rejected: Mail not accepted from Russia (total: 34) > 3 77.34.255.9 > ... > Client host rejected: Mail not accepted from Thailand (total: 6) > 3 113.53.213.186 > ... > Client host rejected: Mail not accepted from Ukraine (total: 11) > 3 79.135.202.145 > > > - Number of messages blocked by content filter (not really important) > > Here neither. I don't use content filters. If you saw my entire A/S > Postfix > config and my user base you'd understand why. > > > - Number of messages accepted (not blocked at any stage) > > This is a gripe of my own. Once you get an accurate method for > counting this > via the mail log, please share it with the pflogsumm dev. My guess is > that > it's not at all straightforward, due to the multiple delivery methods > available. > > > I did check pflogsumm, however most information isn't provided by > pflogsumm > > (same for awstats). At least not with the package debian provides. > > All of the above snippets are from Version: 1.1.0-3 (Lenny) > > It appears pflogsumm meets all of your requirements but one. Maybe not > in the > exact mode of operation you'd like, but this is open source code. > Change it > as you see fit to meet your needs. Just share your patches. :) Getting it in a single number is important for me, however looking at the http://logreporters.sourceforge.net/ link you did give I see that all but one thing is given the way I want it. This last option isn't given the way I like it, but that can be done by parsing the output from postfix-logwatch to combine the last information. Thank you for giving the link. Regards, Mark > > -- > Stan
From: Stan Hoeppner on 1 Aug 2010 14:15
Mark Scholten put forth on 8/1/2010 5:46 AM: > Getting it in a single number is important for me, however looking at the > http://logreporters.sourceforge.net/ link you did give I see that all but > one thing is given the way I want it. This last option isn't given the way I > like it, but that can be done by parsing the output from postfix-logwatch to > combine the last information. Thank you for giving the link. Actually Sahil deserves that credit--I didn't know of logwatch until he mentioned it. And just like you, I prefer the logwatch output over pflogsumm. I've already converted everything over to it here. Thanks again Sahil. -- Stan |