From: ghe on

On Sep 12, 2009, at 7:44 AM, Wietse Venema wrote:

> Mike Cappella:
> [ Charset ISO-8859-1 unsupported, converting... ]
>> On 9/11/09 5:57 PM, ghe wrote:
>>>
>>> On Sep 11, 2009, at 6:18 PM, Wietse Venema wrote:
>>>
>>>>> Also since I didn't find it, I just re-postmapped 'em all.
>>>>
>>>> Postfix includes the name of the file that is out-of-date:
>>>>
>>>> warning: database /etc/postfix/access.db is older than source
>>>> file /etc/postfix/access
>>>>
>>>> This is logged when the output file has not been updated withing
>>>> 100s
>>>> after the input file was modified.
>>>
>>> Ah!! Logwatch was giving me the wrong string to grep for. "older
>>> than
>>> source" is all over the logs, all complaining about my sender
>>> checks.
>>> Thanks, Wietse.
>>>
>>
>> Most log messages in postfix-logwatch are either canonicalized or
>> summarized. In this case, the message summary change was an
>> attempt at
>> being a little more clear, (but perhaps its not). The
>> canonicalization
>> (and uppercasing of the first word) makes strict grep'ing fail. Eg.,
>> the grep:
>
> I think that it is a mistake to replace the specific
>
> warning: database /etc/postfix/client_checks.db is older than
> source file /etc/postfix/client_checks
>
> By the generic
>
> Database file needs update
>
> Because it keeps the reader in the dark (given the generic text,
> the user does not even know what to look for in the logfile).
>
> I agree that many programs produce useless error messages. In the
> case of Postfix, I actually put in effort to make messages useful.
> So would I appreciate it if other people don't dumb down the
> information that Postfix tries to give.

Or how about maybe a section of the (shorter, summarizing) message
from logwatch, enclosed in single quotes, that would find the mail log
messages?

--
Glenn English
ghe(a)slsware.com



From: Mike Cappella on
On 9/12/2009 5:34 AM, ghe wrote:
>
> On Sep 11, 2009, at 7:21 PM, Mike Cappella wrote:
>
>> Most log messages in postfix-logwatch are either canonicalized or
>> summarized. In this case, the message summary change was an attempt at
>> being a little more clear, (but perhaps its not).
>
> It's great ... when timesSeen > 1.

timesSeen > 1 by who? You or the log *summarizer*? It doesn't know how
many events will occur in the future.

It must stash away canonicalized representations of messages, and store
the variant data as keys.

Otherwise, a) memory consumption would be (more) out of control, and b)
it wouldn't be much of a summarizing.

>
>> The canonicalization (and uppercasing of the first word) makes strict
>> grep'ing fail. Eg., the grep:
>>
>> grep 'database is older than source file' would fail.
>
> Which is why I always leave off the first char.

Glenn, read more carefully what I wrote. Your strategy would also have
failed. There is no 'atabase is older than source file' either. The
challenge is in removing the words/phrases that vary, so that they can
be summarized minimally in the tree.

From the postfix-logwatch(1):
--
Postfix-logwatch is able to produce a wide range of reports with data
grouped and sorted as much as possible to reduce noise and highlight
patterns. Brief summary reports provide a quick overview of general
Postfix operations and message delivery, calling out warnings that may
require attention. Detailed reports provide easy to scan, hierarchi-
cally-arranged and organized information, with as much or little detail
as desired.
--

>
>> I thought the summary message:
>>
>> Database file needs update
>>
>> was clearer than the canonicalized:
>>
>> Database is older than source file
>>
>> esp. in context of the verbose output. Its best to use the summary
>> output as a hint to look at more detailed output, and of course the logs.
>
> It is clearer, and I did look. I just looked for the wrong thing. I
> first searched for the string, then just "update".

As per the man page, don't do that. :-) Look at the details in the
Detail section. It tells you what too look for :

10 *Warning: Database file needs update -----------------
9 /etc/postfix/virtual
^^^^^^^^^^^^^^^^^^^^

> If it had said
> "database file 'client_checks' needs updating" I would have found the
> messages.

It essentially does. It is just that the *noun* has to be moved into
the hierarchically arranged tree... else no tree, and you might as well
just look at the logs. See above.

>
> OTOH, "older than source file" would have echoed what I see all the time
> when I change a file in /etc that postfix keeps in its chroot jail.

If people feel strongly about this, I'm happy to change the summary
header from (a) to (b):

(a) database file needs update
(b) database is older than source file

Please consider and understand that you *others* who don't see these
messages all the time in their log files will be equally confused.

> The real problem, I think, came up because this message is so rare. I'm

Rare for you... but, as you stated ("what I see all the time"), perhaps
not! :-) I can't predict how rare/frequent these lines are for the
population of users: I've seen some logs that are littered with
thousands of these.

> very used to all the others, and the .db files are 'never' out of date
> because I use a shell script to add to them. ... And I didn't know
> postfix was always looking out for me. Nice work, Wietse :-)
>

--
Mike
From: Mike Cappella on
On 9/12/2009 6:44 AM, Wietse Venema wrote:

> I think that it is a mistake to replace the specific
>
> warning: database /etc/postfix/client_checks.db is older than
> source file /etc/postfix/client_checks
>
> By the generic
>
> Database file needs update
>
> Because it keeps the reader in the dark (given the generic text,
> the user does not even know what to look for in the logfile).

It doesn't, not as you state it out of context. The generic message is
only a collapsed Summary pushed to the top of the output for easy
scanning. User's are encouraged to look elsewhere for the real details:

From the postfix-logwatch(1):
--
Postfix-logwatch is able to produce a wide range of reports with data
grouped and sorted as much as possible to reduce noise and highlight
patterns. Brief summary reports provide a quick overview of general
Postfix operations and message delivery, calling out warnings that may
require attention. Detailed reports provide easy to scan, hierarchi-
cally-arranged and organized information, with as much or little detail
as desired.
--

>
> I agree that many programs produce useless error messages. In the
> case of Postfix, I actually put in effort to make messages useful.
> So would I appreciate it if other people don't dumb down the
> information that Postfix tries to give.
>
> Wietse

I don't for a moment question the usefulness of your error messages. I
had a choice: a) present line after line of verbatim log messages, each
varying slightly, or b) pull out and promote the primary, equivalent
parts, moving the varying nouns, subjects, or actions into the hierarchy
to allow increasing disclosure as users want it. My goal was the later.

Once this approach is taken, however, some "sentences", whose subjects
have been moved deeper into the hierarchy, are less readable, or present
ambiguity (eg. if there are two subjects such as a source and destination).

To me, this:

97 *Warning: Database file needs update
----------------------------------------------------
80 /etc/postfix/virtual
17 /etc/postfix/transport

looks better and reads easier than this:

Warnings
--------
cleanup (total: 16)
16 database /etc/postfix/virtual.db is older than source file
/etc...
smtpd (total: 47)
47 database /etc/postfix/virtual.db is older than source file
/etc...
trivial-rewrite (total: 34)
17 database /etc/postfix/virtual.db is older than source file
/etc...
17 database /etc/postfix/transport.db is older than source
file /e...

which a) must needlessly must truncate or line-wrap, b) repeats the same
message over and over, and c) presents a by-program breakdown which is
uninteresting.

So I choose brevity, minimalism, and visual clarity for my use.

See my other response to Glenn.

Anyway, this is likely way off topic, so perhaps this can be pushed over
to the postfix-logwatch list for further discussions.

--
Mike

--

Mike
From: "/dev/rob0" on
On Saturday 12 September 2009 13:19:19 Mike Cappella wrote:
> To me, this:
>
> 97 *Warning: Database file needs update
> ----------------------------------------------------
> 80 /etc/postfix/virtual
> 17 /etc/postfix/transport
>
> looks better and reads easier than this:
>
> Warnings
> --------
> cleanup (total: 16)
> 16 database /etc/postfix/virtual.db is older than source file
> /etc...
> smtpd (total: 47)
> 47 database /etc/postfix/virtual.db is older than source file
> /etc...
> trivial-rewrite (total: 34)
> 17 database /etc/postfix/virtual.db is older than source file
> /etc...
> 17 database /etc/postfix/transport.db is older than source
> file /e...

Agreed, but why not this? To wit:

97 *Warning: Database is older than source file
--------------------------------------------------------
80 /etc/postfix/virtual
17 /etc/postfix/transport

> Anyway, this is likely way off topic, so perhaps this can be pushed
> over to the postfix-logwatch list for further discussions.

Just a suggestion, since I'm not a postfix-logwatch user, I won't be
joining your list.
--
Offlist mail to this address is discarded unless
"/dev/rob0" or "not-spam" is in Subject: header