[Pkg-awstats-devel] RFC - cron-related stuff

Thu Apr 1 10:31:59 UTC 2010

Hi,

I am worried if too much effort has gone into extending to support 
multiple config files, and too little in securing leaking statistics 
data to non-admins by default.

Sergey, could you please comment on the individual points below (it 
seems that email never was responded to) - especially the last ones 
about what needs to be solved first.

Kind regards,

  - Jonas

On Thu, Apr 16, 2009 at 02:51:20PM +0200, Jonas Smedegaard wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On Thu, Apr 16, 2009 at 02:18:13PM +0400, Sergey B Kirpichev wrote:
>>On Mon, Apr 13, 2009 at 12:25:54PM +0200, Jonas Smedegaard wrote:
>>> This is your approach (please correct me if I am wrong:
>>>
>>> logrotate[root] < httpd > logs[root:awstats]
>>> awstats[awstats] < logs[:awstats] > stats[awstats:www-data]
>>> awstats[www-data] < stats[:www-data] > httpd
>>>
>>> It won't work out of the box: It requires changes to a logrotate
>>> script not owned by our package, so cannot even optionally be
>>> automated by us.
>>
>>Yes.  But it's easy to customize by local admin (one line changes in
>>logrotate.d/* configs).
>
>We should strive towards working-by-default.
>
>
>>But, after our discussion, the second line is:
>>awstats[awstats] < logs[:awstats] > stats[awstats:awstats]
>>No web access.
>
>
>
>Here's a summary of how I understand your proposal (added leading
>interface and paranthesis around routines breaking the chain from
>working out-of-the-box):
>
>cron:   logrotate[root] < httpd > logs[root:adm]
>admin: (edit logrotate to output logs to root:awstats)
>cron:   awstats[awstats] < logs[:awstats] > stats[awstats:awstats]
>admin: (edit cron to output stats to awstats:www-data)
>cgi:    awstats[www-data] < stats[:www-data] > httpd
>
>In other words, you propose to have neither stats nor CGI usable without
>admin customization.  Or did I miss something?
>
>(please explicitly write all lines of your proposed workflow, similar to
>above, instead of only describing your changes to my description - to
>improve our chance of understanding each other)
>
>
>>> When working, raw logfiles are only usable by awstats - other packages
>>> (e.g. alternative log analyzers, perhaps even security surveillance)
>>> cannot concurrently access them without even more custom changes.
>>
>>Yet simple...  Just add them to awstats group (no security issues).
>
>Right.
>
>Simple for an admin to adjust, but unlikely that other packages would
>want to support special-casing for awstats.
>
>
>Here's an improved draft of my proposal:
>
>cron:   logrotate[root] < httpd > logs[root:adm]
>cron:   cp[awstats-adm] < logs[:adm] > logs[awstats-adm:awstats]
>cron:   awstats < logs[:awstats] > stats[awstats:awstats]
>cron:   cp[awstats-adm] < stats[:awstats] > stats[awstats-adm:adm]
>admin: (edit /etc/default/awstats to cp > stats[awstats-adm:www-data]
>web:    awstats[www-data] < stats[:www-data] > httpd
>
>Provides stats out-of-the-box (but readable only my admins).
>
>Can later be enhanced:
>
>  * admin task can be implemented as debconf question in postinst
>  * cp can be replaced with smarter wrapper (gzip, ip resolving, etc.)
>  * tiny piping tool can be provided, by default useless but optionally
>    (through debconf) suid awstatsadm, for safer-than-direct realtime
>    access to logfiles
>
>
>>> This is my approach, improved to not use root access but another user
>>> member of both adm and awstats groups ([:awstats] means read-only by
>>> members of awstats group, writable only by same user):
>>>
>>> logrotate[root] < httpd > logs[root:adm]
>>> awstats-adm[awstats-adm] < logs[:adm] > logs[awstats-adm:awstats]
>>> awstats < logs[:awstats] > stats[awstats:awstats]
>>> awstats[www-data] < stats[:www-data] > httpd
>>>
>>> It won't work out of the box: Stats are generated but accessible to
>>> noone by default.  Changes needed are possible to automate, however,
>>> as they involve only groups memberships and/or our own config files.
>>>
>>> When working, raw logfiles are untouched, so usable by other packages
>>> too.
>>
>>Personally, I'm dislike this copying stuff (as more error prone).
>
>I agree that it requires more disk space.  And that it restricts ability
>to use fancy features of AWStats.  But I cannot see how it is more
>risky. But how is "cp" more error-prone than allow AWStats access to the
>file (and everything else with same access rights!)?
>
>Please elaborate.
>
>
>>> Yes, we can throw all sorts of files in as examples.  Examples are
>>> unreliable, it is something that we do not support.
>>>
>>> With my proposal we can extend our *supported* routines to support
>>> several approaches to virtual hosting.  And we can extend our
>>> *supported* routines to automatically enable fully working,
>>> web-accessible stats (yes, I _do_ want that to be possible, I just
>>> don't want it to be the default).
>>
>>> With my approach we need the location, possibly in multiple scripts,
>>> so it makes sense instead to save the location in
>>> /etc/default/awstats.
>>
>>If user want's piped logs?  Or gzip'ed?  Now it all can be handled in
>>just one place (/etc/awstats/awstats.example.com.conf).
>
>We should provide a working default setup, not (necessarily) an
>infrastructure supporting all features of AWStats: Our users are always
>welcome to do things differently than what we provide by default.
>
>If the user feels too constrained by the safe default workflow we offer,
>then the user can choose to replace parts of our workflow with less
>secure but more flexible parts - e.g. use awstats to resolve, uncompress
>and read input data.
>
>
>
>
>>> Yet another thing the local admin needs to do manually which we
>>> cannot later automate :-(
>>
>>If we have a working setup for parsing stuff (as package is installed),
>>then we would enable crontab entry.
>
>[Please quote only the parts that you respond to.  Above does not seem
>to be a response to the specific quote, but a more general comment]
>
>Do you have ideas on how we get can stats analyzing out-of-the-box?
>Sorry if I missed such idea pointed out earlier.  As I noted further up,
>it seems to me that your proposal cannot (safely) provide this without
>local editing.
>
>
>
>>Right now (and before), default awstats setup doesn't work from
>>begining.
>>
>>So, my suggestion is to merge first static pages stuff (and use
>>separate awstats:awstats user/group for /var/lib/awstats).  I will do
>>this at weekend or on the next week.
>
>I agree that switching default config to produce static pages by default
>is first step, no matter what.
>
>
>
>>And then we can try to tune crontab entries (include some wrapper to
>>access logs in adm group).
>
>Agreed - but without releasing in between.
>
>Either we should roll back the current changes to cron script before our
>next packaging release, or we postpone package release until we have
>something that works by default (i.e. produces analysis, even if not
>accessible by www-data).
>
>Currently _all_ awstats users customize the install.  Which means _all_
>users will need to re-customize each time we change even a comment in
>the packaged conffiles.  So we really should avoid such changes unless
>we provide something that there is a chance that _some_ users can use
>as-is.
>
>
>>> It is better to provide master config below /etc/default than
>>> requiring the admin to edit multiple configfiles in different
>>> formats.
>>>
>>> Also, files below /etc/default must use a specific format that we can
>>> later automate and provide a debconf interface for.
>>>
>>> Would be _very_ nice if we some day could make it possible to preseed
>>> awstats for use out-of-the-box at various security levels.
>>
>>Then, anyway, we _must_ automate also /etc/awstats/awstats._site_.conf
>>stuff (virtual host names etc).  Doesn't look nice.
>>
>>M.b. we can instead use just some sed/awk/grep stuff in
>>/etc/cron.d/awstats to receive LogFile info?
>
>If by default a cron job written by us deliver the logs to awstats, then
>default awstats.conf should simply point to that path we use, and we can
>add a comment recommending to look at /etc/default/awstats before
>changing it (in /etc/default/awstats the user can either change the
>source path to point to some custom location, or comment it out to
>disable the cron job).
>
>Yes, later we can consider doing fancier stuff like parsing all
>awstats.*.conf files (and all files that they include!!!), but I suggest
>we avoid that as it gets too ugly too quick: In addition to supporting
>includes, AWStats also made no promise to keep its logfile format!
>
>
>  - Jonas
>
>- --
>* Jonas Smedegaard - idealist og Internet-arkitekt
>* Tlf.: +45 40843136  Website: http://dr.jones.dk/
>
>  [x] quote me freely  [ ] ask before reusing  [ ] keep private
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.9 (GNU/Linux)
>
>iEYEARECAAYFAknnKcgACgkQn7DbMsAkQLj7BwCgmLZ0d/zT3o8VrvEcffIPka6T
>yXwAoI8SUTHqyIeRRsk5uATqc6h5WeZc
>=08Ic
>-----END PGP SIGNATURE-----
>
>_______________________________________________
>Pkg-awstats-devel mailing list
>Pkg-awstats-devel at lists.alioth.debian.org
>http://lists.alioth.debian.org/mailman/listinfo/pkg-awstats-devel

-- 
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136  Website: http://dr.jones.dk/

  [x] quote me freely  [ ] ask before reusing  [ ] keep private
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-awstats-devel/attachments/20100401/2a6fff28/attachment.pgp>