Defining the workgroup objectives

Sven Mueller debian at incase.de
Mon Aug 8 16:46:53 UTC 2005


Gerrit Pape wrote on 05/08/2005 21:33:
> On Fri, Aug 05, 2005 at 01:18:54AM +0200, Sven Mueller wrote:
>>Ok, real-world example:
>>drbd (network shared) partition must be mount for
>>mysql daemon to be able to start.
>>mysql must be up and running for postfix to start
>>spampd depends on mysql as well and is needed by postfix
>>postgrey depends on mysql and is needed by postfix
>>Note that the way postfix is configured, it disables greylisting if
>>postgrey isn't up, but starts and works as intended otherwise (accepts
>>mail....)
>>
>>So this is:
>>drbd
>>	depends on networking
>>mysql
>>	depends on drbd
>>postgrey
>>	depends on mysql
>>spampd
>>	depends on mysql
>>postfix
>>	depends on mysql
>>	depends on spampd
>>	depends on postgrey
>
>
> Great, thanks for your interest.  The network is not considered as a
> service in runit currently, but as a 'system's one time task' on boot.

Alright, I accept that.

> So should be drbd I think, it doesn't use a daemon process.

Actually, drbd consists of two parts in my context:
1) The kernel module
2) drbdadm (which sets the drbd disc in question to primary mode and
    mounts it)
Note that the drbd disc can only be mounted on one machine and only that
machine will be able to start the other processes. In a way, the drbd
disc mounted should be equivalent to a service. And currently it is:
/etc/init.d/drbd start
  sets the drbd disc to primary mode and mounts it
/etc/init.d/drbd stop
  umounts the disc and sets it to secondary mode

> I'll start with mysql.  To actually follow the examples, you don't
need to replace
> sysvinit, just install the runit package, not the runit-run package.
>
> Create a mysql run script /etc/mysql/run, and make it executable
>  #!/bin/sh
>  exec 2>&1
>  exec mysqld_safe
>
> Stop mysql through the init script, and disable the init script.  Enable
> the runit service by linking the service directory into /var/service/

More out of curiosity: Does /var/service conform to the FHS?
I don't know for sure, but my quess is it doesn't and would therefore
conflict with Debian policy.

>  # ln -s /etc/mysql /var/service/
>
> You can check the service with 'sv status mysql'.
>
> Now postgrey, create /etc/postgrey/run, and make it executable
>  #!/bin/sh
>  exec 2>&1
>  test -S /var/run/mysqld/mysqld.sock || exit 1
>  exec postgrey --inet=127.0.0.1:60000
>
> Run /etc/init.d/postgrey stop, disable the init script, enable the runit
> service, and check with 'sv status postgrey'
>
>  # ln -s /etc/postgrey /var/service/
>
> In the postgrey run script the 'test -S /var/run/mysqld/mysqld.sock'
> checks for the existence of the mysql socket, and the postgrey service
> daemon is not started until it's there (better would be to actually try
> to connect to the socket).

So runit doesn't implement dependencies, only the run scripts do. I
could get the same with sysvinit-style init-scripts: Just loop in the
init script until the dependency is fulfilled and then start the
service. Actually, you would have to do a bit more since you would
either have to make sure that /etc/init.d/rc backgrounds the scripts
it's calling or make sure the init script is returning even if still
waiting for the service to actually start.

[..}

> Finally postfix, create /etc/postfix/run[0], and chmod +x
>
>  #!/bin/sh
>  exec 2>&1
>  nc -z 127.0.0.1 10025 || exit 1
>  nc -z 127.0.0.1 60000 || exit 1
>  exec /usr/lib/postfix/master
>
> netcat is used to check for the availablity of the postgrey and spampd
> services, master is not started until connecting to both succeeded.  Run
> /etc/init.d/postfix stop, disable the init script, and

Nice trick with nc, I really should keep that in mind. Currently I check
wether a pid file exists and the corresponding /proc/<pid>/exe points at
the right place (resp. /proc/<pid>/cmdline matches). Using netcat is
more appropriate though. Thanks.

>  # ln -s /etc/postfix /var/service/
>
> Hopefully all went fine, check with
>
>  # sv status postfix mysql postgrey spampd
>
> These should be the basic run scripts for these services.  On system
> boot, all will be started (nearly) at once, and the dependencies sort
> out automatically.  More importantly, the dependencies also are
> controlled while whole system uptime, not only on startup.

Mostly: If postfix doesn't terminate just because postgrey crashes, it
will keep accepting connections and deny them temporarily. However _if_
a service terminates, it will obviously only get restarted once its
dependencies are fulfilled again.

>>>Take a look an the sv(8) man page, it implements an LSB init scripts
>>>interface, and lets you send signals to the service daemon, reliably,
>>>without the need of pid files.
>>
>>Cool, once again something I overlooked. Seems I got carried away by the
>>(simple) way you described runit in your mails.

But now it seems I wasn't. runit doesn't seem to implement dependencies
at all. It completely relies on the runscripts to do that. This is
almost exactly what I thought from the beginning. Mind you, this doesn't
make runit a bad system, just unsuitable for what _I_ am looking for.

>>>>What I really expect of any init system are various things:
>>>>1) Make sure no service is started when its dependencies are not met
>
> You can now try, disable the spampd service, and then try to restart
> postfix
>  # sv stop spampd
>  ok: down: spampd: 0s, normally up
>  # sv restart postfix
>  timeout: down: postfix: 0s, normally up, want up
>  #
> You might as well get a success message here, depending on your system.
> Check with 'sv status spampd postfix', or 'sv status /var/service/*'
>  # sv status spampd postfix
>  down: spampd: 31s, normally up
>  down: postfix: 1s, normally up, want up
>  #
> runit will try to restart the postfix service every second, the
> configuration is inconsistent, and it tries to recover.  To make the
> service configuration consistent, stop the postfix service manually, or
> see 2).

As indicated above: This certainly isn't a bad system, but it doesn't
implement dependencies in itself. It relies on the run scripts to
_implement_ them, not just to _specify_ them. What I want is a system
where an init-/runscript only says "I depend on service X" and the
system makes sure it is only started when X is available, while X
provides a means of asking it wether it is truely available (though I
could imagine that an runit-like system just assumes X is up when the
runscript doesn't terminate within some seconds).

>>>>2) Allow a graceful shutdown, taking services down before their
>>>>  dependencies are stopped
>
> Ok, let's say if the postgrey service is stopped, the postfix service
> should be stopped also, before postgrey exits.  You can tell runit to do
> so through a 'control' hook, see runsv(8).  Provide a script for the
> postgrey service that will be run when the service should be taken down,
> and make it executably (you'll need to create the control subdirectory).
> Create /var/service/postgrey/control/d
>  #!/bin/sh
>  sv stop postfix || exit 0
>  exit 1
>
> Now first start all four services again, and then take down the postgrey
> service, check with 'sv status'
>  # sv start spampd mysql postgrey postfix
>  ok: run: spampd: (pid 3228) 0s
>  ok: run: mysql: (pid 3025) 68s
>  ok: run: postgrey: (pid 3075) 67s
>  ok: run: postfix: (pid 3242) 0s
>  # sv stop postgrey
>  ok: down: postgrey: 0s, normally up
>  # sv status postfix postgrey
>  down: postfix: 24s, normally up
>  down: postgrey: 24s, normally up
>  #

Ok. However I asked for the system to tell the admin which services are
actually being stopped/started. In this case, it looks as if I only
stopped postgrey even though both postgrey and postfix have been stopped
by my "stop postgrey" command.

> If you want to have the postgrey service automatically taken up when
> runit is told to start the postfix service, change the postfix run
> script like this
>  #!/bin/sh
>  exec 2>&1
>  sv start spampd
>  nc -z 127.0.0.1 10025 || exit 1
>  sv start postgrey
>  nc -z 127.0.0.1 60000 || exit 1
>  exec /usr/lib/postfix/master

Again, runit doesn't handle the dependencies at all, it completely
relies on the runscripts (or corresponding control hooks) to do that.
IMHO, this is suboptimal because it makes setup much more complex for
the admin without a matchingly high benefit.

>>>>3) Allow manual stopping and starting of services, handling dependencies
>>>>  and giving feedback what services were actually started/stopped or
>>>>  failed to start/stop
>
> We have seen starting/stopping, check which service are running or fail
> to start with 'cd /var/service && sv status *'.

As said above, this is no completely satisfying solution as it seems to
suppress part of the feedback I asked for.

>>>>4) Allow some equivalent to the /etc/init.d/<service> (force-)?reload
>>>>  functionality
>
> Divert the spampd init script, and replace it with a symlink to
> /usr/bin/sv, then run '/etc/init.d/spampd force-reload'.

What does this resolve to? I.e. what does the runit system do if told to
reload or force-reload respectively? Does it always send some signal to
the process it is controlling? How can I control which signal that is?
Because even though HUP is generally accepted as the signal to reload
configuration, some daemons/programs expect USR1 instead.

>>>>On a truely dependency based init system, I would also expect the system
>>>>to allow the local admin to:
>>>>a) ask the system which services some service depends on
>>>>b) ask the system which services depend on some service
>
> This also is possible, but would require some better organization
> of the dependency information, I haven't documented yet anythin about
> it.

Right. Because it seems that runit after all doesn't handle
dependencies. The run-scripts it calls do that. But to me those belong
to the service itself, not to runit. As such, I would expect that
whoever maintains a service-providing package X (such as I do with
spampd) to provide the runscript and some information on which other
services X always depends (plus possibly some information about services
it can depend on depending on configuration). Ideally, this dependency
information should be provided in the form of some script output, so
that the maintainer can provide a script which generates the list of
dependencies based on actual configuration instead of statically
providing that information. The init system should provide some means to
the admin to override that dependency-information, explicitly adding or
removing dependencies to that dynamic list.

An example is again spampd: It normally doesn't depend on any other
daemon, just as spamassassin doesn't need any running daemon. However,
spamassassin (and thus spampd) can be configured to use SQL database
tables. Now, the dependency information script for spampd would normally
return an empty string as its output. However, when configured to use
some mysql database for whitelisting (as an example), it would return
"mysql". Now imagine the following: spamassassin is configured to use
DCC (distributed checksum clearinghouse), but DCC is configured not to
use the publicly available server, but a private one on the same host.
In this case, the local admin would want to have DCC started/running
before spampd starts, but spampd can't figure this dependency out
automatically. Furthermore, the spamassassin setup is made to use a
non-local mySQL database but this isn't immediately obvious from the
configuration. In this case, the local admin would want to remove mysql
from the list of dependencies. The overrides for this could be specified
in a file like this (again, only an example, but it could be some
starting point):
spampd: -mysql +dccd

> I hope the examples help.  I admit, runit does things differently, and
> it's most probably not easy to grasp at a first glance, if you're not
> familiar with it.

Well, the examples certainly helped. And if you think of the run scripts
as an actual part of runit itself, you could even say it implements a
dependency based system. But with the same justification, I could
implement such dependencies in sysvinit-Scripts and call that a
dependency based system. You understand what I mean? I would expect that
a true dependency based system grabs dependency information from some
source (as I indicated above, I would prefer them to be dynamically
generated from scripts), builds a dependency tree and acts according to
that tree. This would allow run/start scripts to be really dumb, with
the dependency handling implemented only in one single place. If a
service has dependencies which could change with configuration, it would
need to either make it explicit that the admin needs to cover this or
provide some semi-intelligent script which calculates them. An example
would be spampd, though I remove most of the commandline options in this
example to make my point more obvious:

/etc/initsys/spampd/run:
#!/bin/sh
exec spampd --no-detach

/etc/initsys/spampd/avail:
#!/bin/sh
if nc -z 127.0.0.1 10025; then
	echo spampd available
	exit 0
else
	echo spampd unavailable
	exit 1
fi

/etc/initsys/spampd/depends:
#!/bin/sh

if grep -qE '_dsn.*DBI:mysql:.*:localhost'
            /etc/spamassassin/local.cf 2>/dev/null; then
    MYSQL="mysql "
fi
# something similar for postgresql storing the result in POSTGRES
echo "${MYSQL}${POSTGRES}"

/etc/initsys/spampd/prestart:
#!/bin/sh
# whatever needs to be done immediately before spampd gets started

[...]

Get the idea? An idea I saw in some program a while back (I don't
remember which) was to make a configuration file a plain file if it is
not executable and interpret its content in that case, while calling it
and interpreting the output if it has the executable bit set. In that
case an /etc/initsys/<service>/depends file could be a plain text file
containing the static dependencies for those services which only have
static dependencies, while it could stay being a script for those
services with more complex dependencies.

> [0] Most probably it's desired to run the postfix checks before starting
> the master, this seems to work:
[...]

Might be good to do that, right. BTW: What is the difference to runit if
a runscript terminates with an exitcode of 1 resp. 0?

cu,
sven
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 186 bytes
Desc: OpenPGP digital signature
Url : http://lists.alioth.debian.org/pipermail/initscripts-ng-devel/attachments/20050808/496af404/signature.pgp


More information about the initscripts-ng-devel mailing list