[Pkg-puppet-devel] Splitting puppetq?

[Digant C Kasundra]

----- "Nigel Kersten" <nigel at explanatorygap.net> wrote:

> On Fri, Jan 29, 2010 at 10:36 AM, Digant C Kasundra
> <digant at stanford.edu> wrote:
> >
> >> I've yet to do much more than play with queuing, so you can
> describe
> >> the scenario you have where you'd want puppetqd on a host that
> >> doesn't
> >> have puppetmasterd installed?
> >
> > Absolutely.  We commissioned the puppetq work specifically for the
> purpose of having storeconfigs running on a different server than the
> puppetmasters.  When we turned on storeconfigs on the puppetmasters,
> we found that the puppetmasters could no longer handle the load and
> would take a long time compiling manifests, sometimes so long that
> they would timeout.  So in our configuration, the puppetmasters use
> asynchronous storeconfigs which has the puppetmasters sending the
> server information to ActiveMQ using stomp.  The ActiveMQ server sits
> on a different physical machine from the puppetmasters and this
> machine also runs puppetqd which harvests the information from
> ActiveMQ and stores it into MySQL.  By doing this, we didn't have to
> scale up our number of puppetmasters but we did add two additional
> servers to our infrastructure to run the puppetqd and ActiveMQ.
> 
> ok. Those are pretty much the same reasons why I haven't deployed
> storeconfigs :)
> 
> Would you expect puppetqd to be a recommended dependency of
> puppetmasterd?

That sounds right.  

You should play with it now that it doesn't kill the puppetmasters.  We're using the information in the storeconfigs DB to feed into our "enterprise CMDB."  Quite handy.  Better than allowing our CMDB folks to setup probes and root scripts to gather the data on our machines.

-- 
Digant C Kasundra <digant at stanford.edu>
Technical Lead, ITS Unix Systems and Applications, Stanford University