[Pkg-openldap-devel] Bug#465915: segmentation fault in slapd under some load
Arthur de Jong
adejong at debian.org
Fri Feb 15 15:01:22 UTC 2008
Subject: segmentation fault in slapd under some load
Package: slapd
Version: 2.4.7-5
Severity: important
I have a test directory with 2032 DNs in it to test nss-ldapd. When I
run my test scripts (which result in a large number of LDAP requests)
the server occasionally crashes. Lately this seems to happen more often
so much so that it's become reproducible.
I have installed slapd-dbg and have reproduces the crash below. If you
need more information (e.g. contents of /var/lib/ldap) or the test
scripts just let me know. (the test scripts are from nss-ldapd and
basically do just searches with paging)
The crash is mostly this (full logs attached)
# db4.2_recover -h /var/lib/ldap
# gdb /usr/sbin/slapd
[...]
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) r -d 273 -h ldap:/// ldaps:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf
Starting program: /usr/sbin/slapd -d 273 -h ldap:/// ldaps:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf
[Thread debugging using libthread_db enabled]
[...]
=> send_search_entry: conn 8 dn="cn=Nona Delmore+uid=ndelmore,ou=lotsofpeople,dc=test,dc=tld"
<= send_search_entry: conn 7 exit.
entry_decode: "uid=mherlihy,ou=lotsofpeople,dc=test,dc=tld"
<= entry_decode(uid=mherlihy,ou=lotsofpeople,dc=test,dc=tld)
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb521ab90 (LWP 28384)]
slap_send_search_entry (op=0x84263a0, rs=0xb521a168)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/result.c:900
900 /build/buildd/openldap2.3-2.4.7/servers/slapd/result.c: No such file or directory.
in /build/buildd/openldap2.3-2.4.7/servers/slapd/result.c
(gdb) bt
#0 slap_send_search_entry (op=0x84263a0, rs=0xb521a168)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/result.c:900
#1 0xb78314ab in bdb_search (op=0x84263a0, rs=0xb521a168)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/back-bdb/search.c:869
#2 0x08077d13 in fe_op_search (op=0x84263a0, rs=0xb521a168)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/search.c:368
#3 0x0807853c in do_search (op=0x84263a0, rs=0xb521a168)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/search.c:217
#4 0x080757c6 in connection_operation (ctx=0xb521a248, arg_v=0x84263a0)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/connection.c:1083
#5 0x08075ed6 in connection_read_thread (ctx=0xb521a248, argv=0xf)
at /build/buildd/openldap2.3-2.4.7/servers/slapd/connection.c:1210
#6 0xb7f56a44 in ?? () from /usr/lib/libldap_r-2.4.so.2
#7 0xb521a248 in ?? ()
#8 0x0000000f in ?? ()
#9 0x00000000 in ?? ()
(gdb)
The crash does not occur on the same DN every time and sometimes an
assertion is triggered instead of a Segmentation fault (though I don't
think I've seen that with slapd-dbg installed). I've checked permissions
on /var/lib/ldap. Running slapindex seems to fix the situation sometimes
(for a while) and it seems that if you let slapd settle down a bit the
problem does not occur immediately.
If I increase the debug much level further (haven't tested all
combinations) the problem does not seem to appear. Also, I have only
been able to reproduce the crash under valgrind with -d 1:
# db4.2_recover -h /var/lib/ldap
# valgrind --leak-check=full /usr/sbin/slapd -d 1 -h ldap:/// ldaps:/// ldapi:/// -g openldap -u openldap -f /etc/ldap/slapd.conf
==12385== Thread 6:
==12385== Invalid read of size 4
==12385== at 0x42A9160: pthread_mutex_lock (pthread_mutex_lock.c:51)
[...]
==12385== Address 0x2c is not stack'd, malloc'd or (recently) free'd
==12385==
==12385== Process terminating with default action of signal 11 (SIGSEGV)
==12385== Access not within mapped region at address 0x2C
==12385== at 0x42A9160: pthread_mutex_lock (pthread_mutex_lock.c:51)
[...]
==12385== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 104 from 1)
==12385== malloc/free: in use at exit: 78,857,822 bytes in 21,789 blocks.
==12385== malloc/free: 41,385 allocs, 19,596 frees, 88,025,431 bytes allocated.
[...]
==12385== LEAK SUMMARY:
==12385== definitely lost: 139 bytes in 4 blocks.
==12385== indirectly lost: 336 bytes in 28 blocks.
==12385== possibly lost: 1,296 bytes in 9 blocks.
==12385== still reachable: 78,856,051 bytes in 21,748 blocks.
==12385== suppressed: 0 bytes in 0 blocks.
==12385== Reachable blocks (those to which a pointer was found) are not shown.
==12385== To see them, rerun with: --leak-check=full --show-reachable=yes
Killed
Full logs of all tests as well as slapd.conf are attached.
-- System Information:
Debian Release: lenny/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.6.24-1-686 (SMP w/2 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/bash
Versions of packages slapd depends on:
ii adduser 3.105 add and remove users and groups
ii coreutils 6.10-3 The GNU core utilities
ii debconf [debconf-2.0] 1.5.19 Debian configuration management sy
ii libc6 2.7-8 GNU C Library: Shared libraries
ii libdb4.2 4.2.52+dfsg-4 Berkeley v4.2 Database Libraries [
ii libgnutls26 2.2.1-3 the GNU TLS library - runtime libr
ii libldap-2.4-2 2.4.7-5 OpenLDAP libraries
ii libltdl3 1.5.26-1 A system independent dlopen wrappe
ii libperl5.8 5.8.8-12 Shared Perl library
ii libsasl2-2 2.1.22.dfsg1-17+b1 Cyrus SASL - authentication abstra
ii libslp1 1.2.1-7.1 OpenSLP libraries
ii libwrap0 7.6.dbs-14 Wietse Venema's TCP wrappers libra
ii perl [libmime-base64- 5.8.8-12 Larry Wall's Practical Extraction
ii psmisc 22.6-1 Utilities that use the proc filesy
ii unixodbc 2.2.11-16 ODBC tools libraries
Versions of packages slapd recommends:
ii libsasl2-modules 2.1.22.dfsg1-17+b1 Cyrus SASL - pluggable authenticat
-- debconf information:
slapd/internal/adminpw: (password omitted)
* slapd/password1: (password omitted)
* slapd/password2: (password omitted)
* slapd/allow_ldap_v2: false
slapd/password_mismatch:
slapd/tlsciphersuite:
slapd/suffix_change: false
slapd/fix_directory: true
slapd/invalid_config: true
* shared/organization: Test domain for nss-ldapd
slapd/slave_databases_require_updateref:
slapd/dump_database_destdir: /var/backups/slapd-VERSION
slapd/upgrade_slapcat_failure:
slapd/slurpd_obsolete:
slapd/autoconf_modules: true
* slapd/purge_database: false
* slapd/domain: test.tld
* slapd/backend: BDB
* slapd/no_configuration: false
slapd/migrate_ldbm_to_bdb: false
* slapd/move_old_database: true
slapd/dump_database: when needed
slapd/upgrade_slapadd_failure:
--
-- arthur - adejong at debian.org - http://people.debian.org/~adejong --
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gdb-slapd-d256.txt.gz
Type: application/x-gzip
Size: 2964 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20080215/a2b0b4b9/attachment-0005.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gdb-slapd-d273.txt.gz
Type: application/x-gzip
Size: 267713 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20080215/a2b0b4b9/attachment-0006.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: valgrind-slapd-d1.txt.gz
Type: application/x-gzip
Size: 190559 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20080215/a2b0b4b9/attachment-0007.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: valgrind-slapd-d256.txt.gz
Type: application/x-gzip
Size: 3391 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20080215/a2b0b4b9/attachment-0008.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: valgrind-slapd-d273.txt.gz
Type: application/x-gzip
Size: 379264 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20080215/a2b0b4b9/attachment-0009.bin
-------------- next part --------------
# This is the main slapd configuration file. See slapd.conf(5) for more
# info on the configuration options.
#######################################################################
# Global Directives:
# Features to permit
#allow bind_v2
# Schema and objectClass definitions
include /etc/ldap/schema/core.schema
include /etc/ldap/schema/cosine.schema
include /etc/ldap/schema/nis.schema
include /etc/ldap/schema/inetorgperson.schema
include /etc/ldap/schema/samba.schema-3.0
include /etc/ldap/schema/trustMode.schema
include /etc/ldap/schema/misc.schema
# Schema check allows for forcing entries to
# match schemas for their objectClasses's
#schemacheck on
# Where the pid file is put. The init.d script
# will not stop the server if you change this.
pidfile /var/run/slapd/slapd.pid
# List of arguments that were passed to the server
argsfile /var/run/slapd/slapd.args
# Read slapd.conf(5) for possible values
loglevel 0
# Where the dynamically loaded modules are stored
modulepath /usr/lib/ldap
moduleload back_bdb
# The maximum number of entries that is returned for a search operation
sizelimit 500
sizelimit size.prtotal=unlimited
# The tool-threads parameter sets the actual amount of cpu's that is used
# for indexing.
tool-threads 1
#######################################################################
# Specific Backend Directives for bdb:
# Backend specific directives apply to this backend until another
# 'backend' directive occurs
backend bdb
#######################################################################
# Specific Backend Directives for 'other':
# Backend specific directives apply to this backend until another
# 'backend' directive occurs
#backend <other>
#######################################################################
# Specific Directives for database #1, of type bdb:
# Database specific directives apply to this databasse until another
# 'database' directive occurs
database bdb
# The base of your directory in database #1
suffix "dc=test,dc=tld"
checkpoint 512 30
# rootdn directive for specifying a superuser on the database. This is needed
# for syncrepl.
#rootdn "cn=admin,dc=test,dc=tld"
#rootpw ""
# Where the database file are physically stored for database #1
directory "/var/lib/ldap"
# For the Debian package we use 2MB as default but be sure to update this
# value if you have plenty of RAM
dbconfig set_cachesize 0 2097152 0
# Sven Hartge reported that he had to set this value incredibly high
# to get slapd running at all. See http://bugs.debian.org/303057
# for more information.
# Number of objects that can be locked at the same time.
dbconfig set_lk_max_objects 1500
# Number of locks (both requested and granted)
dbconfig set_lk_max_locks 1500
# Number of lockers
dbconfig set_lk_max_lockers 1500
# Indexing options for database #1
index objectClass eq
# Save the time that the entry gets modified, for database #1
lastmod on
# Where to store the replica logs for database #1
# replogfile /var/lib/ldap/replog
# The userPassword by default can be changed
# by the entry owning it if they are authenticated.
# Others should not be able to see it, except the
# admin entry below
# These access lines apply to database #1 only
access to attrs=userPassword,shadowLastChange
by dn="cn=admin,dc=test,dc=tld" write
by anonymous auth
by self write
by * none
# Ensure read access to the base for things like
# supportedSASLMechanisms. Without this you may
# have problems with SASL not knowing what
# mechanisms are available and the like.
# Note that this is covered by the 'access to *'
# ACL below too but if you change that as people
# are wont to do you'll still need this if you
# want SASL (and possible other things) to work
# happily.
access to dn.base="" by * read
# The admin dn has full write access, everyone else
# can read everything.
access to *
by dn="cn=admin,dc=test,dc=tld" write
by * read
# For Netscape Roaming support, each user gets a roaming
# profile for which they have write access to
#access to dn=".*,ou=Roaming,o=morsnet"
# by dn="cn=admin,dc=test,dc=tld" write
# by dnattr=owner write
#######################################################################
# Specific Directives for database #2, of type 'other' (can be bdb too):
# Database specific directives apply to this databasse until another
# 'database' directive occurs
#database <other>
# The base of your directory for database #2
#suffix "dc=debian,dc=org"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20080215/a2b0b4b9/attachment-0001.pgp
More information about the Pkg-openldap-devel
mailing list