Supposed fix to #317518: New version of libsql-statement-perl is very, very much slower

Paul Beardsell paul at beardsell.com
Tue Oct 20 21:32:15 UTC 2009


Please refer http://rt.cpan.org/Public/Bug/Display.html?id=16579

I've been looking at the source code of various versions of SQL::Statement.

Ty suggested a number of patches to address the performance problems I
reported.  He supplied a patch file.

My bug report "#317518: New version of libsql-statement-perl is very, very
much slower" has been closed and the reasoning goes as follows.  The last
Debian version was 1.5, the next version in Debian is 2.*, and the patch is
reported as being applied in version 1.16.2.  I can't find the 1.16.2 code
but I have found the 1.18 code.  Much of the patch is *not* to be found in
1.18

The differences from 1.* to 2.* code are too great for me to get my head
around in a short time, but I suspect if the patch is NOT to be found in
1.18, and it was reported as beinbg done in 1.16, that it is also not to be
found in 2.*

Paul Beardsell
Paul at Beardsell.com


2009/10/20 Jens Rehsack <rehsack at googlemail.com>

> 2009/10/20 Jeff Zucker <jeff at vpservices.com>:
> > Greg,
> >
> > I am no longer the primary maintainer of SQL::Statement, I am cc'ing Jens
> > Rehsack, the new primary maintainer.
>
> Maybe giving me the maintainership instead adding me as Co-Maintainer
> (and adding you back as Co-Maintainer) would stop that confusion?
> I don't think - I think the people just taking the first name they
> find and neither
> look into the release history nor in the ChangeLog.
>
> > In comparing the pre 1.-0 release to current releases you are comparing
> > apples to oranges since the pre-1.0 release supported almost no actual
> SQL
> > while the post releases have gotten successively richer in the SQL they
> > support.  My $0.02 is that if speed, rather than compatibility with other
> > SQL implementations is a concern, one is best off using Text::CSV_XS
> > directly or importing the data into SQLite or another db for processing.
> > OTOH, it's possible Jens has or will be introducing speed improvements I
> > know nothing about.
>
> Jens plans to re-write the entire module when he find enough time or enough
> sponsors. As long as neither is present, Jens still fixes the current
> issues.
> I don't plan to invest much time and effort into the current
> implementation,
> because currently there're to many side-effects and duplicated code in the
> module.
>
> > One other consideration - all of the drivers (DBD::CSV, etc.) that use
> > SQL::Statement are also able to use DBI::SQL::Nano (comes with the DBI
> > distro) which has about the same feature set as the pre 1-0 release and
> > which can be much faster than the full SQL::Statement for some tasks.
>
> Nope! AFAIK (and what I've seen from Merijn) DBD::CSV requires
> SQL::Statement.
>
> Best regards,
> Jens
>
> > --
> > Jeff
> >
> > rehsack at googlemail.com
> >
> > On Tue, Oct 20, 2009 at 2:35 AM, Paul Beardsell <paul at beardsell.com>
> wrote:
> >>
> >> Jeff,
> >>
> >> (cc Greg, the Debian maintainer of libsql-statement-perl)
> >>
> >> I understand you are perhaps the developer/maintainer or primary contact
> >> re the Perl package SQL::Statement.
> >>
> >> I am an enthusuastic user of the pre 1.0 release of SQL::Statement as
> >> found in Debian "Woody" Linux.  1.0 and later releases of SQL::Statement
> in
> >> more current versions of Debian Linux have been found by me to be much,
> much
> >> slower than the pre-1.0 releases on text CSV databases of only 2000
> rows.  I
> >> reported this bug to Debian and they now say it is "fixed" but no
> evidence
> >> as to what has been done is presented other than to say they have
> accepted a
> >> new upstream release, 1.22.
> >>
> >> I understand the reason for the performance fall was the re-write of
> >> C-code into pure Perl.  Fine, but if that is the reason I am sure that
> you
> >> have not re-introduced the old C code!  I also note a number of
> performance
> >> improvements were reported between 1.11 and 1.14 but *my* benchmarking
> >> showed a 20% reduction in speed between from 1.11 to 1.14.
> >>
> >> (1) Are you aware of any changes made to SQL::Statement between 1.14 and
> >> 1.22 which may have restored the excellent performance of the pre-1.0
> >> releases of SQL::Statement?
> >>
> >> (2) It also strikes me that possibly the problem is not in
> SQL::Statement
> >> but in DBD::CSV.  What do you think?
> >>
> >> Regards (and thanks for an [otherwise!] excellent package),
> >> Paul
> >>
> >> Paul Beardsell
> >> Paul at Beardsell.com
> >>
> >>
> >> 2009/10/20 Paul Beardsell <paul at beardsell.com>
> >>>
> >>> Hi, I have been notified that the bug I reported "#317518: New version
> of
> >>> libsql-statement-perl is very, very much slower" is reported as fixed
> and
> >>> has been closed by you.  The notification says that details of the
> changes
> >>> are to be found "below" but whereas I see lots of admin detail I
> nothing of
> >>> what has been done to supposedly restore the previous performance of
> this
> >>> package.  What has been done?  How do I find this out?
> >>>
> >>> Thanks,
> >>> Paul
> >>>
> >>> Paul Beardsell
> >>> Paul at Beardsell.com
> >>>
> >>>
> >>> 2009/10/19 Debian Bug Tracking System <owner at bugs.debian.org>
> >>>>
> >>>> This is an automatic notification regarding your Bug report
> >>>> which was filed against the libsql-statement-perl package:
> >>>>
> >>>> #317518: New version of libsql-statement-perl is very, very much
> slower
> >>>>
> >>>> It has been closed by gregor herrmann <gregoa at debian.org>.
> >>>>
> >>>> Their explanation is attached below along with your original report.
> >>>> If this explanation is unsatisfactory and you have not received a
> >>>> better one in a separate message then please contact gregor herrmann
> >>>> <gregoa at debian.org> by
> >>>> replying to this email.
> >>>>
> >>>>
> >>>> --
> >>>> 317518: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=317518
> >>>> Debian Bug Tracking System
> >>>> Contact owner at bugs.debian.org with problems
> >>>>
> >>>>
> >>>> ---------- Forwarded message ----------
> >>>> From: gregor herrmann <gregoa at debian.org>
> >>>> To: 317518-close at bugs.debian.org
> >>>> Date: Mon, 19 Oct 2009 17:03:41 +0000
> >>>> Subject: Bug#317518: fixed in libsql-statement-perl 1.22-1
> >>>> Source: libsql-statement-perl
> >>>> Source-Version: 1.22-1
> >>>>
> >>>> We believe that the bug you reported is fixed in the latest version of
> >>>> libsql-statement-perl, which is due to be installed in the Debian FTP
> >>>> archive:
> >>>>
> >>>> libsql-statement-perl_1.22-1.diff.gz
> >>>>  to
> >>>>
> pool/main/libs/libsql-statement-perl/libsql-statement-perl_1.22-1.diff.gz
> >>>> libsql-statement-perl_1.22-1.dsc
> >>>>  to
> >>>> pool/main/libs/libsql-statement-perl/libsql-statement-perl_1.22-1.dsc
> >>>> libsql-statement-perl_1.22-1_all.deb
> >>>>  to
> >>>>
> pool/main/libs/libsql-statement-perl/libsql-statement-perl_1.22-1_all.deb
> >>>> libsql-statement-perl_1.22.orig.tar.gz
> >>>>  to
> >>>>
> pool/main/libs/libsql-statement-perl/libsql-statement-perl_1.22.orig.tar.gz
> >>>>
> >>>>
> >>>>
> >>>> A summary of the changes between this version and the previous one is
> >>>> attached.
> >>>>
> >>>> Thank you for reporting the bug, which will now be closed.  If you
> >>>> have further comments please address them to 317518 at bugs.debian.org,
> >>>> and the maintainer will reopen the bug report if appropriate.
> >>>>
> >>>> Debian distribution maintenance software
> >>>> pp.
> >>>> gregor herrmann <gregoa at debian.org> (supplier of updated
> >>>> libsql-statement-perl package)
> >>>>
> >>>> (This message was generated automatically at their request; if you
> >>>> believe that there is a problem with it please contact the archive
> >>>> administrators by mailing ftpmaster at debian.org)
> >>>>
> >>>>
> >>>> -----BEGIN PGP SIGNED MESSAGE-----
> >>>> Hash: SHA1
> >>>>
> >>>> Format: 1.8
> >>>> Date: Mon, 19 Oct 2009 18:12:55 +0200
> >>>> Source: libsql-statement-perl
> >>>> Binary: libsql-statement-perl
> >>>> Architecture: source all
> >>>> Version: 1.22-1
> >>>> Distribution: unstable
> >>>> Urgency: low
> >>>> Maintainer: Debian Perl Group
> >>>> <pkg-perl-maintainers at lists.alioth.debian.org>
> >>>> Changed-By: gregor herrmann <gregoa at debian.org>
> >>>> Description:
> >>>>  libsql-statement-perl - Perl SQL parsing and processing engine
> >>>> Closes: 317518 531238
> >>>> Changes:
> >>>>  libsql-statement-perl (1.22-1) unstable; urgency=low
> >>>>  .
> >>>>   [ gregor herrmann ]
> >>>>   * debian/control: Changed: Switched Vcs-Browser field to ViewSVN
> >>>>     (source stanza).
> >>>>   * debian/control: Added: ${misc:Depends} to Depends: field.
> >>>>  .
> >>>>   [ Rene Mayorga ]
> >>>>   * New upstream release
> >>>>   * debian/control:
> >>>>     + wrap long lines
> >>>>     + add myself to uploaders
> >>>>     + add libdbd-csv-perl to B-D-I
> >>>>   * debian/rules; updated using dh-make-perl
> >>>>   * debhelper and debian/compat set to 7
> >>>>   * libsql-statement-perl.doc; keep README
> >>>>   * debian/copyright:
> >>>>     + use (new)format
> >>>>     + add copyright years to upstream
> >>>>     + add upstream's missing copyright statement
> >>>>     + split copyright stanzas for debian/*
> >>>>  .
> >>>>   [ Ryan Niebur ]
> >>>>   * New upstream release
> >>>>   * Add myself to uploaders
> >>>>   * Add libclone-perl and libparams-util-perl to B-D-I and Depends
> >>>>   * Update ryan52's email address
> >>>>  .
> >>>>   [ Nathan Handler ]
> >>>>   * debian/watch: Update to ignore development releases.
> >>>>  .
> >>>>   [ gregor herrmann ]
> >>>>   * Change my email address.
> >>>>   * New upstream release 1.22.
> >>>>     + update dependencies
> >>>>     + closes: #531238, #317518
> >>>>     + remove patch and quilt framework, RT#13080 (Debian bug #311788)
> is
> >>>>       resolved
> >>>>     + update and compact copyright
> >>>>   * Minimize debian/rules, adjust build deps.
> >>>>   * Set Standards-Version to 3.8.3; remove version from perl
> dependency.
> >>>>   * Rename debian/README.update.Debian to debian/README.Debian, add
> >>>>     debian/NEWS about possible problems, install upstream Todo.
> >>>> Checksums-Sha1:
> >>>>  8ea04c5a0f6c41a289302357ee075b1f6270c54a 1580
> >>>> libsql-statement-perl_1.22-1.dsc
> >>>>  fbab6660699617cc0c21102e08e6e88cbe02dae0 100115
> >>>> libsql-statement-perl_1.22.orig.tar.gz
> >>>>  cc274e2d425832bd5b9a2c9ad9cfb6741a361ccd 4785
> >>>> libsql-statement-perl_1.22-1.diff.gz
> >>>>  a31143c90b288c381e11ef84a3a22022274773f3 150546
> >>>> libsql-statement-perl_1.22-1_all.deb
> >>>> Checksums-Sha256:
> >>>>  3137632930c07f758f066f59261011bcb585e1dd337aad6d8f050d72cbf23d6b 1580
> >>>> libsql-statement-perl_1.22-1.dsc
> >>>>  56fa1b3875abaa474441857a4ebbd8fbcaa0112527bc4c764d37ca782bd9d467
> 100115
> >>>> libsql-statement-perl_1.22.orig.tar.gz
> >>>>  7da01f746f20b4fec71623a190cb30601fbb8cdc90d8c6674026e1d1d99a918d 4785
> >>>> libsql-statement-perl_1.22-1.diff.gz
> >>>>  3b47ce8dc7e66a1d6f51cf9e5571f663f59c952925b59d0fe1b763157f6668aa
> 150546
> >>>> libsql-statement-perl_1.22-1_all.deb
> >>>> Files:
> >>>>  40fe0dbc79fdd60c2861260ef228936b 1580 perl optional
> >>>> libsql-statement-perl_1.22-1.dsc
> >>>>  78c591dc36b6e011abd59717f6450870 100115 perl optional
> >>>> libsql-statement-perl_1.22.orig.tar.gz
> >>>>  5bbe6fca7e17704af4cfb8728348226b 4785 perl optional
> >>>> libsql-statement-perl_1.22-1.diff.gz
> >>>>  5ddc15298d77f8620710b60f0d08e3eb 150546 perl optional
> >>>> libsql-statement-perl_1.22-1_all.deb
> >>>>
> >>>> -----BEGIN PGP SIGNATURE-----
> >>>> Version: GnuPG v1.4.10 (GNU/Linux)
> >>>>
> >>>> iEYEARECAAYFAkrckHAACgkQOzKYnQDzz+SpTQCfSo3kzS3vV0B+A7DVhLlNKIaK
> >>>> kKEAn1M2KOJXT+RHhFNju2VIYBe/GlpH
> >>>> =0IYS
> >>>> -----END PGP SIGNATURE-----
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ---------- Forwarded message ----------
> >>>> From: Paul Beardsell <Paul at Beardsell.com>
> >>>> To: Debian Bug Tracking System <submit at bugs.debian.org>
> >>>> Date: Sat, 09 Jul 2005 12:00:27 +0100
> >>>> Subject: New version of libsql-statement-perl is very, very much
> slower
> >>>> Package: libsql-statement-perl
> >>>> Version: 1.11-1
> >>>> Severity: normal
> >>>>
> >>>>
> >>>> I depend upon libsql-statement-perl to manage small
> >>>> plain text databases of approx 2000 rows.  Performance
> >>>> was fine in the version of libsql-statement-perl
> >>>> which came with woody but the newer version in sarge
> >>>> is very, very much slower.  I understand this may
> >>>> be because because C code in the former version has
> >>>> been replaced with Perl code.  Fine, but this makes
> >>>> the package nearly unusable for me because of the
> >>>> performance problem.  I do not want to have to switch
> >>>> over to SQLite.
> >>>>
> >>>> -- System Information:
> >>>> Debian Release: 3.1
> >>>>  APT prefers testing
> >>>>  APT policy: (70, 'testing'), (60, 'unstable')
> >>>> Architecture: i386 (i686)
> >>>> Kernel: Linux 2.6.8-2-686
> >>>> Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
> >>>>
> >>>> Versions of packages libsql-statement-perl depends on:
> >>>> ii  perl                          5.8.4-8    Larry Wall's Practical
> >>>> Extraction
> >>>>
> >>>> -- no debconf information
> >>>>
> >>>>
> >>>
> >>
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-perl-maintainers/attachments/20091020/e453c31f/attachment-0001.htm>


More information about the pkg-perl-maintainers mailing list