[Pkg-crosswire-devel] ICU and stability issues

[Matthew Talbert]

On Mon, Jan 26, 2009 at 7:24 PM, DM Smith <dmsmith555 at yahoo.com> wrote:
>
> On Jan 26, 2009, at 5:58 PM, Matthew Talbert wrote:
>
>>> Matthew Talbert wrote:
>>>
>>>> However, GnomeSword and BibleTime both provide their own string casing
>>>> function for this purpose, both using standard unicode definitions to
>>>> provide casing. So, as I've said, even if SWORD is built with ICU, it
>>>> will provide no benefit to the end user, and may lead to stability
>>>> issues.
>>>
>>> I think it would be a real help in making this choice appropriately if we
>>> could get someone (you?) to create a replicable test case, where a libsword
>>> 1.5.11-based setup compiled with ICU reliably causes stability issues, but
>>> one without ICU does not.  Ideally the app used in the test would be a
>>> non-interactive command line tool, so we're not dependent on big GUI apps
>>> and users clicking on a bunch of things in order to test.
>>>
>>> Likewise, I'd be happy to see someone (else) create a test case where a
>>> search fails when using the 1.5.11 library with ICU disabled, but works with
>>> it enabled.  Maybe this is as simple as
>>>
>>> sudo installmgr -ri X Y
>>> diatheke <some options here>
>>>
>>> (But right now, I don't have a known set of X, Y and <some options here>
>>> that I know will act as such a test case!).
>>>
>>> Once we have repeatable test cases, we can then either fix the stability
>>> issue with a patch at build time, or we can make an informed decision decide
>>> to build without ICU.  If there is an existing (replicable) bug report that
>>> has good info on this, do please point me at it.
>>>
>>> If libsword packagers are going to get squeezed between frontend
>>> developers who say ICU causes stability issues and should be disabled, and
>>> others in the sword-devel community who say it is really does cause loss of
>>> functionality to disable it, so it should be enabled... then I'd like to see
>>> some clear test evidence, from both sets of people, to help  us packagers
>>> make a sane choice!  Is that realistic?
>>>
>>
>> I have asked for those saying GnomeSword will not search correctly
>> without ICU to submit just such a test case, but have not seen any.
>>
>> There is a thread here,
>> http://www.crosswire.org/pipermail/sword-devel/2009-January/030641.html,
>> that while not an official bug report, does indicate the problems, and
>> upstream's action of immediately disabling many of the filters
>> constitutes to me an admission that they do cause problems.
>
> Matthew,
> I saw the checkin disabling many of the transliteration filters. I think it
> was stated here that none of the Linux front-ends expose the transliteration
> filters. None of the modules mention them. So I can't see how they would
> cause a problem. I didn't see any other changes to the SWORD engine
> regarding ICU or any other admission to ICU causing a problem.

You may well be right. I am still learning about this. I was under the
impression that the transliteration filters were used somehow even
without the front-ends exposing them.

> The only filters that use ICU are utf8arshaping, utf8bidireorder, and
> uft8nfc, utf8nfkd. The first two are used to display right-to-left texts.
> The other two are used by module builders, but I couldn't find anywhere else
> that they were used by the engine.
>
> I found that diatheke uses ICU extensively.
>
> Maybe I don't understand SWORD well enough, but I can't see where including
> ICU and not using transliterators causes a problem.
>
> I'd suggest, that sword-tools are compiled with ICU and, perhaps, that
> libsword7 is not.

I think this is a good idea. However, wouldn't this require that the
tools be statically linked? Or do they not depend on libsword at all?

> While uconv (not iconv) can eliminate the need for ICU in osis2mod and
> tei2mod, I added it precisely because people were creating modules that were
> not NFC. I think it is a necessary convenience.
>
> In Him,
>        DM
>
>