Fix Sphinx' returning of malformed or incorrectly encoded fragments / excerpts

[dvglc]

We currently have two workarounds, in sphinx:highlight() and sphinx:excerpts(), for dealing with malformed or incorrectly encoded fragments / excerpts returned by Sphinx:

svsal/modules/sphinx.xql

Line 705 in 7776af3

if (validation:jaxp(util:base64-decode($response//httpclient:body), false())) then

svsal/modules/sphinx.xql

Line 774 in 7776af3

if (validation:jaxp(util:base64-decode($response//httpclient:body), false())) then

Whenever a malformed / badly encoded text occurs, it is simply filtered out. This seems to successfully suppress ugly server errors, but we should find out why (and when) Sphinx produces erroneous fragments/excerpts in the first place.

Interestingly, if one tries to prevent such errors before sending excerpts to Sphinx, e.g. by handcrafted string replacings of suspicious characters, like so:

replace(replace($requestDoc, '%3D', '='), '%26amp%3B', '&'),

Sphinx' highlighting of qery terms in excerpts doesn't work in all cases.

(See also: 7776af3 and 03a62e3)

[dvglc]

Although the workaround is the same in both cases, perhaps the malformed fragments in sphinx:highlight are really something different than the erroneously encoded excerpts in sphinx:excerpts. But perhaps the underlying issue is the same.

[dvglc]

This might have been resolved through the latest changes with the EXPath Http Client, but needs further testing/observation.