Diacritic-Insensitive Search Support (Czech characters)

[janroz]

Hello,

I'm using TNTSearch for a project and encountered an issue with diacritic-insensitive searching. Specifically, the library seems unable to find matches when the search term differs in diacritics (e.g., searching for "OSVC" does not match "OSVČ").

I was wondering if there is a known workaround for this, or if there is a potential for a future update that might address this issue? This functionality is very important for applications dealing with languages using diacritics.

Adding this to loadConfig didn't work:

'charset' => 'utf8'

Thank you for your assistance.

[janroz]

Nobody?

[janroz]

For now I solved it by using EdgeNgramTokenizer, after some tweeks it works well:

'tokenizer' => \TeamTNT\TNTSearch\Support\EdgeNgramTokenizer::class,

[somegooser]

What did you tweek with edgengram?

1. Did you try utf8mb4 as charset?

2. What you also could try is "transliterate". Translate all special characters to default english. Examples:

`
['À' => 'A', 'Á' => 'A', 'Â' => 'A', 'Ã' => 'A', 'È' => 'E', 'É' => 'E']

`