Exception when searching large datasets for common query

[thannaske]

Description

When searching for a string that is relatively common throughout a collection of indexed data, the results can not be fetched or displayed because the search with the Scout driver results in the following exception:

Typesense\\Exceptions\\ObjectUnprocessable(code: 0): Only upto 250 hits can be fetched per page

Steps to reproduce

1. Start a fresh Laravel project, install Scout and the Typesense Scout driver according to documentation
2. Add a model and make it searchable
3. Add a large dataset (~ 2000+ records with "lorem ipsum" text)
4. Query the dataset and try to display the results in a paginated way:

$items = Item::search($request->input('query'))
    ->options(['query_by' => 'content'])->latest()
    ->paginate(perPage: 10, pageName: 'items')
    ->appends($request->only('query'));

Expected Behavior

I expect the search to succeed, even when there are more then 250 hits, because that's why I use pagination on my frontend.

Actual Behavior

Following exception is thrown:

Typesense\\Exceptions\\ObjectUnprocessable(code: 0): Only upto 250 hits can be fetched per page

Metadata

Laravel Version: 10.43.0
PHP Version: 8.2.12
Scout Version: 10.8.2
Typesense PHP SDK Version: 4.9.1
Typesense Version: 0.25.2
OS: Docker @ Apple Silicon

[thannaske]

@jasonbosco @kishorenc It's my understanding that this repository has been adopted to the typesense organisation. This gave me the impression that this package is officially supported in some way, but looking at other issues and open MRs it does look kind of abandoned by its original author and maintainer. Are there any plans to keep this plugin alive now that it has been officially adapted by Laravel Scout?

[jasonbosco]

@thannaske The features from the driver in this repo have indeed been merged into Laravel Scout natively.

So we've temporarily paused development in this repo and plan to instead address any issues or improvements in the Laravel Scout driver instead.

Now if there are any Typesense-specific features that would be hard to implement in Laravel Scout natively (since we need to maintain consistency with all the other drivers), then at that point we plan to add those features into this driver and maintain it as a "Scout Extended Driver" of sorts. But it's too early to tell if we'd want to do this. So we're essentially in a holding pattern on this repo for now.

So I would recommend switching to the native Laravel Scout driver and then report any issues in the Laravel Scout repo and we can address them there.