Site Search for React Application

[Amik-TJ]

Is a crawling feature available for React applications? I was attempting to build an index and then crawl a react site, however it was unable to index the react project. Please help!

Thank you very much for creating this package.

[freekmurze]

By default, the underlying crawler doesn't execute JavaScript.

You can make the crawler execute JavaScript by creating your own search profile. In that profile you should call executeJavaScript on the $crawler that's passed to the configureCrawler method.

Let me know if this works for you.

[Amik-TJ]

I think there are some dependencies of puppeteer and Browsershot. Can you please elaborate this ! I am still facing problem after calling the executeJavaScript() on $crawler.

[Amik-TJ]

This two-line addition was made by myself in SiteSearch.php file. And I have also installed puppeteer.

   Symfony\Component\Process\Exception\ProcessFailedException 

  The command "node ^"C:^\xampp^\htdocs^\Laravel_Test^\laravel-sitemap-spatie^\vendor^\spatie^\browsershot^\src/../bin/browser.js^" ^"
^{^\^"url^\^":^\^"https:^\/^\/www.designersdofuturo.com^\/_files^\/ugd^\/ebb388_f988cb6176754f83b3efc51f7cda2e4d.pdf^\^",^\^"action^\^
":^\^"content^\^",^\^"options^\^":^{^\^"args^\^":^[^],^\^"viewport^\^":^{^\^"width^\^":800,^\^"height^\^":600^}^}^}^"" failed.

Exit Code: 1(General error)

Working directory: C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie

Output:
================


Error Output:
================
Error: net::ERR_ABORTED at https://www.designersdofuturo.com/_files/ugd/ebb388_f988cb6176754f83b3efc51f7cda2e4d.pdf
    at navigate (C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\node_modules\puppeteer\lib\cjs\puppeteer\common\FrameManager.js:1
55:23)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async FrameManager.navigateFrame (C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\node_modules\puppeteer\lib\cjs\puppeteer\
common\FrameManager.js:130:21)
    at async Frame.goto (C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\node_modules\puppeteer\lib\cjs\puppeteer\common\FrameMana
ger.js:500:16)
    at async Page.goto (C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\node_modules\puppeteer\lib\cjs\puppeteer\common\Page.js:11
67:16)
    at async callChrome (C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\vendor\spatie\browsershot\bin\browser.js:195:26)

  at C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\vendor\spatie\browsershot\src\Browsershot.php:824
    820▕         if ($exitCode === 2) {
    821▕             throw new ElementNotFound($this->additionalOptions['selector']);
    822▕         }
    823▕
  ➜ 824▕         throw new ProcessFailedException($process);
    825▕     }
    826▕
    827▕     protected function getFullCommand(array $command)
    828▕     {

  1   C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\vendor\spatie\browsershot\src\Browsershot.php:550
      Spatie\Browsershot\Browsershot::callBrowser(["https://www.designersdofuturo.com/_files/ugd/ebb388_f988cb6176754f83b3efc51f7cda2e
4d.pdf", "content"])

  2   C:\xampp\htdocs\Laravel_Test\laravel-sitemap-spatie\vendor\spatie\crawler\src\Handlers\CrawlRequestFulfilled.php:128
      Spatie\Browsershot\Browsershot::bodyHtml()

But getting some error