Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tried to stop a LoopingCall that was not running #305

Open
luanbe opened this issue Aug 13, 2021 · 4 comments
Open

Tried to stop a LoopingCall that was not running #305

luanbe opened this issue Aug 13, 2021 · 4 comments

Comments

@luanbe
Copy link

luanbe commented Aug 13, 2021

I also added two custom periodic monitors and imported monitor

from spidermon.contrib.scrapy.monitors import (
    ErrorCountMonitor,
    FinishReasonMonitor,
    ItemValidationMonitor,
    UnwantedHTTPCodesMonitor,
)

I didn't find the reason that got this error.

Traceback (most recent call last):
  File "/app/python/lib/python3.8/site-packages/scrapy/utils/defer.py", line 157, in maybeDeferred_coro
    result = f(*args, **kw)
  File "/usr/local/lib/python3.8/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File "/app/python/lib/python3.8/site-packages/spidermon/contrib/scrapy/extensions.py", line 131, in spider_closed
    task.stop()
  File "/app/python/lib/python3.8/site-packages/twisted/internet/task.py", line 197, in stop
    assert self.running, "Tried to stop a LoopingCall that was " "not running."
AssertionError: Tried to stop a LoopingCall that was not running.
@luanbe luanbe changed the title Error caught on signal handler: <bound method Spidermon.spider_closed of <spidermon.contrib.scrapy.extensions.Spidermon object at 0x7fb0e1d6f6a0>> Tried to stop a LoopingCall that was not running Aug 13, 2021
@mushtaqak
Copy link

Any update on this issue? I also faced the same issue today.

@mushtaqak
Copy link

I got this error when my spider had a few timeout requests and spider close action was triggered but failed with following reason and later above error

File "/app/python/lib/python3.9/site-packages/spidermon/runners.py", line 29, in run
    return self.run_suite()
  File "/app/python/lib/python3.9/site-packages/spidermon/[runners.py](http://runners.py/)", line 45, in run_suite
    [self.run](http://self.run/)_actions()
  File "/app/python/lib/python3.9/site-packages/spidermon/[runners.py](http://runners.py/)", line 73, in run_actions
    [self.run](http://self.run/)_monitors_failed()
  File "/app/python/lib/python3.9/site-packages/spidermon/[runners.py](http://runners.py/)", line 93, in run_monitors_failed
    [action.run](http://action.run/)(self.result, [self.data](http://self.data/))
TypeError: run() missing 1 required positional argument: 'data'

@VMRuiz
Copy link
Collaborator

VMRuiz commented Jun 1, 2023

Hello, could you please provide more details about this so we can replicate it?

What Spidermon and Scrapy version were you using? What Reactor was installed?

I see some weird URLS like:

Are you sure the monitor and/or actions were properly configured?

@mushtaqak
Copy link

mushtaqak commented Jun 5, 2023

Hi @VMRuiz

I actually added one "close spider" action dynamically if certain condition is fulfilled to the moniter suite that runs periodically
self.monitors_failed_actions.append(CloseSpiderAction)

We are using scrapy == 2.5.1 and spidermon == 1.17.0 versions.

and twisted reacter is installed:

from scrapy.utils.reactor import install_reactor
install_reactor("twisted.internet.asyncioreactor.AsyncioSelectorReactor")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants