Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

程序运行__main___出错 #582

Open
NocTurn294 opened this issue May 10, 2024 · 4 comments
Open

程序运行__main___出错 #582

NocTurn294 opened this issue May 10, 2024 · 4 comments
Labels
failed 程序运行出错

Comments

@NocTurn294
Copy link

为了更好的解决问题,请认真回答下面的问题。等到问题解决,请及时关闭本issue。

  • 问:请您指明哪个版本运行出错(github版/PyPi版/全部)?

答:python3.12

  • 问:您使用的是否是最新的程序(是/否)?

答:是

  • 问:爬取任意用户都会运行出错吗(是/否)?

答:是

  • 问:若只有爬特定微博时才出错,能否提供出错微博的weibo_id或url(非必填)?

答:否

  • 问:若您已提供出错微博的weibo_id或url,可忽略此内容,否则能否提供出错账号的user_id及您配置的since_date,方便我们定位出错微博(非必填)?

答:

  • 问:如果方便,请您描述出错详情,最好附上错误提示。

答:运行___main____.py
或者运行终端命令都会出现这个问题,

Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\parser\index_parser.py", line 37, in get_user
    self.user.weibo_num = string_to_int(user_info[0][3:-1])
                                        ~~~~~~~~~^^^
IndexError: list index out of range
None
****************************************************************************************************
'NoneType' object has no attribute 'nickname'
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\spider.py", line 228, in _get_filepath
    dir_name = self.user.nickname
               ^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'nickname'
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\writer\csv_writer.py", line 25, in __init__
    with open(self.file_path, 'a', encoding='utf-8-sig',
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
****************************************************************************************************
Progress:   0%|          | 0/1 [00:02<?, ?it/s]
list index out of range
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\parser\page_parser.py", line 66, in get_one_page
    is_exist = info[0].xpath("div/span[@class='ctt']")
               ~~~~^^^
IndexError: list index out of range
cannot unpack non-iterable NoneType object
Traceback (most recent call last):
  File "D:\code\PythonCode\spider_2\weiboSpider\weibo_spider\spider.py", line 180, in get_weibo_info
    weibos, self.weibo_id_list, to_continue = PageParser(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot unpack non-iterable NoneType object```
@NocTurn294 NocTurn294 added the failed 程序运行出错 label May 10, 2024
@dataabc
Copy link
Owner

dataabc commented May 10, 2024

可能是爬取速度太快,被暂时限制了。一般情况下,一段时间后限制会自动解除。可通过降低爬取速度避免被限制。

@MarthaWWW
Copy link

出现了一样的问题

@QinXie123
Copy link

出现了一样的问题,降低爬取速度,隔了几天均无法进行爬取

@MarthaWWW
Copy link

更换在不同浏览器的cookie就行,edge的cookie不行就换chrome,亲测有用

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed 程序运行出错
Projects
None yet
Development

No branches or pull requests

4 participants