- 34
- 0
2018-09-19 11:58:25 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.zhihu.com/question/265749263> (referer: https://www.zhihu.com/question/265749263/answer/298529974)
2018-09-19 11:58:25 [scrapy.extensions.logstats] INFO: Crawled 20 pages (at 5 pages/min), scraped 10 items (at 5 items/min)
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/24252236>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/24252236>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/264756735>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/264756735>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/264756735>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/264756735>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/23846254>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/23846254>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/23846254>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/60410308>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/264756735>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/23846254>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/23846254>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/23846254>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/265749263>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/265749263>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/265749263>
None
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/265749263>
None
2018-09-19 11:58:25 [scrapy.core.engine] INFO: Closing spider (finished)
2018-09-19 11:58:25 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 7823,
'downloader/request_count': 20,
'downloader/request_method_count/GET': 20,
'downloader/response_bytes': 7113,
'downloader/response_count': 20,
'downloader/response_status_count/400': 20,
'dupefilter/filtered': 9,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 9, 19, 3, 58, 25, 849646),
'item_scraped_count': 28,
'log_count/DEBUG': 202,
'log_count/INFO': 10,
'request_depth_max': 3,
'response_received_count': 20,
'scheduler/dequeued': 20,
'scheduler/dequeued/memory': 20,
'scheduler/enqueued': 20,
'scheduler/enqueued/memory': 20,
'start_time': datetime.datetime(2018, 9, 19, 3, 55, 14, 766276)}
2018-09-19 11:58:25 [scrapy.core.engine] INFO: Spider closed (finished)### 问题描述
爬虫在运行一段时间后自动关闭,爬取了一部分的数据,但是start_urls(start_url
s中有多条链接,)中的链接并未爬取完的情况下爬虫就自动关闭了
请问
2018-09-19 11:58:25 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.zhihu.com/question/265749263> None
这个反馈是在表达什么,为何会自动关闭
0
打赏
收藏
点击回答
网站公告
- 扫一扫访问手机版
回答动态

- 神奇的四哥:发布了悬赏问题阿里云幻兽帕鲁服务器更新之后。服务器里面有部分玩家要重新创建角色是怎么回事啊?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题函数计算不同地域的是不能用内网吧?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题ARMS可以创建多个应用嘛?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题在ARMS如何申请加入公测呀?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题前端小程序接入这个arms具体是如何接入监控的,这个init方法在哪里进行添加?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题阿里云幻兽帕鲁服务器刚到期,是不是就不能再导出存档了呢?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题阿里云幻兽帕鲁服务器的游戏版本不兼容 尝试更新怎么解决?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题阿里云幻兽帕鲁服务器服务器升级以后 就链接不上了,怎么办?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题阿里云幻兽帕鲁服务器转移以后服务器进不去了,怎么解决?预计能赚取 0积分收益

- 神奇的四哥:发布了悬赏问题阿里云幻兽帕鲁服务器修改参数后游戏进入不了,是什么情况?预计能赚取 0积分收益
- 回到顶部
- 回到顶部

