Skip to content
#

scrapy-redis

Here is 1 public repository matching this topic...

项目整体分为scrapy-redis分布式爬虫爬取数据、基于ElasticSearch数据检索和前端界面展示三大模块。做此项目是为了熟悉scrapy-redis的基本流程,以及其背后的原理,同时熟悉ElasticSearch的使用。本项目可以作为一个基于ES存储的简单但是相对全面的全栈开发的Demo。项目中所采用的组件均在win10本地环境搭建(伪分布),旨在演示项目流程。你可以参考该项目,并将其扩展到多个主机上,实现分布式ES以及分布式Scrapy。

  • Updated Dec 8, 2022
  • CSS

Improve this page

Add a description, image, and links to the scrapy-redis topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the scrapy-redis topic, visit your repo's landing page and select "manage topics."

Learn more