Skip to content

A spider project with webmagic. Using dynamic proxy to download pages to conquer auti-spider, store the data from youku.com in mysql.

License

Notifications You must be signed in to change notification settings

kekewang/Spider

Repository files navigation

Spider

This is a project which can fetch the page of web site like youku.com, also will store the data in database.

The spider project depends on the webmagic, a web spider framework which is developed by Java. Please refer to http://webmagic.io/

Anything about this project, please let me know!

E-mail: wangke7127@gmail.com

About

A spider project with webmagic. Using dynamic proxy to download pages to conquer auti-spider, store the data from youku.com in mysql.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published