Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
  • Loading branch information
SMSadegh19 committed Feb 4, 2022
2 parents c957eaa + c257817 commit 692a39c
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,17 @@
Python script for crawling ResearchGate.net papers

## About the script
This code start the crawling from urls in ```start.txt``` and give paper details in ```crawled.json```.
This code start crawling process by urls in ```start.txt``` and give paper details in ```crawled.json```.

## requirements
## Requirements
First install Python.
Then install these libraries:
```
pip install selenium
pip install webdriver-manager
```

## Parameters
```MAX_FETCH_COUNT```: How many pages you want to crawl?

```MAX_CACHED_NUM```: We renew ```crawled.json``` after crawling each ```MAX_CACHED_NUM``` papers.

0 comments on commit 692a39c

Please sign in to comment.