Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JS Crawling via chrome headless #288

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open

JS Crawling via chrome headless #288

wants to merge 26 commits into from

Commits on Jan 29, 2019

  1. fixed

    Arnaud Pichon committed Jan 29, 2019
    Configuration menu
    Copy the full SHA
    38cb059 View commit details
    Browse the repository at this point in the history

  2. download chrome version

    Arnaud Pichon committed Jan 29, 2019
    Configuration menu
    Copy the full SHA
    0762c30 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2019

  1. Download chromium & chrome driver

    Arnaud Pichon committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    2f79b72 View commit details
    Browse the repository at this point in the history

  2. Working python 2.7 chrome installer

    Arnaud Pichon committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    f9179e8 View commit details
    Browse the repository at this point in the history

  3. remove useless changes

    Arnaud Pichon committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    6b3514b View commit details
    Browse the repository at this point in the history

  4. change environement variable name

    Arnaud Pichon committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    c7ce4e0 View commit details
    Browse the repository at this point in the history

  5. add versions link

    Arnaud Pichon committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    1a707b0 View commit details
    Browse the repository at this point in the history

  6. improve downloader script 

    create dir not only on Mac + executable + better verbosity
    @boogheta
    boogheta committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    383471c View commit details
    Browse the repository at this point in the history

  7. ensure chromedriver binary is executable

    @boogheta
    boogheta committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    95cff57 View commit details
    Browse the repository at this point in the history

  8. make this script an actual developers testcase

    @boogheta
    boogheta committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    7f3cb40 View commit details
    Browse the repository at this point in the history

  9. remove old phantom binaries

    @boogheta
    boogheta committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    a550912 View commit details
    Browse the repository at this point in the history

  10. do not restrict clicks to link

    @boogheta
    boogheta committed Jan 30, 2019
    Configuration menu
    Copy the full SHA
    1e5285d View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2019

  1. start preparing paths mess

    @boogheta
    boogheta committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    736b8c0 View commit details
    Browse the repository at this point in the history

  2. start migrating phantomJs to Chromium headless

    @boogheta
    boogheta committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    38a3170 View commit details
    Browse the repository at this point in the history

  3. move JS crawling to crawler directory (for docker compat)

    @boogheta
    boogheta committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    8f17e20 View commit details
    Browse the repository at this point in the history

  4. functional call of scrapy spider with headless chrome

    @boogheta
    boogheta committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    3b00778 View commit details
    Browse the repository at this point in the history

  5. lightfixes

    @boogheta
    boogheta committed Jan 31, 2019
    Configuration menu
    Copy the full SHA
    2f1daf5 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2019

  1. install chromium in docker

    @boogheta
    boogheta committed Feb 1, 2019
    Configuration menu
    Copy the full SHA
    be94807 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into grab-chrome

    @boogheta
    boogheta committed Feb 1, 2019
    Configuration menu
    Copy the full SHA
    c9203d0 View commit details
    Browse the repository at this point in the history

  3. fix chromium install in docker

    @boogheta
    boogheta committed Feb 1, 2019
    Configuration menu
    Copy the full SHA
    0a3c8a1 View commit details
    Browse the repository at this point in the history

  4. actually working chrome in docker

    @boogheta
    boogheta committed Feb 1, 2019
    Configuration menu
    Copy the full SHA
    b2dcee2 View commit details
    Browse the repository at this point in the history

  5. reactivate phantom options

    @boogheta
    boogheta committed Feb 1, 2019
    Configuration menu
    Copy the full SHA
    250e4c9 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2019

  1. Merge branch 'master' into grab-chrome

    ArnaudMolo committed Oct 21, 2019
    Configuration menu
    Copy the full SHA
    73dd15b View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2019

  1. Add fixed versions for chromium & chromium driver

    ArnaudMolo committed Oct 23, 2019
    Configuration menu
    Copy the full SHA
    7c49941 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2019

  1. 'Dsiable GPU' makes chromium crash

    ArnaudMolo committed Oct 24, 2019
    2 Configuration menu
    Copy the full SHA
    89eda93 View commit details
    Browse the repository at this point in the history

  2. rename all panthomjs

    ArnaudMolo committed Oct 24, 2019
    7 Configuration menu
    Copy the full SHA
    ae14897 View commit details
    Browse the repository at this point in the history