Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Page fetching time increasing exponentially #17

Open
Mideen opened this issue Jul 25, 2018 · 0 comments
Open

Page fetching time increasing exponentially #17

Mideen opened this issue Jul 25, 2018 · 0 comments

Comments

@Mideen
Copy link

Mideen commented Jul 25, 2018

When I tried with Aquarium, I hit Aquarium (It is running on my localhost:8050) with 100 parallel threads.
It takes the time exponentially from 4 sec to 40 sec. My machine has 4 CPU(s) core. I tried with one Splash instance and 5 Splash instance. There is no much time difference between them.

My docker-compose.yml is

version: '2'

services:
    haproxy:
        image: haproxy:1.7
        ports:
            # stats
            - "8036:8036"

            # splash
            - "8050:8050"
        links:
            - splash0
            - splash1
            - splash2
            - splash3
            - splash4
        volumes:
            - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro

    splash0:
        image: scrapinghub/splash:3.2
        command: --max-timeout 3600 --slots 50 --maxrss 1200 --verbosity 1
        expose:
            - 8050
        mem_limit: 1680m
        memswap_limit: 2160m
        restart: always

    splash1:
        image: scrapinghub/splash:3.2
        command: --max-timeout 3600 --slots 50 --maxrss 1200 --verbosity 1
        expose:
            - 8050
        mem_limit: 1680m
        memswap_limit: 2160m
        restart: always

    splash2:
        image: scrapinghub/splash:3.2
        command: --max-timeout 3600 --slots 50 --maxrss 1200 --verbosity 1
        expose:
            - 8050
        mem_limit: 1680m
        memswap_limit: 2160m
        restart: always

    splash3:
        image: scrapinghub/splash:3.2
        command: --max-timeout 3600 --slots 50 --maxrss 1200 --verbosity 1
        expose:
            - 8050
        mem_limit: 1680m
        memswap_limit: 2160m
        restart: always

    splash4:
        image: scrapinghub/splash:3.2
        command: --max-timeout 3600 --slots 50 --maxrss 1200 --verbosity 1
        expose:
            - 8050
        mem_limit: 1680m
        memswap_limit: 2160m
        restart: always

Is there any way to speed up the Splash on parellel hitting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant