Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How is it possible to run multiple threads, with diff proxys #2860

Closed
Akshay-Shingala opened this issue Jun 18, 2024 · 1 comment
Closed
Labels
duplicate The answer/solution already exists somewhere question Someone is looking for answers UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode

Comments

@Akshay-Shingala
Copy link

def access_site(cmd):
os.system(cmd)

urls = [
'https://example.com',
'https://example.com',
['https://example.com'
]

processes = []
for url in urls:
cmd = f'pytest w.py --proxy=USER_NAME:PASSWORD@pr.oxylabs.io:7777 --gui --uc --no-summary -q'
process = thread.Thread(target=run_test, args=(cmd, ))
processes.append(process)
process.start()
for i in processes:
i.join()

this is my code

@mdmintz mdmintz added duplicate The answer/solution already exists somewhere question Someone is looking for answers UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode labels Jun 18, 2024
@mdmintz
Copy link
Member

mdmintz commented Jun 18, 2024

Duplicate of #2835 / #2828 (comment)

See the example that uses ThreadPoolExecutor with UC Mode:

import sys
from concurrent.futures import ThreadPoolExecutor
from seleniumbase import Driver
sys.argv.append("-n")  # Tell SeleniumBase to do thread-locking as needed

def launch_driver(url):
    driver = Driver(uc=True)
    try:
        driver.get(url=url)
        driver.sleep(2)
    finally:
        driver.quit()

urls = ['https://seleniumbase.io/demo_page' for i in range(3)]
with ThreadPoolExecutor(max_workers=len(urls)) as executor:
    for url in urls:
        executor.submit(launch_driver, url)

Or use pytest multithreading with pytest-xdist and the parameterized library: #2394 (comment)

from parameterized import parameterized
from seleniumbase import BaseCase
BaseCase.main(__name__, __file__, "-n3")

class ProxyTests(BaseCase):
    @parameterized.expand(
        [
            ["host1:port1"],
            ["host2:port2"],
            ["host3:port3"],
        ]
    )
    def test_multiple_proxies(self, proxy_string):
        self.get_new_driver(
            undetectable=True, proxy=proxy_string, multi_proxy=True
        )
        self.driver.get("https://browserleaks.com/webrtc")
        self.sleep(30)

multi_webrtc_checks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate The answer/solution already exists somewhere question Someone is looking for answers UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode
Projects
None yet
Development

No branches or pull requests

2 participants