Download file in headless chrome #5159

jonhkr · 2017-12-02T23:33:53Z

Meta -

OS: OSX
Selenium Version: 3.8.1
Browser: Chrome

Expected Behavior -

Permit files to be downloaded in headless mode.

Actual Behavior -

Files aren't downloaded.

Steps to reproduce -

Launch chrome in headless mode and try to download any file.

Looks like it is a feature to prevent sites from downloading files when running chrome in headless mode.
https://bugs.chromium.org/p/chromium/issues/detail?id=696481

To permit downloads it's necessary to send a command to chrome.
I'm using java and I couldn't find a way to make it work.

I even tried sending a post request to /session/:sessionId/chromium/send_command but that didn't work.

The text was updated successfully, but these errors were encountered:

p0deje · 2017-12-04T06:20:35Z

Apparently, this new command needs to be added to all the bindings.

jonhkr · 2017-12-04T15:57:27Z

I managed to get it working by issuing the above request directly to the chromedriver but using a development build of chromium.

jonhkr · 2017-12-04T16:00:19Z

This is how I did it:

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

The method send commands to the browser to avoid blocking of downloading in headless mode. (The issue is described here: SeleniumHQ#5159)

nstubi · 2018-01-03T13:02:22Z

Hello,

Any plans to support setDownloadBehavior in order to download files using chrome in headless mode?

Any doc/examples in .Net would be much appreciated.

Regards,
Nicolas

shreyanshp · 2018-01-11T12:26:27Z

Even I am facing this same issue

of downloading in headless mode. (The issue is described here: SeleniumHQ#5159)

daljitcheema · 2018-02-01T14:39:10Z

Has anyone got this working with Ruby/Rails? Also, can you get a development build of chrome on Heroku?

pulkitsharma07 · 2018-02-13T20:18:14Z

@p0deje, @jonhkr
I am working on integrating this with the Ruby bindings, but for some reason I am not able to download files in headless mode.

Could you please verify this spec, it is failing for me: pulkitsharma07@1142e0b#diff-b1ade8967288d0002579a914345a9f56R72

For me even https://github.com/SeleniumHQ/selenium/blob/master/javascript/node/selenium-webdriver/test/chrome/devtools_test.js#L51 is failing.

I am using Chrome 64 and chromedriver 2.35, not sure what could be causing this.

luke-hill · 2018-02-14T09:12:25Z

I've popped an optimisation in, also, whilst not having worked in this area of the specs. Is there a reason you verbosely create the driver, there are helpers such as create_driver! which will check to see if an instance exists and/or create one.

pulkitsharma07 · 2018-02-14T10:09:31Z

No particular reason, I missed that. The branch is not yet ready, Will do the changes to use create_driver!.

But, still, the main issue is that I am not able to download files in headless mode.

jbustillos · 2018-02-15T01:47:22Z

Is there an example for .NET?

p0deje · 2018-02-15T03:13:02Z

@pulkitsharma07 I cannot make it work too, maybe something has changed in recent Chrome/ChromeDriver?

pulkitsharma07 · 2018-02-15T15:09:16Z

So, in my headless tests Chrome is crashing on clicking the element to download the file, whereas it is not crashing with the same scenario in normal mode.

chromedriver logs:

  [1518706724.071][DEBUG]: DEVTOOLS COMMAND Input.dispatchMouseEvent (id=32) {
     "button": "left",
     "clickCount": 1,
     "modifiers": 0,
     "type": "mouseReleased",
     "x": 20,
     "y": 16
  }
  [1518706724.374][SEVERE]: Unable to receive message from renderer
  [1518706724.374][INFO]: Waiting for pending navigations...
  [1518706724.374][INFO]: Done waiting for pending navigations. Status: disconnected: not connected to DevTools
  [1518706724.374][DEBUG]: DevTools request: http://localhost:12345/json
  [1518706724.375][DEBUG]: DevTools request failed
  [1518706724.375][INFO]: RESPONSE ClickElement

pulkitsharma07 · 2018-02-15T19:31:18Z

@p0deje I have filed https://bugs.chromium.org/p/chromedriver/issues/detail?id=2270. Please take a look.

This adds commands to hit the send_command and send_command_and_get_result endpoints Addresses SeleniumHQ#5159

pulkitsharma07 · 2018-03-08T18:33:35Z

Upgrading to Chrome 65/chromedriver 2.36 fixed my crash issue with headless chrome.

HaveSpacesuit · 2019-03-18T19:53:34Z

Here is a .NET implementation to enable downloads using headless chrome:

ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.AddArgument("--headless");

ChromeDriver chromeDriver = new ChromeDriver(chromeOptions);

Dictionary<string, object> commandParameters = new Dictionary<string, object>
{
    ["behavior"] = "allow",
    ["downloadPath"] = @"C:\Downloads"
};

chromeDriver.ExecuteChromeCommand("Page.setDownloadBehavior", commandParameters);

rishi1992 · 2019-03-19T14:26:14Z

This is how I did it:

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

This fix is not working with my configuration. I'm still unable to download a file in headless mode.

rishi1992 · 2019-03-19T14:31:19Z

public static WebDriver getChromeDriver() throws IOException {
//Initialize ChromeDriver

	ChromeOptions options = new ChromeOptions();
	Map<String, Object> prefs = new HashMap<>();
	//String downloadFilepath = System.getProperty("user.dir");
	prefs.put("safebrowsing.enabled", false);
	prefs.put("download.default_directory", System.getProperty("user.dir"));
	prefs.put("download.prompt_for_download", false);
	prefs.put("download.directory_upgrade", true);
	prefs.put("cmd", "Page.setDownloadBehavior");
	prefs.put("profile.default_content_settings.popups", 0);
	prefs.put("behavior", "allow");
	options.setExperimentalOption("prefs", prefs);
	options.addArguments("--disable-notifications");
	options.addArguments("--start-maximized");
	options.addArguments("disable-infobars");
	options.addArguments("--disable-gpu");
	options.addArguments("--headless");
	options.addArguments("window-size=1980,1080");
	options.addArguments("--allow-running-insecure-content");
	options.addArguments("--disable-extensions");
	options.addArguments("--no-sandbox");
	options.addArguments("--test-type");
	options.addArguments("--disable-web-security");
	//options.setBinary("C:\\Program Files (x86)\\Google\\Chrome");
	String chromedriverpath_1 = System.getProperty("user.dir");
	String chromedriverpath = chromedriverpath_1+"//chromedriver_2_44.exe";
	//System.out.println(chromedriverpath);
	System.setProperty("webdriver.chrome.driver", chromedriverpath);
	//System.setProperty("user.dir","//chromedriver_2.44.exe");



	ChromeDriverService driverService = ChromeDriverService.createDefaultService();
	ChromeDriver driver = new ChromeDriver(driverService, options);
	
	
	ObjectMapper objectMapper = new ObjectMapper();
	HttpClient httpClient = HttpClientBuilder.create().build();
	String command = null;
	try {
		command = objectMapper.writeValueAsString(prefs);
	} catch (JsonProcessingException e) {
		e.printStackTrace();
	}

	String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
	HttpPost request = new HttpPost(u);
	request.addHeader("content-type", "application/json");
	request.setEntity(new StringEntity(command));
	httpClient.execute(request);
	return driver;
}

amishad02 · 2019-05-15T19:08:33Z

@rishi1992 did you found any solution can you please share.

JDCain · 2019-05-17T15:50:20Z

@amishad02 I use the following successfully

var driver = new ChromeDriver(driverService, options);

var param = new Dictionary<string, string> {{"behavior", "allow"}, {"downloadPath", DownloadPath}
};

var cmdParam = new Dictionary<string, object> {{"cmd", "Page.setDownloadBehavior"}, {"params", param}};

var url = driverService.ServiceUrl + "session/" + driver.SessionId + "/chromium/send_command";
var cli = new WebClient {Headers = {[HttpRequestHeader.ContentType] = "application/json"}};
_ = cli.UploadString(url, JsonConvert.SerializeObject(cmdParam));

pouellet · 2019-05-17T16:12:11Z

Anyone had success sending the /session/:sessionId/chromium/send_command command to a headless chrome process that was launched in a selenium grid? This solution seems to work well for a standalone selenium chrome, but I couldn't get it to work when proxied through the selenium grid.

rishi1992 · 2019-05-20T04:58:15Z

This has worked for me,

// Set the ChromeDriver path
System.out.println("launching Chrome browser");
String chromedriverPath = System.getProperty("user.dir") + "\chromedriver_2_44.exe";
System.setProperty("webdriver.chrome.driver", chromedriverPath);
// Set the download folder path
String downloadPath = System.getProperty("user.dir");
//String downloadPath = "E:\seleniumdownload";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("safebrowsing.enabled", "true");
chromePrefs.put("downloadPath", downloadPath); //This is one of the main line to be written
chromePrefs.put("behavior", "allow"); //This is one of the main line to be written
// Initialize Chrome options
ChromeOptions options = new ChromeOptions();
options.addArguments("--disable-notifications");
options.addArguments("--start-maximized");
options.addArguments("disable-infobars");
options.addArguments("--disable-gpu");
options.addArguments("--headless");
options.addArguments("window-size=1980,1080");
options.addArguments("--allow-running-insecure-content");
options.addArguments("--disable-extensions");
options.addArguments("--no-sandbox");
options.addArguments("--ignore-certificate-errors");
// Initialize driver
ChromeDriverService driverService = ChromeDriverService.createDefaultService();
driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");
commandParams.put("params", chromePrefs);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);
String u = driverService.getUrl().toString() + "/session/" + ((RemoteWebDriver) driver).getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

automationleg · 2019-05-22T11:32:46Z

Any good solution/workaround in Python ?
I tried many ideas posted here and over internet, but nothing seems to work. I have a scenario when I just click on a download button and file get's downloaded(what is not happening in headless).

chevi · 2019-06-06T12:22:56Z

Any good solution/workaround in Python ?
I tried many ideas posted here and over internet, but nothing seems to work. I have a scenario when I just click on a download button and file get's downloaded(what is not happening in headless).

Please check, if there is a new tab is opened, when you click on a download link (for example if has target="_blank" attribute). In my case download in headless with above solution doesn't work for downloads in a new tabs. So you can remove target="_blank" attribute by JS or get href and try to download by direct link in the same tab. This works for me.

cgoldberg · 2019-06-06T13:29:18Z

to enable headless downloads in Python:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True 
driver = Chrome(options=options)
params = {'behavior': 'allow', 'downloadPath': '/path/for/download'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

automationleg · 2019-06-22T22:37:24Z

Thanks cgoldberg - that works fine!

deeptechs · 2019-07-10T07:04:38Z

to enable headless downloads in Python:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True 
driver = Chrome(options=options)
params = {'behavior': 'allow', 'downloadPath': '/path/for/download'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

@cgoldberg What is your env? Chrome and chromedriver versions, standalone selenium or Grid?

Thanks

ander7agar · 2019-07-16T17:26:55Z

to enable headless downloads in Python:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True 
driver = Chrome(options=options)
params = {'behavior': 'allow', 'downloadPath': '/path/for/download'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

@cgoldberg I am using this similar procedure with selenium-webdriver for Node and it does not work for me. I suspect that it is because in my case a new tab is opened to download file (target "_blank"). This is executed through JavaScript code when I click on a button.

Does this happen to anyone else?

ander7agar · 2019-07-16T17:31:16Z

Is there any way to make this permanent? That is, I want it to be enabled by default when a new instance of the browser is opened.

ckempster · 2019-07-22T13:27:48Z

Is there any way to make this permanent? That is, I want it to be enabled by default when a new instance of the browser is opened.

Make use of Hooks

ckempster · 2019-07-22T13:28:41Z

Is there any way to make this permanent? That is, I want it to be enabled by default when a new instance of the browser is opened.

Make use of Hooks

An example I call from my hooks is 👍
private void createChromeBrowser() { WebDriverManager.chromedriver().setup(); String downloadPath = System.getProperty("user.dir"); HashMap<String, Object> chromePrefs = new HashMap<String, Object>(); chromePrefs.put("safebrowsing.enabled", "true"); chromePrefs.put("downloadPath", downloadPath); chromePrefs.put("behavior", "allow"); ChromeOptions options = new ChromeOptions(); options.addArguments("--headless"); options.addArguments("--window-size=1366,768"); options.addArguments("--disable-infobars"); options.addArguments("--incognito"); options.addArguments("--enable-javascript"); options.addArguments("--disable-websecurity"); options.addArguments("--start-maximised"); options.addArguments("--ignore-certificate-errors"); options.addArguments("--disable-popup-blocking"); options.addArguments("--fast-start"); options.addArguments("--javascript-harmony"); options.addArguments("--no-sandbox"); driver = new ChromeDriver(options); }

jonsamwell · 2019-08-06T02:55:56Z

The command does not seem to be working now in Chrome 74+ (using Proctractor 5.4.2 and node 10.16.0)

browser.driver.sendChromiumCommand('Page.setDownloadBehavior', {
      behavior: 'allow',
      downloadPath: fullDownloadDirectoryPath
    });

Fails with the error

UnsupportedOperationError: POST /session/ac771ec3019f7b4c7191e2c9ae2b2056/chromium/send_command

Any idea why it is failing or how to get headless Chrome downloads working again?

lmtierney · 2019-08-06T03:05:23Z

Because the endpoint for that command changed in chromedriver to /session/:sessionId/goog/cdp/execute

jonsamwell · 2019-08-06T03:35:42Z

@lmtierney many thanks for this. I am still getting a 500 internal server error now if I use that endpoint uri instead?

http://192.168.1.189:54193/wd/hub/session/d47c025d923e19c77aacab20f69809f6/goog/cdp/execute

aditya2020 · 2019-10-27T17:50:08Z

This is how I did it:

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

ChromeDriver driver = new ChromeDriver(driverService, options);

Map<String, Object> commandParams = new HashMap<>();
commandParams.put("cmd", "Page.setDownloadBehavior");

Map<String, String> params = new HashMap<>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepathString);
commandParams.put("params", params);

ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();

String command = objectMapper.writeValueAsString(commandParams);

String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

HttpPost request = new HttpPost(u);
request.addHeader("content-type", "application/json");
request.setEntity(new StringEntity(command));
httpClient.execute(request);

This fix is not working with my configuration. I'm still unable to download a file in headless mode.

I agree. My Chrome version is "78.0.3904.70". This doesnt work for me either. I have the same code

NN88 · 2020-01-17T20:32:56Z

This worked for me:

Environment

ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin18]
Rails 4.2.10
chromedriver: 79.0.3945.36
webdrivers (4.0.1)
capybara (3.24.0)

Other resources

https://stackoverflow.com/questions/48810757/setting-default-download-directory-and-headless-chrome

Implementation

options = Selenium::WebDriver::Chrome::Options.new
  options.add_argument('--window-size=1500,1500')
  options.add_argument('--headless')
  options.add_argument('--no-sandbox')
  options.add_argument('--disable-gpu')
  options.add_argument('--disable-popup-blocking')

  options.add_preference(:download,
                          directory_upgrade: true,
                          prompt_for_download: false,
                          default_directory: ENV['downloads_folder']
                          )

  options.add_preference(:browser, set_download_behavior: { behavior: 'allow' })

Capybara.register_driver :headless_chrome do |app|
  driver = Capybara::Selenium::Driver.new(
                                app,
                                browser: :chrome,
                                options: options
                                )

  
  bridge = driver.browser.send(:bridge)
  path = '/session/:session_id/chromium/send_command'
  path[':session_id'] = bridge.session_id

  bridge.http.call(:post, path, cmd: 'Page.setDownloadBehavior',
                                params: {
                                  behavior: 'allow',
                                  downloadPath: ENV['downloads_folder'],
                              })

  driver
end

Capybara.javascript_driver = ENV['capybara_js_driver'].to_sym

sanathfp · 2020-01-23T09:03:26Z

I'm trying to get this working on javascript using nodejs. Is there a solution yet?

victorigualada · 2021-04-26T13:52:50Z

I'm trying to get this working on javascript using nodejs. Is there a solution yet?

I've found a solution to this in Node.JS. You need to set the download path through the driver#setDownloadPath(path) method:

selenium/javascript/node/selenium-webdriver/chromium.js

Line 750 in 83e80c2

async setDownloadPath(path) {

diemol · 2021-10-17T18:56:36Z

Closing as this went stale. If the issue still persists, please open an issue with the ChromeDriver team.

github-actions · 2021-11-17T00:06:22Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

p0deje added C-dotnet C-java C-nodejs C-py C-rb D-chrome I-enhancement labels Dec 4, 2017

IlyaNaumovich added a commit to IlyaNaumovich/selenium that referenced this issue Dec 26, 2017

Preparing chrome driver to download in headless mode

aa2afab

The method send commands to the browser to avoid blocking of downloading in headless mode. (The issue is described here: SeleniumHQ#5159)

IlyaNaumovich mentioned this issue Dec 26, 2017

Preparing chrome driver to download in headless mode #5262

Closed

1 task

jleyba added a commit that referenced this issue Jan 8, 2018

[js] Add support for /session/:sessionId/chromium/send_command (#5159)

f16c321

jleyba removed the C-nodejs label Jan 8, 2018

IlyaNaumovich added a commit to IlyaNaumovich/selenium that referenced this issue Jan 12, 2018

The method send commands to the browser to avoid blocking

6785a3f

of downloading in headless mode. (The issue is described here: SeleniumHQ#5159)

IlyaNaumovich added a commit to IlyaNaumovich/selenium that referenced this issue Jan 20, 2018

The method send commands to the browser to avoid blocking

81858d5

of downloading in headless mode. (The issue is described here: SeleniumHQ#5159)

rsshilli mentioned this issue Feb 7, 2018

How to get Chrome headless to download files? xolvio/chimp#679

Closed

lmtierney added a commit to lmtierney/selenium that referenced this issue Mar 6, 2018

[py] add support for chrome devtools send_command endpoints

4432d08

This adds commands to hit the send_command and send_command_and_get_result endpoints Addresses SeleniumHQ#5159

pulkitsharma07 mentioned this issue Mar 8, 2018

[rb] Adding method to send DevTools commands to Chrome and a helper method to enable downloads on headless Chrome. #5583

Closed

1 task

Naktibalda mentioned this issue Jun 17, 2019

Download a file in headless mode Codeception/Codeception#5518

Closed

arnzel mentioned this issue Aug 19, 2019

Download files when using Headless Browser FluentLenium/FluentLenium#929

Closed

saimadao mentioned this issue Jun 6, 2021

Download file in headless mode robotframework/SeleniumLibrary#1731

Closed

diemol closed this as completed Oct 17, 2021

github-actions bot locked and limited conversation to collaborators Nov 17, 2021

Download file in headless chrome #5159

Download file in headless chrome #5159

Comments

jonhkr commented Dec 2, 2017

Meta -

Expected Behavior -

Actual Behavior -

Steps to reproduce -

p0deje commented Dec 4, 2017

jonhkr commented Dec 4, 2017

jonhkr commented Dec 4, 2017

nstubi commented Jan 3, 2018

shreyanshp commented Jan 11, 2018

daljitcheema commented Feb 1, 2018

pulkitsharma07 commented Feb 13, 2018 • edited Loading

luke-hill commented Feb 14, 2018

pulkitsharma07 commented Feb 14, 2018

jbustillos commented Feb 15, 2018

p0deje commented Feb 15, 2018

pulkitsharma07 commented Feb 15, 2018 • edited Loading

pulkitsharma07 commented Feb 15, 2018

pulkitsharma07 commented Mar 8, 2018

HaveSpacesuit commented Mar 18, 2019

rishi1992 commented Mar 19, 2019

rishi1992 commented Mar 19, 2019

amishad02 commented May 15, 2019

JDCain commented May 17, 2019

pouellet commented May 17, 2019

rishi1992 commented May 20, 2019

automationleg commented May 22, 2019

chevi commented Jun 6, 2019

cgoldberg commented Jun 6, 2019

automationleg commented Jun 22, 2019

deeptechs commented Jul 10, 2019

ander7agar commented Jul 16, 2019

ander7agar commented Jul 16, 2019

ckempster commented Jul 22, 2019

ckempster commented Jul 22, 2019

jonsamwell commented Aug 6, 2019 • edited Loading

lmtierney commented Aug 6, 2019

jonsamwell commented Aug 6, 2019

aditya2020 commented Oct 27, 2019 • edited Loading

NN88 commented Jan 17, 2020

Environment

Other resources

Implementation

sanathfp commented Jan 23, 2020

victorigualada commented Apr 26, 2021

diemol commented Oct 17, 2021

github-actions bot commented Nov 17, 2021

pulkitsharma07 commented Feb 13, 2018 •

edited

Loading

pulkitsharma07 commented Feb 15, 2018 •

edited

Loading

jonsamwell commented Aug 6, 2019 •

edited

Loading

aditya2020 commented Oct 27, 2019 •

edited

Loading