Skip to content
This repository has been archived by the owner on Aug 31, 2021. It is now read-only.

"the command line is too long" #30

Open
MitchPope opened this issue Oct 7, 2020 · 4 comments
Open

"the command line is too long" #30

MitchPope opened this issue Oct 7, 2020 · 4 comments

Comments

@MitchPope
Copy link

I'm using the module to merge a lot of single page pdf's into one but when there's more than about 100, I get the error message:
Error: Command failed: java -jar "C:\Users\oem\Desktop\electrion-workspace\prints processing\node_modules\electron\dist\resources\easy-pdf-merge\jar\pdfbox.jar" PDFMerger
// a very long list of file paths//
The command line is too long.

I'm assuming I can get around this by merging smaller groups together as temp files then merging them together but that might be slower to run so before I rewrite my code I want to check if this is a bug/artificial limitation? Another command line pdf tool listed as a bug fix so I have hope.
@karuppiah7890
Copy link
Owner

karuppiah7890 commented Oct 7, 2020 •
edited
Loading

I'm not sure about this one. I'm actually not maintaining this library much - it's a small piece of JavaScript code which delegates most of the work to PDFBox https://pdfbox.apache.org/ as you might already know.

This sounds like a nodejs level issue? I used NodeJS APIs to execute commands

https://github.com/karuppiah7890/easy-pdf-merge/blob/master/source/lib/PDFMerger.js#L1

I'm guessing it might have given the error that "hey, you asked me to run this command, but this command is too long". Gotta dig into it though by using lots of files. So, can't say for sure till then. And I don't have so much files. I could simply try to reuse one single PDF and try it 100s of times in one go maybe

@karuppiah7890
Copy link
Owner

karuppiah7890 commented Oct 7, 2020 •
edited
Loading

I just tried the below and things still worked fine for me

var path = require('path');
var merge = require(`./source/index.js`);

let pdfs = [`test${path.sep}github cheat sheet.pdf`, `test${path.sep}text_extraction.pdf`]

let allPDFs = []

for (let i = 0; i < 500; i++) {
  allPDFs.push(...pdfs);
}

console.log(allPDFs.length);
console.log(allPDFs);

merge(allPDFs, `test${path.sep}Out.pdf`, function (err) {

  if (err)
    return console.log(err);

  console.log("Success");
});

You can find similar code in the repo test directory with test data. Maybe my file names are not too long I guess? I still have 1000 of PDFs as input

$ npm test

> easy-pdf-merge@0.2.5 test /Users/karuppiahn/oss/github.com/karuppiah7890/easy-pdf-merge
> node test.js

1000
[
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  'test/github cheat sheet.pdf',
  'test/text_extraction.pdf',
  ... 900 more items
]
Success

$ ls -alh test
total 394008
drwxr-xr-x   5 karuppiahn  staff   160B Jul 22 20:29 .
drwxr-xr-x  14 karuppiahn  staff   448B Aug 23 22:38 ..
-rw-r--r--   1 karuppiahn  staff   180M Oct  7 19:57 Out.pdf
-rwxr-xr-x   1 karuppiahn  staff   369K Jul 22 20:29 github cheat sheet.pdf
-rw-r--r--   1 karuppiahn  staff    13K Jul 22 20:29 text_extraction.pdf

@pguardiario
Copy link

I worked around this by putting them in a folder and using a glob but I have to make an ugly patch

@Miniland1333
Copy link
Contributor

FYI this issue appears to be somewhat shell dependent, at least for when I was encountering it. When running through nodeJS on windows+CMD, it seems to have a lower limit than say Bash on Ubuntu. In my logs, I found the following command encountered this error:

java -jar -Xmx3G "/root/nodePrint/pdfbox.jar" PDFMerger "./PDF/Letter/order/771ff9/000.pdf" "./PDF/Letter/order/771ff9/001.pdf" "./PDF/Letter/order/771ff9/002.pdf" "./PDF/Letter/order/771ff9/003.pdf" "./PDF/Letter/order/771ff9/005.pdf" "./PDF/Letter/order/771ff9/006.pdf" "./PDF/Letter/order/771ff9/007.pdf" "./PDF/Letter/order/771ff9/008.pdf" "./PDF/Letter/order/771ff9/009.pdf" "./PDF/Letter/order/771ff9/010.pdf" "./PDF/Letter/order/771ff9/011.pdf" "./PDF/Letter/order/771ff9/012.pdf" "./PDF/Letter/order/771ff9/013.pdf" "./PDF/Letter/order/771ff9/014.pdf" "./PDF/Letter/order/771ff9/015.pdf" "./PDF/Letter/order/771ff9/016.pdf" "./PDF/Letter/order/771ff9/017.pdf" "./PDF/Letter/order/771ff9/018.pdf" "./PDF/Letter/order/771ff9/019.pdf" "./PDF/Letter/order/771ff9/020.pdf" "./PDF/Letter/order/771ff9/021.pdf" "./PDF/Letter/order/771ff9/022.pdf" "./PDF/Letter/order/771ff9/023.pdf" "./PDF/Letter/order/771ff9/024.pdf" "./PDF/Letter/order/771ff9/025.pdf" "./PDF/Letter/order/771ff9/026.pdf" "./PDF/Letter/order/771ff9/027.pdf" "./PDF/Letter/order/771ff9/028.pdf" "./PDF/Letter/order/771ff9/029.pdf" "./PDF/Letter/order/771ff9/030.pdf" "./PDF/Letter/order/771ff9/031.pdf" "./PDF/Letter/order/771ff9/032.pdf" "./PDF/Letter/order/771ff9/033.pdf" "./PDF/Letter/order/771ff9/034.pdf" "./PDF/Letter/order/771ff9/035.pdf" "./PDF/Letter/order/771ff9/036.pdf" "./PDF/Letter/order/771ff9/037.pdf" "./PDF/Letter/order/771ff9/038.pdf" "./PDF/Letter/order/771ff9/039.pdf" "./PDF/Letter/order/771ff9/040.pdf" "./PDF/Letter/order/771ff9/041.pdf" "./PDF/Letter/order/771ff9/042.pdf" "./PDF/Letter/order/771ff9/043.pdf" "./PDF/Letter/order/771ff9/044.pdf" "./PDF/Letter/order/771ff9/045.pdf" "./PDF/Letter/order/771ff9/046.pdf" "./PDF/Letter/order/771ff9/047.pdf" "./PDF/Letter/order/771ff9/048.pdf" "./PDF/Letter/order/771ff9/049.pdf" "./PDF/Letter/order/771ff9/050.pdf" "./PDF/Letter/order/771ff9/051.pdf" "./PDF/Letter/order/771ff9/052.pdf" "./PDF/Letter/order/771ff9/053.pdf" "./PDF/Letter/order/771ff9/054.pdf" "./PDF/Letter/order/771ff9/055.pdf" "./PDF/Letter/order/771ff9/056.pdf" "./PDF/Letter/order/771ff9/057.pdf" "./PDF/Letter/order/771ff9/058.pdf" "./PDF/Letter/order/771ff9/059.pdf" "./PDF/Letter/order/771ff9/060.pdf" "./PDF/Letter/order/771ff9/061.pdf" "./PDF/Letter/order/771ff9/062.pdf" "./PDF/Letter/order/771ff9/063.pdf" "./PDF/Letter/order/771ff9/064.pdf" "./PDF/Letter/order/771ff9/065.pdf" "./PDF/Letter/order/771ff9/066.pdf" "./PDF/Letter/order/771ff9/067.pdf" "./PDF/Letter/order/771ff9/068.pdf" "./PDF/Letter/order/771ff9/069.pdf" "./PDF/Letter/order/771ff9/070.pdf" "./PDF/Letter/order/771ff9/071.pdf" "./PDF/Letter/order/771ff9/072.pdf" "./PDF/Letter/order/771ff9/073.pdf" "./PDF/Letter/order/771ff9/074.pdf" "./PDF/Letter/order/771ff9/075.pdf" "./PDF/Letter/order/771ff9/076.pdf" "./PDF/Letter/order/771ff9/077.pdf" "./PDF/Letter/order/771ff9/078.pdf" "./PDF/Letter/order/771ff9/079.pdf" "./PDF/Letter/order/771ff9/080.pdf" "./PDF/Letter/order/771ff9/081.pdf" "./PDF/Letter/order/771ff9/082.pdf" "./PDF/Letter/order/771ff9/083.pdf" "./PDF/Letter/order/771ff9/084.pdf" "./PDF/Letter/order/771ff9/085.pdf" "./PDF/Letter/order/771ff9/086.pdf" "./PDF/Letter/order/771ff9/087.pdf" "./PDF/Letter/order/771ff9/088.pdf" "./PDF/Letter/order/771ff9/089.pdf" "./PDF/Letter/order/771ff9/090.pdf" "./PDF/Letter/order/771ff9/091.pdf" "./PDF/Letter/order/771ff9/092.pdf" "./PDF/Letter/order/771ff9/093.pdf" "./PDF/Letter/order/771ff9/094.pdf" "./PDF/Letter/order/771ff9/095.pdf" "./PDF/Letter/order/771ff9/096.pdf" "./PDF/Letter/order/771ff9/097.pdf" "./PDF/Letter/order/771ff9/098.pdf" "./PDF/Letter/order/771ff9/099.pdf" "./PDF/Letter/order/771ff9/100.pdf" "./PDF/Letter/order/771ff9/101.pdf" "./PDF/Letter/order/771ff9/102.pdf" "./PDF/Letter/order/771ff9/103.pdf" "./PDF/Letter/order/771ff9/104.pdf" "./PDF/Letter/order/771ff9/105.pdf" "./PDF/Letter/order/771ff9/106.pdf" "./PDF/Letter/order/771ff9/107.pdf" "./PDF/Letter/order/771ff9/108.pdf" "./PDF/Letter/order/771ff9/109.pdf" "./PDF/Letter/order/771ff9/110.pdf" "./PDF/Letter/order/771ff9/111.pdf" "./PDF/Letter/order/771ff9/112.pdf" "./PDF/Letter/order/771ff9/113.pdf" "./PDF/Letter/order/771ff9/114.pdf" "./PDF/Letter/order/771ff9/115.pdf" "./PDF/Letter/order/771ff9/116.pdf" "./PDF/Letter/order/771ff9/117.pdf" "./PDF/Letter/order/771ff9/118.pdf" "./PDF/Letter/order/771ff9/119.pdf" "./PDF/Letter/order/771ff9/120.pdf" "./PDF/Letter/order/771ff9/121.pdf" "./PDF/Letter/order/771ff9/122.pdf" "./PDF/Letter/order/771ff9/123.pdf" "./PDF/Letter/order/771ff9/124.pdf" "./PDF/Letter/order/771ff9/125.pdf" "./PDF/Letter/order/771ff9/126.pdf" "./PDF/Letter/order/771ff9/127.pdf" "./PDF/Letter/order/771ff9/128.pdf" "./PDF/Letter/order/771ff9/129.pdf" "./PDF/Letter/order/771ff9/130.pdf" "./PDF/Letter/order/771ff9/131.pdf" "./PDF/Letter/order/771ff9/132.pdf" "./PDF/Letter/order/771ff9/133.pdf" "./PDF/Letter/order/771ff9/134.pdf" "./PDF/Letter/order/771ff9/135.pdf" "./PDF/Letter/order/771ff9/136.pdf" "./PDF/Letter/order/771ff9/137.pdf" "./PDF/Letter/order/771ff9/138.pdf" "./PDF/Letter/order/771ff9/139.pdf" "./PDF/Letter/order/771ff9/140.pdf" "./PDF/Letter/order/771ff9/141.pdf" "./PDF/Letter/order/771ff9/142.pdf" "./PDF/Letter/order/771ff9/143.pdf" "./PDF/Letter/order/771ff9/144.pdf" "./PDF/Letter/order/771ff9/145.pdf" "./PDF/Letter/order/771ff9/146.pdf" "./PDF/Letter/order/771ff9/147.pdf" "./PDF/Letter/order/771ff9/148.pdf" "./PDF/Letter/order/771ff9/149.pdf" "./PDF/Letter/order/771ff9/150.pdf" "./PDF/Letter/order/771ff9/151.pdf" "./PDF/Letter/order/771ff9/152.pdf" "./PDF/Letter/order/771ff9/153.pdf" "./PDF/Letter/order/771ff9/154.pdf" "./PDF/Letter/order/771ff9/155.pdf" "./PDF/Letter/order/771ff9/156.pdf" "./PDF/Letter/order/771ff9/157.pdf" "./PDF/Letter/order/771ff9/158.pdf" "./PDF/Letter/order/771ff9/159.pdf" "./PDF/Letter/order/771ff9/160.pdf" "./PDF/Letter/order/771ff9/161.pdf" "./PDF/Letter/order/771ff9/162.pdf" "./PDF/Letter/order/771ff9/163.pdf" "./PDF/Letter/order/771ff9/164.pdf" "./PDF/Letter/order/771ff9/165.pdf" "./PDF/Letter/order/771ff9/166.pdf" "./PDF/Letter/order/771ff9/167.pdf" "./PDF/Letter/order/771ff9/168.pdf" "./PDF/Letter/order/771ff9/169.pdf" "./PDF/Letter/order/771ff9/170.pdf" "./PDF/Letter/order/771ff9/171.pdf" "./PDF/Letter/order/771ff9/172.pdf" "./PDF/Letter/order/771ff9/173.pdf" "./PDF/Letter/order/771ff9/174.pdf" "./PDF/Letter/order/771ff9/175.pdf" "./PDF/Letter/order/771ff9/176.pdf" "./PDF/Letter/order/771ff9/177.pdf" "./PDF/Letter/order/771ff9/178.pdf" "./PDF/Letter/order/771ff9/179.pdf" "./PDF/Letter/order/771ff9/180.pdf" "./PDF/Letter/order/771ff9/181.pdf" "./PDF/Letter/order/771ff9/182.pdf" "./PDF/Letter/order/771ff9/183.pdf" "./PDF/Letter/order/771ff9/184.pdf" "./PDF/Letter/order/771ff9/185.pdf" "./PDF/Letter/order/771ff9/186.pdf" "./PDF/Letter/order/771ff9/187.pdf" "./PDF/Letter/order/771ff9/188.pdf" "./PDF/Letter/order/771ff9/189.pdf" "./PDF/Letter/order/771ff9/190.pdf" "./PDF/Letter/order/771ff9/191.pdf" "./PDF/Letter/order/771ff9/192.pdf" "./PDF/Letter/order/771ff9/193.pdf" "./PDF/Letter/order/771ff9/194.pdf" "./PDF/Letter/order/771ff9/195.pdf" "./PDF/Letter/order/771ff9/196.pdf" "./PDF/Letter/order/771ff9/197.pdf" "./PDF/Letter/order/771ff9/198.pdf" "./PDF/Letter/order/771ff9/199.pdf" "./PDF/Letter/order/771ff9/200.pdf" "./PDF/Letter/order/771ff9/201.pdf" "./PDF/Letter/order/771ff9/202.pdf" "./PDF/Letter/order/771ff9/203.pdf" "./PDF/Letter/order/771ff9/204.pdf" "./PDF/Letter/order/771ff9/205.pdf" "./PDF/Letter/order/771ff9/206.pdf" "./PDF/Letter/order/771ff9/207.pdf" "./PDF/Letter/order/771ff9/208.pdf" "./PDF/Letter/order/771ff9/209.pdf" "./PDF/Letter/order/771ff9/210.pdf" "./PDF/Letter/order/771ff9/211.pdf" "./PDF/Letter/order/771ff9/212.pdf" "./PDF/Letter/order/771ff9/213.pdf" "./PDF/Letter/order/771ff9/214.pdf" "./PDF/Letter/order/771ff9/215.pdf" "./PDF/Letter/order/771ff9/216.pdf" "./PDF/Letter/order/771ff9/217.pdf" "./PDF/Letter/order/771ff9/218.pdf" "./PDF/Letter/order/771ff9/219.pdf" "./PDF/Letter/order/771ff9/220.pdf" "./PDF/Letter/order/771ff9/221.pdf" "./PDF/Letter/order/771ff9/222.pdf" "./PDF/Letter/order/771ff9/223.pdf" "./PDF/Letter/order/771ff9/224.pdf" "./PDF/Letter/order/771ff9/225.pdf" "./PDF/Letter/order/771ff9/226.pdf" "./PDF/Letter/order/771ff9/227.pdf" "./PDF/Letter/order/771ff9/228.pdf" "./PDF/Letter/order/771ff9/229.pdf" "./PDF/Letter/order/771ff9/230.pdf" "./PDF/Letter/order/771ff9/231.pdf" "./PDF/Letter/order/771ff9/232.pdf" "./PDF/Letter/order/771ff9/233.pdf" "./PDF/Letter/order/771ff9/234.pdf" "./PDF/Letter/order/771ff9/235.pdf" "./PDF/Letter/order/771ff9/236.pdf" "./PDF/Letter/order/771ff9/237.pdf" "./PDF/Letter/order/771ff9/238.pdf" "./PDF/Letter/order/771ff9/239.pdf" "./PDF/Letter/order/771ff9/240.pdf" "./PDF/Letter/order/771ff9/241.pdf" "./PDF/Letter/order/771ff9/242.pdf" "./PDF/Letter/order/771ff9/243.pdf" "./PDF/Letter/order/771ff9/244.pdf" "./PDF/Letter/order/771ff9/245.pdf" "./PDF/Letter/order/771ff9/246.pdf" "./PDF/Letter/order/771ff9/247.pdf" "./PDF/Letter/order/771ff9/248.pdf" "./PDF/Letter/order/771ff9/249.pdf" "./PDF/Letter/order/771ff9/250.pdf" "./PDF/Letter/order/771ff9/251.pdf" "./PDF/Letter/order/771ff9/252.pdf" "./PDF/Letter/order/771ff9/253.pdf" "./PDF/Letter/order/771ff9/254.pdf" "./PDF/Letter/order/771ff9/255.pdf" "./PDF/Letter/order/771ff9/256.pdf" "./PDF/Letter/order/771ff9/257.pdf" "./PDF/Letter/order/771ff9/258.pdf" "./PDF/Letter/order/771ff9/259.pdf" "./PDF/Letter/order/771ff9/260.pdf" "./PDF/Letter/order/771ff9/261.pdf" "./PDF/Letter/order/771ff9/262.pdf" "./PDF/Letter/order/771ff9/263.pdf" "./PDF/Letter/order/771ff9/264.pdf" "./PDF/Letter/order/771ff9/265.pdf" "./PDF/Letter/order/771ff9/266.pdf" "./PDF/Letter/order/771ff9/267.pdf" "./PDF/Letter/order/771ff9/268.pdf" "./PDF/Letter/order/771ff9/269.pdf" "./PDF/Letter/order/771ff9/270.pdf" "./PDF/Letter/order/771ff9/271.pdf" "./PDF/Letter/order/771ff9/272.pdf" "./PDF/Letter/order/771ff9/273.pdf" "./PDF/Letter/order/771ff9/274.pdf" "./PDF/Letter/order/771ff9/275.pdf" "./PDF/Letter/order/771ff9/276.pdf" "./PDF/Letter/Finished/output-dir/Full.pdf"

Since PDFBox does not have a built in method to pass in a directory, my personal workaround was to modify my version of PDFMerger.js to allow opts.cwd to be passed to exec. That way, instead of passing in long relative paths I can just pass the filenames. @MitchPope, do you know what shell/OS you were using when you encountered this issue yourself?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MitchPope @karuppiah7890 @Miniland1333 @pguardiario