git.commit results in stdout maxBuffer exceeded

[loveencounterflow]

In my gulpfile I have

gulp.task 'git-add-all', ->
  return gulp.src '*'
    .pipe git.add { args: '--all' }

gulp.task 'git-commit', ->
  return gulp.src '*'
    .pipe git.commit "gulp autocommit"

When i do

gulp git-add-all && gulp git-commit

i'm getting

[16:22:56] 'git-commit' errored after 1.06 s
[16:22:56] Error: stdout maxBuffer exceeded.
  at Socket.<anonymous> (child_process.js:699:13)
  at Socket.emit (events.js:95:17)
  at Socket.<anonymous> (_stream_readable.js:765:14)
  at Socket.emit (events.js:92:17)
  at emitReadable_ (_stream_readable.js:427:10)
  at emitReadable (_stream_readable.js:423:5)
  at readableAddChunk (_stream_readable.js:166:9)
  at Socket.Readable.push (_stream_readable.js:128:10)
  at Pipe.onread (net.js:529:21)

Is this a gulp-git error or am i doing something wrong?

[stephenlacy]

It is a node child_process.exec buffer error.
The amount of data flowing through stdout is larger than the max default value.

It can be changed in gulp-git

[loveencounterflow]

Where and how can i change this limitation in gulp-git?

IMHO it is a known fact that the buffer size of child_process.exec is limited. Also, we know that when git gets to process a lot of files, it also spits out a lot of messages. I think it's a flaw in the gulp-git implementation that this data is not altogether silently discarded but complains when exceeding a certain limit; this makes workflows susceptible to hard-to-detect bugs that will crop up only if a given git step has to process that one file entry, that one line of discardable output that exceeds an arbitrary limit.

http://www.hacksparrow.com/difference-between-spawn-and-exec-of-node-js-child_process.html puts it well:

child_process.exec returns the whole buffer output from the child process. By default the buffer size is set at 200k. If the child process returns anything more than that, you program will crash with the error message "Error: maxBuffer exceeded". You can fix that problem by setting a bigger buffer size in the exec options. But you should not do it because exec is not meant for processes that return HUGE buffers to Node. You should use spawn for that. So what do you use exec for? Use it to run programs that return result statuses, instead of data.

Any reason to use child_process.exec instead of child_process.spawn at all? gulp is inherently asynchronous, and so should be gulp plugins. Just my 2 cents.

[stephenlacy]

PR Welcome.

[loveencounterflow]

So i researched a bit into this, and apart from the fact that spawn takes a command and an array of options where exec expects a single string as written on the command line, spawn executes a command directly whereas exec uses the shell, meaning that metacharacters and variables will get expanded. This would seem to entail that if there was a drop-in replacement for exec that underneath uses spawn (but otherwise tries to accept arguments and call the callback as exec does) would still need to deal with shell expansion issues. Without shell expansions, we should expect some changes in behavior.

That said, exec accepts as second argument an options object where maxBuffer may be set. I haven't tried yet but { maxBuffer: Infinity } should remove the buffer error.

[stringparser]

@loveencounterflow was { maxBuffer: Infinity } the solution?

[stephenlacy]

I'll add the alteration if it is determined that the Infinity will fix it.

[pfraces-ob]

Indeed, {maxBuffer: Infinity } solves the issue. The setting can be passed to gulp-git methods through the options object:

gulp.task('commit', function () {
    gulp.src('src/')
        .pipe(git.add({ args: '--all', maxBuffer: Infinity }))
        .pipe(git.commit('auto-generated commit', { maxBuffer: Infinity }));
});