attachData from Buffer on server causes corrupted files.

[rw1018]

I'm using redactor wysiwyg editor, which has an image upload implementation that sends a post request to a given url then expects back some JSON like: { "filelink": "/static/img.jpg" }.

I've created a server-side route that receives the post request, buffers the file, then creates and uploads a FS file instance.

Router.route('/s3', function() {
    var buffers = [];
    var totalLength = 0;
    var imageUpload = Meteor.bindEnvironment(function(file){
      Images.insert(file);
    }, function(e) {
      throw e;
    });

    this.request.on('data', function (chunk) {
        buffers.push(chunk);
        totalLength += chunk.length;
        var body = Npm.require('querystring').parse(chunk.toString())
    });

    this.request.on('end', function() {
      var data = Buffer.concat(buffers);
      var newFile = new FS.File();
      newFile.attachData(data, {type: 'image/png'}, function(error){
        if (error) throw error;
        newFile.name('test.png');
        imageUpload(newFile);
      });
    });

The problem is that every image I upload ends up corrupted and a couple of hundred bytes smaller than the original.

Any suggestions how to solve this?

[zhenyasav]

I had a similar issue on the client recently, and worked around by delaying the insert after the attachData by 1 second. Kind of dirty, but I'm glad you're also reporting the symptom.

[aldeed]

Try calling newFile.size(totalLength) after your newFile.name call.

Also, you should technically be able to attach the stream directly, but there are some issues with that in some cases, which I'll hopefully fix within the next week.

var newFile = new FS.File();
newFile.name('test.png');
newFile.attachData(this.request, {type: 'image/png'});
newFile = Images.insert(newFile);

I don't know if it will work for you or not because right now there are issues with it not properly setting the size using this method.

[raix]

@aldeed could it be pause or missing buffering on request stream?

[rw1018]

@zhenyasav Would you mind posting how you delayed the insert?

@aldeed I tried setting the size of the file object -- I can confirm that setting the size doesn't work.

I've been trying to figure this out for awhile now. My best guess is that there is some problem with the encoding when passing the buffer to attachData. The corrupted files all seem to be a slightly larger file size than the originals.

[zhenyasav]

Based on the feedback from @raix and @aldeed I doubt we are experiencing the same issue (mine was a client-side problem), but the delay is just a simple setTimeout. I basically took some time between when the async callback fires and when I proceed to do the insert in the cont continuation. Disclaimer: this issue was among the weirdest, and this isn't a solution at all, just a hack that worked at the time. Hope that helps somehow.

var attachDataHelper = function(file, data, cont) {
  file.attachData(data, function() {
    var args = arguments;
    Meteor.setTimeout(function() {
      if (typeof (cont && cont.apply) === "function") cont.apply(this, args);
    }, 1000);
  });
};

[raix]

Yeah - @zhenyasav you dont want to do that.. I'm sure its a timing issue,
Ref: https://github.com/CollectionFS/Meteor-cfs-access-point/blob/master/access-point-handlers.js#L183