Getting file size 0 after upload

[piyushdhamecha]

Hi guys,

I am facing one issue.

I have created middleware with multer and multer s3 to upload objects to aws s3

all seems working fine, objects are uploading properly, but in some of the files the size is 0

I investigated and found that if the file size is more than 5MB then it returns a 0

Middleware code uploadAwsS3.js

const path = require('path');
const multer = require('multer');
const multerS3 = require('multer-s3');
const { S3Client } = require('@aws-sdk/client-s3');

const filePath = 'newUploads';
const month = ['january', 'february', 'march', 'april', 'may', 'june', 'july', 'august', 'september', 'october', 'november', 'december'];

// create s3 instance using S3Client
// (this is how we create s3 instance in v3)
const s3 = new S3Client({
  credentials: {
    accessKeyId: process.env.AWS_S3_KEY, // store it in .env file to keep it safe
    secretAccessKey: process.env.AWS_S3_SECRET,
  },
  region: process.env.AWS_S3_REGION, // this is the region that you select in AWS account
});

const s3Storage = multerS3({
  s3, // s3 instance
  bucket: process.env.AWS_S3_BUCKET, // change it as per your project requirement
  contentType: multerS3.AUTO_CONTENT_TYPE,
  acl: 'public-read', // storage access type
  metadata: (req, file, cb) => {
    cb(null, { fieldname: file.fieldname });
  },
  key: (req, file, cb) => {
    const d = new Date();
    const current_month = month[d.getMonth()];
    const currentFolder = current_month.concat(new Date().getFullYear().toString());

    const filename = path.parse(file.originalname).name;
    const model = filename.split('_').slice(-2, -1)[0];
    const dir = `${filePath}/${model}/${currentFolder}`;
    const orgFileParse = path.parse(file.originalname);

    const fileName = `${dir}/${orgFileParse.name}_${Date.now()}${orgFileParse.ext}`;

    cb(null, fileName);
  },
});

// function to sanitize files and send error for unsupported files
const sanitizeFile = (file, cb) => {
  // Define the allowed extension
  const fileExts = ['.png', '.jpg', '.jpeg', '.gif', '.zip', '.mp4'];

  // Check allowed extensions
  const isAllowedExt = fileExts.includes(
    path.extname(file.originalname.toLowerCase()),
  );

  if (isAllowedExt) {
    return cb(null, true); // no errors
  }
  // pass error msg to callback, which can be displaye in frontend
  return cb('Error: File type not allowed!');
};

// our middleware
const uploadImage = multer({
  storage: s3Storage,
  fileFilter: (req, file, callback) => {
    sanitizeFile(file, callback);
  }
});

module.exports = uploadImage;

Router

router.post('/', uploadAwsS3.array('files'), DocumentUploadController.create);

[realtrynna]

I am experiencing the same phenomenon.

[alexmarco]

Hi, i have the same problem

[ABaaS-dev]

Hello, I'm also facing the same issue.
Is there any plan on fixing this issue ?
Thanks

[HofmT]

Does this PR fix the issue for you, guys? https://github.com/anacronw/multer-s3/pull/204/files

[piyushdhamecha]

Does this PR fix the issue for you, guys? https://github.com/anacronw/multer-s3/pull/204/files

Need to check, @HofmT have you checked?

[HofmT]

@piyushdhamecha I just tried it, the callback is now returning the proper value for currentSize

Here's an extract from my debugging logs with the fix:

Before creating an upload object
progress  {
  loaded: 4539430,
  total: undefined,
  part: 2,
  Key: 'avatars/7wmT5UFW7-lCeOjMU0_VAr~ONjJsR6V0Vkrsp~bMcQm2ATxs24.jpg',
  Bucket: 'plural-development-files'
}
progress  {
  loaded: 9782310,
  total: undefined,
  part: 1,
  Key: 'avatars/7wmT5UFW7-lCeOjMU0_VAr~ONjJsR6V0Vkrsp~bMcQm2ATxs24.jpg',
  Bucket: 'plural-development-files'
}
currentSize  9782310

This is my modified code from index.js:

upload.on('httpUploadProgress', function (ev) {
      console.log("progress ", ev);
      currentSize = (ev.total) ? ev.total : ev.loaded
    })

    util.callbackify(upload.done.bind(upload))(function (err, result) {
      if (err) return cb(err)

      console.log('currentSize ', currentSize);
      cb(null, {
        size: currentSize,
        bucket: opts.bucket,
        key: opts.key,
        acl: opts.acl,
        contentType: opts.contentType,
        contentDisposition: opts.contentDisposition,
        contentEncoding: opts.contentEncoding,
        storageClass: opts.storageClass,
        serverSideEncryption: opts.serverSideEncryption,
        metadata: opts.metadata,
        location: result.Location,
        etag: result.ETag,
        versionId: result.VersionId
      })
    })

[piyushdhamecha]

@HofmT nice solution, will wait for PR to be merged and update with the latest package and test it, And thanks for the solution

[tuffnutt]

Does this PR fix the issue for you, guys? https://github.com/anacronw/multer-s3/pull/204/files

This PR works, and you can use it by running the following command to install the specific commit:
npm install anacronw/multer-s3#b0f0ffb55c1642b699a032a48c3bc1b942b6a10e
This should fix the issue.
Thanks @HofmT

[Tom5om]

That fix seems to work! Can we get it merged please?