File and Image Uploads with Express and Firebase Cloud Functions

Published almost 3 years ago.

I recently spent longer than I'd care to admit trying to figure out how to properly do file or image uploads with Express and Firebase Cloud Functions.

TL;DR do not try and use Multer for this. As it turns out, Cloud Functions introduced a breaking middleware for Multer which parses the body of the request. The result is that Multer will always return a blank req.body, req.file and req.files.

So what are your options? Google documents them both here, but let me save you a click:

  1. Use busboy to directly parse multipart form-data and do something with it (sample code).
  2. Upload directly Google Cloud Storage directly using Signed URLs (sample code)

My solution was to create and use a middleware based off the code samples Google had that make things feel more Multer-y.

// middleware.js

exports.filesUpload = function (req, res, next) {
  // See https://cloud.google.com/functions/docs/writing/http#multipart_data
  const busboy = new Busboy({
    headers: req.headers,
    limits: {
      // Cloud functions impose this restriction anyway
      fileSize: 10 * 1024 * 1024,
    },
  });

  const fields = {};
  const files = [];
  const fileWrites = [];
  // Note: os.tmpdir() points to an in-memory file system on GCF
  // Thus, any files in it must fit in the instance's memory.
  const tmpdir = os.tmpdir();

  busboy.on("field", (key, value) => {
    // You could do additional deserialization logic here, values will just be
    // strings
    fields[key] = value;
  });

  busboy.on("file", (fieldname, file, filename, encoding, mimetype) => {
    const filepath = path.join(tmpdir, filename);
    console.log(
      `Handling file upload field ${fieldname}: ${filename} (${filepath})`
    );
    const writeStream = fs.createWriteStream(filepath);
    file.pipe(writeStream);

    fileWrites.push(
      new Promise((resolve, reject) => {
        file.on("end", () => writeStream.end());
        writeStream.on("finish", () => {
          fs.readFile(filepath, (err, buffer) => {
            const size = Buffer.byteLength(buffer);
            console.log(`${filename} is ${size} bytes`);
            if (err) {
              return reject(err);
            }

            files.push({
              fieldname,
              originalname: filename,
              encoding,
              mimetype,
              buffer,
              size,
            });

            try {
              fs.unlinkSync(filepath);
            } catch (error) {
              return reject(error);
            }

            resolve();
          });
        });
        writeStream.on("error", reject);
      })
    );
  });

  busboy.on("finish", () => {
    Promise.all(fileWrites)
      .then(() => {
        req.body = fields;
        req.files = files;
        next();
      })
      .catch(next);
  });

  busboy.end(req.rawBody);
};

Similar to Multer, this middleware will ensure req.body are any text fields in form-data and req.files is an array of uploaded files.

const express = require("express");

const { filesUpload } = require("./middleware");

app = express();

app.post("/upload", filesUpload, function (req, res) {
  // will contain all text fields
  req.body;
  // will contain an array of file objects
  /*
    {
      fieldname: 'image',       String - name of the field used in the form
      originalname,             String - original filename of the uploaded image
      encoding,                 String - encoding of the image (e.g. "7bit")
      mimetype,                 String - MIME type of the file (e.g. "image/jpeg")
      buffer,                   Buffer - buffer containing binary data
      size,                     Number - size of buffer in bytes
    }
  */
  req.files;
});

exports = app;

Hope this helps.

Mike Sukmanowsky

๐Ÿ‘‹๐Ÿป Hi! I'm Mike the author of this post and others.

If you'd like to get in touch, you can reach me on Twitter or LinkedIn.