Phase 2 — Building REST APIs lesson-0014

File Uploads

File uploads are one of the highest-risk features in any API. Done right they're straightforward. Done wrong they're a major attack surface and an operational headache.

How File Uploads Work Under the Hood

Files are transmitted as multipart/form-data — a special Content-Type that splits the HTTP body into named "parts", each separated by a boundary string. Each part has its own headers and can carry either text or binary data. This is fundamentally different from JSON:

POST /api/v1/users/42/avatar
Content-Type: multipart/form-data; boundary=----FormBoundary7MA4YWxk

------FormBoundary7MA4YWxk
Content-Disposition: form-data; name="avatar"; filename="photo.jpg"
Content-Type: image/jpeg

[binary JPEG data here...]
------FormBoundary7MA4YWxk
Content-Disposition: form-data; name="caption"

My profile photo
------FormBoundary7MA4YWxk--

express.json() cannot parse this format. You need Multer — a middleware that parses multipart bodies and makes files available on req.file or req.files.

Multer — Setup & Configuration

npm install multer
// src/middleware/upload.js
const multer = require('multer');
const path   = require('path');
const crypto = require('crypto');
const { BadRequestError } = require('../errors/AppError');

// ── Allowed MIME types by category ────────────────────────────────
const MIME_TYPES = {
  image:    ['image/jpeg', 'image/png', 'image/webp', 'image/gif'],
  document: ['application/pdf', 'text/plain'],
  video:    ['video/mp4', 'video/webm'],
};

// ── File filter factory ────────────────────────────────────────────
const filterByType = (allowedTypes) => (req, file, cb) => {
  if (allowedTypes.includes(file.mimetype)) {
    cb(null, true);
  } else {
    // Pass error to Multer — it forwards to Express error handler
    cb(new BadRequestError(
      `Invalid file type. Allowed: ${allowedTypes.join(', ')}`
    ));
  }
};

// ── Disk storage ──────────────────────────────────────────────────
const diskStorage = multer.diskStorage({
  destination: (req, file, cb) => cb(null, 'uploads/'),
  filename: (req, file, cb) => {
    // NEVER trust file.originalname — path traversal risk
    const ext = path.extname(file.originalname).toLowerCase();
    cb(null, `${crypto.randomUUID()}${ext}`);
  },
});

// ── Memory storage (for cloud upload, use sparingly) ──────────────
// Stores the file as a Buffer in req.file.buffer — no disk write
// ONLY use if immediately streaming to cloud (S3, etc.)
// Never use for large files — exhausts Node's heap memory
const memoryStorage = multer.memoryStorage();

// ── Pre-configured upload instances ───────────────────────────────
exports.uploadAvatar = multer({
  storage: diskStorage,
  fileFilter: filterByType(MIME_TYPES.image),
  limits: {
    fileSize:  5 * 1024 * 1024, // 5 MB
    files:     1,
  },
});

exports.uploadDocuments = multer({
  storage: diskStorage,
  fileFilter: filterByType(MIME_TYPES.document),
  limits: { fileSize: 20 * 1024 * 1024 }, // 20 MB
});

Route Handlers

const { uploadAvatar, uploadDocuments } = require('../middleware/upload');

// Single file — req.file
router.post('/avatar',
  uploadAvatar.single('avatar'),  // 'avatar' = the form field name
  async (req, res) => {
    if (!req.file) throw new BadRequestError('No file provided');

    // req.file contains:
    // fieldname, originalname, encoding, mimetype,
    // destination, filename, path, size
    const url = `/uploads/${req.file.filename}`;
    await db.user.update({ where: { id: req.user.id }, data: { avatarUrl: url } });
    res.json({ data: { url, size: req.file.size } });
  }
);

// Multiple files, same field — req.files (array)
router.post('/gallery',
  uploadDocuments.array('images', 10),  // max 10 files
  async (req, res) => {
    const urls = req.files.map(f => `/uploads/${f.filename}`);
    res.json({ data: { urls, count: urls.length } });
  }
);

// Multiple fields, different names — req.files (object)
router.post('/product',
  uploadAvatar.fields([
    { name: 'thumbnail', maxCount: 1 },
    { name: 'gallery',   maxCount: 8 },
  ]),
  async (req, res) => {
    const thumbnail = req.files.thumbnail?.[0];
    const gallery   = req.files.gallery ?? [];
    res.json({ thumbnail: thumbnail?.filename, gallery: gallery.map(f => f.filename) });
  }
);

Storage Options

Local Disk

  • ✅ Zero extra setup — good for dev
  • ✅ Fast reads (same machine)
  • ❌ Files lost if server is destroyed
  • ❌ Doesn't scale horizontally
  • ❌ Requires manual backup
  • ❌ Can fill your server's disk

Use for: development only

AWS S3 / Cloudflare R2

  • ✅ Durable (11 nines) + replicated
  • ✅ CDN-ready for global delivery
  • ✅ Scales infinitely
  • ✅ Pre-signed URLs for private files
  • ❌ Latency vs local disk
  • ❌ Costs (R2 cheaper than S3)

Use for: all production systems

Cloud Upload: AWS S3 with Pre-signed URLs

npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner
// src/services/storage.js
const { S3Client, PutObjectCommand, GetObjectCommand, DeleteObjectCommand }
  = require('@aws-sdk/client-s3');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');
const crypto = require('crypto');

const s3 = new S3Client({ region: process.env.AWS_REGION });

// Upload a file Buffer directly to S3
exports.uploadFile = async (buffer, mimetype, folder = 'uploads') => {
  const key = `${folder}/${crypto.randomUUID()}`;
  await s3.send(new PutObjectCommand({
    Bucket:      process.env.S3_BUCKET,
    Key:         key,
    Body:        buffer,
    ContentType: mimetype,
  }));
  return key; // store the key in DB, not the full URL
};

// Generate a time-limited signed URL for private file access
// Use this instead of making files public — safer by default
exports.getSignedUrl = async (key, expiresIn = 3600) => {
  return getSignedUrl(
    s3,
    new GetObjectCommand({ Bucket: process.env.S3_BUCKET, Key: key }),
    { expiresIn } // seconds — 1 hour default
  );
};

// Delete a file from S3
exports.deleteFile = async (key) => {
  await s3.send(new DeleteObjectCommand({
    Bucket: process.env.S3_BUCKET, Key: key,
  }));
};
// Route: upload to memory then stream to S3 (no disk write)
const storage = require('../services/storage');

router.post('/avatar',
  uploadToMemory.single('avatar'), // multer with memoryStorage
  async (req, res) => {
    if (!req.file) throw new BadRequestError('No file provided');

    const key = await storage.uploadFile(
      req.file.buffer,
      req.file.mimetype,
      'avatars'
    );

    // Store the key in DB, not the full URL (URLs change, keys don't)
    await db.user.update({ where: { id: req.user.id }, data: { avatarKey: key } });
    res.json({ data: { key } });
  }
);

Security Checklist — Non-Negotiable

  • Limit file size — always set limits.fileSize. Without it, a 10GB upload exhausts your server's disk or memory.
  • Validate MIME type — use fileFilter to whitelist allowed types. Note: MIME type comes from the client and can be spoofed.
  • Check magic bytes — for real security, read the first bytes of the file and check the file-type package. MIME type alone isn't enough.
  • Randomise filenames — never use file.originalname. An attacker sends ../../.env as the filename to overwrite your env file.
  • Store outside web root — files in your web root can be directly accessed by URL. Serve files through your API, not directly.
  • Authenticate uploads — don't allow unauthenticated file uploads. Rate-limit them separately since they're more expensive than regular requests.
  • Use pre-signed URLs for private files — don't make S3 buckets public. Generate time-limited signed URLs when a user requests their file.
  • Strip EXIF metadata — images contain GPS coordinates, device info, even thumbnails. Use sharp to strip metadata: await sharp(buffer).rotate().toBuffer().
  • Scan for malware — integrate ClamAV or a cloud scanning service for user-uploaded documents. A PDF can contain JavaScript.
  • Delete orphaned files — if a DB transaction fails after a file upload, delete the file to prevent orphans accumulating on disk/S3.

Image Processing with Sharp

npm install sharp
const sharp = require('sharp');

const processAvatar = async (buffer) => {
  return sharp(buffer)
    .rotate()              // auto-rotate based on EXIF orientation
    .resize(400, 400, {    // resize to 400×400
      fit:      'cover',   // crop to fill — no distortion
      position: 'center',
    })
    .webp({ quality: 80 }) // convert to WebP — smaller than JPEG
    .withMetadata(false)   // strip ALL metadata including GPS
    .toBuffer();
};
Phase 2 Capstone: Task Management API
Build a complete REST API with: full CRUD for tasks and projects, JWT authentication, Zod validation on all endpoints, centralised error handling with custom error classes, file attachment uploads (Multer → local disk for now), structured middleware stack (helmet, cors, rate-limit, morgan), and a /health endpoint. No database yet — in-memory is fine. Phase 3 adds the database.

🧠 Check Your Understanding


Go Deeper

Primary source: Multer documentation — covers all storage engines and options.

Security: OWASP — Unrestricted File Upload — real attack examples.

Image processing: sharp.pixelplumbing.com — the fastest Node.js image processing library.

Ask your teacher: "Walk me through the complete file lifecycle: upload, store, serve securely, delete." or "How do I implement direct browser-to-S3 uploads to avoid large files hitting my server?"