Yesterday we had a crash on a company's website. The server collapsed due to a lack of bandwidth and memory. The reason? The marketing team had decided to update the autumn catalog and dumped hundreds of 10 Megabyte RAW photos directly into the FTP folder that the system reads for the online store.

We are very used to dealing with numbers and text strings, but we treat images as second-class citizens. We simply save the .jpg path in the relational database and wash our hands of it. That is over. I've shoved image processing directly into the nightly data flow.

Automating graphic compression

The solution involves intercepting the files before they touch the web server, normalizing them, resizing them to three standard sizes (thumbnail, catalog, and detailed zoom) and compressing them for the web.

Although there are brutal command-line tools like ImageMagick (whose mogrify command can do it in a bash script), I preferred to integrate it into our Python pipeline using the PIL library (well, its friendly fork, Pillow).

import os
from PIL import Image

ruta_origen = '/datos/ftp/marketing/crudos/'
ruta_destino = '/var/www/ecommerce/imagenes/optimizadas/'

def optimizar_catalogo():
    for archivo in os.listdir(ruta_origen):
        if archivo.lower().endswith(('.jpg', '.jpeg', '.png')):
            ruta_completa = os.path.join(ruta_origen, archivo)

            try:
                # Abrimos la imagen original
                img = Image.open(ruta_completa)

                # Forzamos formato RGB (por si vienen CMYK de imprenta)
                if img.mode != 'RGB':
                    img = img.convert('RGB')

                # Redimensionamos manteniendo la proporción (máximo 800x800)
                img.thumbnail((800, 800), Image.ANTIALIAS)

                # Guardamos comprimido con calidad 85% progresivo
                nombre_salida = os.path.join(ruta_destino, "web_" + archivo)
                img.save(nombre_salida, 'JPEG', quality=85, progressive=True)

                print "Optimizada:", archivo
            except Exception as e:
                print "Error con", archivo, ":", str(e)

optimizar_catalogo()

This script is super dumb, but it reduced the homepage load by 90%. Marketing's FTP is still a disaster, but now our ETL acts as a sanitary filter.

Reflection: The image as pure data

Treating images programmatically has made me think. For now, to us an image is just a block of binary data (BLOB) that takes up disk space. But, this has to change.

We have to change the mindset: When we shove an image into the corporate ETL, we won't just resize it. We will run a model that will extract the predominant color, detect if the model in the photo is wearing glasses, or even automatically tag the clothing items, inserting those metadata directly into our database to improve searches. Images are about to become databases in themselves.