A few weeks ago, a company called Snowflake had the largest software IPO in history. Everyone in the data sector has been using it or hearing about it for a while, but now it has caught the attention of CFOs.

BigQuery charges you per Terabyte queried. If someone runs a poorly filtered SELECT * on a historical log table, the cost can be several thousand dollars in five seconds. This forces you to evaluate Snowflake.

The revolution of separating storage and compute

Both BigQuery and Snowflake are massively parallel processing (MPP) databases, but their philosophy is different. BigQuery is "serverless" magic: you run the query, and Google decides how many thousands of CPUs to use. You have no control.

Snowflake's technical genius is having absolutely and physically separated storage from processing (compute). The data sits inert and cheap in Amazon S3 or Google Cloud Storage. On top of that, you can create "Virtual Warehouses," which are nothing more than temporary compute clusters.

This solves the classic concurrency problem. I can have an "X-Large" cluster spun up just for the heavy nightly ETL process, and a "Small" cluster for the business analysts' reports, both hitting the same data without blocking each other.

-- In Snowflake, compute is a resource you turn on and off
CREATE WAREHOUSE Marketing_WH 
WITH 
  WAREHOUSE_SIZE = 'SMALL' 
  AUTO_SUSPEND = 300 -- Shuts down automatically after 5 min of inactivity
  AUTO_RESUME = TRUE;

-- We use this specific "engine" for our query
USE WAREHOUSE Marketing_WH;
SELECT * FROM Ventas_Historicas WHERE region = 'EMEA';

Reflection: FinOps and spending control

Technically, BigQuery's dynamic network routing seems more advanced and impressive than Snowflake's cluster architecture. However, Snowflake won the corporate battle because it solves a human problem: the fear of the bill.

Knowing that a Small size Warehouse costs you exactly 2 credits per hour, and that it automatically shuts down after 5 minutes of no use, gives you peace of mind. Data engineers are slowly becoming financial engineers (FinOps). It's no longer enough for the SQL query to be fast; now we have to justify whether the business value of that answer outweighs the CPU cents it cost to run.