← warin.me · Data Science and Engineeringภาษาไทย

ML DATA ENGINEERING · FEATURE PLATFORM

Feature engineering is expensive. Rebuilding the same truth is even more expensive.

A useful feature is not merely a formula in a notebook. It is a time-aware, tested and reproducible data product that must behave consistently during training and prediction. A feature store provides the system around that promise.

01

Design

Turn model intent into explicit feature definitions.

02

Reproduce

Reconstruct historical values without future leakage.

03

Serve

Deliver the same meaning for batch training and online inference.

04

Govern

Track owners, lineage, freshness, versions and quality.

01 · THE ECONOMICS

The formula may be cheap. The lifecycle is not.

A ratio can take one line of SQL. Making it correct for five years of training data, available within milliseconds, monitored every day and understandable to another team is a different engineering problem.

DISCOVERY

Finding signal

Domain interviews, profiling, experiments and many features that never reach production.

COMPUTE

Historical recomputation

Large joins, rolling windows, backfills and repeated training datasets consume compute and engineering time.

CORRECTNESS

Time-aware truth

Point-in-time joins must use only information available when each prediction would have occurred.

OPERATIONS

Keeping it alive

Freshness, failed pipelines, schema changes, skew, drift, latency, incident response and ownership remain after launch.

DUPLICATION

Parallel definitions

Different teams rebuild “customer activity” with different windows, filters and missing-value policies.

RISK

Silent model damage

Leakage or training-serving skew may improve an offline score while weakening real predictions.

TOTAL FEATURE COSTdefinition + history + compute + serving + quality + coordination + change

02 · FROM COLUMN TO DATA PRODUCT

A feature needs a contract, not only a name.

“orders_30d” is incomplete unless we know the entity, event time, inclusion rules, null policy, freshness, owner and version.

ENTITYcustomer_id
FEATUREcompleted_orders_30d
EVENT TIMEas_of_timestamp
WINDOW[t − 30 days, t)
DEFAULT0, not NULL
OWNERCustomer ML Team
A small symbol with a large consequence

The interval ends at t but excludes t. This prevents an event occurring at prediction time from entering a historical training row before it would have been known.

03 · WHAT A FEATURE STORE DOES

It coordinates definitions, computation and delivery.

A feature store is not simply a database with a fashionable name. It is an ML data system that connects a registry, historical retrieval and—in low-latency use cases—online serving.

SOURCESorders · events · profiles
TRANSFORMbatch / stream pipelines
OFFLINE STOREhistorical training features
ONLINE STORElatest low-latency values
CONSUMERStraining · batch scoring · API
REGISTRY / CONTROL PLANEdefinition · schema · owner · lineage · version

04 · OFFLINE AND ONLINE ARE DIFFERENT JOBS

One definition may need two physical paths.

Training wants large, correct historical datasets. Real-time inference may want the latest value in milliseconds. The store coordinates meaning; it does not erase the physical trade-off.

Offline store

Optimized for scans, history, point-in-time retrieval, backfills and training-set generation. Often built on a warehouse or lakehouse.

Online store

Optimized for keyed lookup and predictable low latency. Usually holds the latest feature values, not all history.

Feature registry

Describes feature identity and ownership. It helps discovery and reuse but cannot guarantee that a poor definition becomes meaningful.

05 · LAB: DEFINE HISTORICAL FEATURES

Build features as of each prediction time.

This simplified PostgreSQL-compatible example teaches the principle. Production systems need tuned storage, incremental computation and tested orchestration.

CREATE TABLE ml.training_events (
  customer_id INTEGER NOT NULL,
  prediction_time TIMESTAMP NOT NULL,
  label INTEGER
);

-- One row represents one customer at one prediction time.
CREATE VIEW ml.v_customer_features_training AS
SELECT
  t.customer_id,
  t.prediction_time,
  t.label,
  COUNT(o.order_id) FILTER (
    WHERE o.status = 'completed'
  ) AS completed_orders_30d,
  COALESCE(SUM(o.amount) FILTER (
    WHERE o.status = 'completed'
  ), 0) AS completed_value_30d,
  MAX(o.ordered_at) AS latest_known_order_time
FROM ml.training_events AS t
LEFT JOIN raw.orders AS o
  ON o.customer_id = t.customer_id
 AND o.ordered_at >= t.prediction_time - INTERVAL '30 days'
 AND o.ordered_at <  t.prediction_time
GROUP BY t.customer_id, t.prediction_time, t.label;
Point-in-time ruleEvery joined record must have event time earlier than prediction_time. Ingestion time may also matter when events arrive late.

06 · LAB: MATERIALIZE THE LATEST VALUES

Prepare an online-shaped table without pretending it is an online platform.

The primary key models one latest feature vector per entity. A production online store also needs atomic writes, freshness guarantees, availability, latency SLOs and recovery.

CREATE TABLE ml.customer_features_online (
  customer_id INTEGER PRIMARY KEY,
  completed_orders_30d INTEGER NOT NULL,
  completed_value_30d DECIMAL(14,2) NOT NULL,
  feature_timestamp TIMESTAMP NOT NULL,
  feature_version VARCHAR(20) NOT NULL
);

INSERT INTO ml.customer_features_online
SELECT
  c.customer_id,
  COUNT(o.order_id) FILTER (WHERE o.status='completed'),
  COALESCE(SUM(o.amount) FILTER (WHERE o.status='completed'), 0),
  CURRENT_TIMESTAMP,
  'customer_activity_v1'
FROM raw.customers c
LEFT JOIN raw.orders o
  ON o.customer_id=c.customer_id
 AND o.ordered_at >= CURRENT_TIMESTAMP - INTERVAL '30 days'
GROUP BY c.customer_id
ON CONFLICT (customer_id) DO UPDATE SET
  completed_orders_30d=EXCLUDED.completed_orders_30d,
  completed_value_30d=EXCLUDED.completed_value_30d,
  feature_timestamp=EXCLUDED.feature_timestamp,
  feature_version=EXCLUDED.feature_version;

07 · WHEN A FEATURE STORE EARNS ITS COST

Do not build a platform before the coordination problem exists.

A feature store has its own platform cost. It becomes worthwhile when reuse, time correctness, serving consistency and governance save more than the system costs to operate.

STRONG SIGNALS
  • Several models reuse the same entities and features.
  • Historical reconstruction and leakage are recurring problems.
  • Online prediction needs low-latency, fresh features.
  • Teams disagree about definitions or rebuild pipelines.
  • Ownership, lineage and compliance need a shared control plane.
MAY BE TOO EARLY
  • One model, one team and a small batch dataset.
  • Features are used once and change rapidly during exploration.
  • A tested warehouse table or dbt model already satisfies the contract.
  • There is no operational owner for another platform.
  • The real bottleneck is label quality, not feature delivery.

08 · OPERATING CHECKLIST

Reuse is safe only when meaning remains visible.

A feature platform should reduce hidden work, not hide the work so completely that nobody can challenge its assumptions.

01

Point-in-time correctness

Test that historical rows never read future events.

02

Training-serving parity

Compare offline and online values for sampled entities.

03

Freshness & latency

Measure feature age and retrieval latency against explicit SLOs.

04

Quality & drift

Watch null rate, ranges, distribution shifts and entity coverage.

05

Version & lineage

Trace code, source, window, owner and models using each version.

06

Cost & retirement

Measure compute and storage, then retire unused features safely.

CHALLENGES

Design the system, then defend the trade-off.

01

Find the leakage bug in a historical join that uses CURRENT_TIMESTAMP instead of prediction_time.

02

Define “customer spend in 7 days” including entity, window boundaries, event time, default and owner.

03

Propose an offline-to-online validation query and an acceptable mismatch threshold.

04

Design a backfill after the cancelled-order rule changes. Which models and datasets are affected?

05

Estimate monthly cost for computing 200 features for 20 million entities. State every assumption.

06

Argue against adopting a feature store for a small batch project, then state the condition that would change your decision.

THE CENTRAL IDEA

A feature store does not make feature engineering cheap. It makes the cost visible, reusable and governable.