MES and historian data: what production records to keep when replacing a system

Event records and raw sensor history are different archives#

MES event records and historian sensor data need different archive rules because they differ in size, shape and meaning. Event records describe what happened in production: an order started, a lot was consumed, a line went down, a check failed. Historian data is a time series of measurements from tags such as temperatures, pressures, speeds and flows.

The common planning error is treating both as one block of data. Event records are cheap to keep whole; raw sensor history needs a deliberate, size-aware decision, tag by tag.

The keep list for MES event records#

Keep every MES event record that connects a product to how it was made. Customers, auditors and engineers ask about exactly these records, and they are rarely large.

Export event records with their native IDs and timestamps, and note whether the MES stored local time or UTC. Daylight saving shifts and mixed time conventions are a common reason lot genealogy and historian values stop lining up once the old system is gone.

• 1Production orders with planned and actual quantities, start and end times.
• 2Lot and batch genealogy: which material lots went into which finished lots.
• 3Routing steps executed, with equipment, operator and timestamps.
• 4Downtime events with reason codes and operator comments.
• 5In-process quality checks, results, holds and dispositions.
• 6Recipe or parameter versions used for each order.
• 7Scrap, rework and yield records by step.
• 8Electronic signatures and approvals, where the MES records them.

A size-aware rule for historian tags#

The size-aware rule for historian data is to tier tags by what they explain, not by how much disk they use. Start from the tag list in the historian's configuration and classify every tag before setting retention.

Write the tier decision down tag by tag, with the name of the person who approved it. A dropped tag cannot be recovered, and a signed list shows later that the choice was deliberate rather than an accident of the migration.

Event windows are what make Tier 2 useful. When the MES records a downtime or quality event, keep the raw values for related tags in a window around it, sized to how fast that process moves, and keep summaries everywhere else. Size the result from the plant's real tag counts and sample rates before committing to storage.

Compression settings matter because historian values are usually not every reading a sensor produced. Most historians store values by exception and compression rules, so record those settings with the archive. Otherwise a later analyst may treat a flat line as a stable process when it was really a deadband.

Context that makes old production data readable#

Context is what turns archived production data from numbers into evidence. A tag value means little unless you know the equipment it measured, its units, its normal range and what changed on the line around it.

Archive the tag dictionary, units and engineering ranges, the equipment hierarchy, instrument replacements and calibration dates, line modifications, product and recipe codes, and shift calendars. For a PI System or a similar historian, that means the tag configuration and asset model as well as the archive files that hold the values.

For the MES side, the equivalent is master data: product codes, equipment lists, reason code tables and status definitions. Export them in the same pass as the event records, because a reason code that cannot be decoded is just a number.

Illustrative: a coatings plant retiring its MES#

Illustrative: a fictional industrial coatings manufacturer is replacing a home-grown MES and an aging historian with a new platform. The integrator plans to migrate open orders, master data and recent history only.

The plant operations lead runs the keep list. All lot genealogy, batch records, downtime events and quality holds are exported in full with product and equipment codes. Historian tags are tiered: viscosity, temperature and mixer load tags on the batch reactors stay at full resolution, utility tags are summarized by shift, and redundant tags are dropped by signed decision.

The archive stays in the company's own storage. When a customer complaint arrives about an old lot, quality traces the lot to its raw materials and pulls the reactor temperature profile for that batch.

Regulated records and customer requirements come first#

Regulatory rules and customer contracts can set their own requirements for production records, so the size-aware rule sits underneath them, not above them. Food, drug, medical device and aerospace manufacturers in particular may hold batch, device history or traceability records with defined retention expectations.

Customer quality agreements can also require you to keep inspection results, lot traceability or process data for a stated period. Check those obligations with your quality team and counsel before any record family is summarized or dropped.

Where such an obligation applies, keep the records in a form that preserves their integrity: original values, audit trails and electronic signatures, not a re-keyed copy. A cleaned-up extract that drops the audit trail may fall short of the requirement even when every value is present.

How SourceX approaches production records#

SourceX approaches MES and historian archives as physical-world operating records: what a plant planned, what the equipment did and how people responded. Event records with operator comments and quality dispositions carry the human decisions; sensor history supplies the physical context around them.

Large historian archives stay in the manufacturer's own storage or ship on encrypted drives, because SourceX never hosts multi-TB datasets. Under the SourceX Enterprise Data Value Framework, scale and recency raise value while preparation cost lowers net value, and any release through the SourceX five-step transaction needs the manufacturer's approval and a SourceX Evidence Packet.