Timestamps and Time Zones in Delivered Datasets: ISO 8601 and Event Ordering

Why timestamps are a common failure point in event datasets

Timestamps break event datasets because they look self-explanatory while hiding four separate decisions: encoding, time scale, zone semantics and precision. A support ticket export from one system, a CRM activity log from another and a build log from a CI runner can each be internally consistent and still disagree by hours once merged. For the agent-training, process-model and evaluation work these logs feed, the damage is quiet: the sequence "ticket reopened, then escalated" silently becomes "escalated, then reopened."

The common failure modes are well known to anyone who has received logs from enterprise systems:

• Naive local strings, such as 2026-03-08 02:30:00, with no offset. That wall-clock time does not exist in US Eastern on the spring DST transition date, and on the fall transition the 1:00-2:00 hour occurs twice.
• Unit ambiguity in epoch integers. 1772000000 is seconds; 1772000000000 is milliseconds; a nanosecond epoch exceeds the 2^53 safe-integer range of JavaScript and JSON parsers that decode numbers as doubles, so it loses precision silently.
• Mixed precision: one source truncates to seconds while another keeps microseconds, so events from the coarse source cluster at .000 and sort ahead of finer ones.
• Zone relabeling: an export tool writes local server time but appends Z, which is worse than no offset at all because it looks correct.

These are the issues the rest of this page addresses, field by field. For the broader delivery spec these rules slot into, see the dataset delivery formats and schemas hub.

ISO 8601 vs Unix timestamps: which to require

Require RFC 3339 strings for text formats such as JSONL and CSV, and typed timestamp columns for columnar formats such as Parquet and Arrow. RFC 3339 is a profile of ISO 8601 for Internet timestamps; it names a single instant relative to UTC and sidesteps local daylight-saving questions. Full ISO 8601 allows many variants (week dates, ordinal dates, basic format without separators, reduced precision), so "ISO 8601" alone is too loose to put in a contract.

Epoch integers are compact and sort trivially, but they carry no self-describing unit and no zone, and Unix time does not count leap seconds. If a supplier's systems emit epoch values, accept them only with a declared unit in the data dictionary and a column name that encodes it, such as event_ts_epoch_ms.

RFC 3339 has been updated by RFC 9557, which defines a bracketed suffix for a time zone name (for example 2026-03-09T14:05:00-04:00[America/New_York]). As of October 2026, many warehouse loaders and parsers do not yet accept that suffix, so a separate zone column is the more portable choice.

Real exports show what "good" looks like: Amazon Connect returns chat transcript items with AbsoluteTime as an ISO 8601 string with milliseconds and a Z suffix [3]. When a supplier's source system already emits that shape, ask them to pass it through unchanged rather than reformatting it. The conversation transcript delivery schema covers the rest of that record.

Parquet timestamp time zone semantics

Parquet distinguishes instants from local date-times with a single flag, and your spec should name the value you expect. The normative format specification, including logical type definitions, lives in the separate parquet-format repository rather than in reader or writer libraries [1]. The TIMESTAMP logical type carries isAdjustedToUTC and a time unit of MILLIS, MICROS or NANOS. With the flag set to true, the stored value is an instant normalized to UTC; with it false, the value is a local date-time with no zone attached. The older ConvertedType annotations could not express these parameters, and legacy INT96 timestamps written by older Hive, Impala and Spark versions carry no explicit semantics at all. Because file metadata is written in the footer after the data [2], a reviewer can inspect each column's timestamp annotation with parquet-tools, PyArrow's pq.read_schema or DuckDB's parquet_schema() without scanning rows.

Engines map the flag to their own types, and that mapping is where deliveries go wrong:

• Spark reads UTC-adjusted columns as TIMESTAMP (session-zone rendered) and non-adjusted columns as TIMESTAMP_NTZ in recent versions.
• BigQuery distinguishes TIMESTAMP (instant) from DATETIME (civil time); Snowflake has TIMESTAMP_NTZ, TIMESTAMP_LTZ and TIMESTAMP_TZ.
• PostgreSQL timestamptz stores the instant but discards the original offset, so a supplier exporting from it cannot recover local time unless a zone column exists.
• pandas with datetime64[ns] cannot represent dates outside roughly 1677-2262, which breaks sentinel values such as 9999-12-31 in "valid to" columns.

The practical rule: event times are TIMESTAMP(isAdjustedToUTC=true, MICROS) unless the dictionary justifies otherwise, and any civil-time field (a scheduled appointment, a business-hours SLA clock) is declared as non-adjusted with a companion zone column. For more on choosing between the two file families, see Parquet vs JSONL for licensed training data.

Store UTC, and keep the original offset when local time matters

Store the canonical instant in UTC and carry local context in separate fields, because an offset alone is not a time zone. -05:00 could be US Eastern in winter or US Central in summer; only an IANA name such as America/Chicago lets you recompute local time, business-day boundaries and DST effects. Because the tz database itself changes as governments alter rules, record the tzdata version the supplier used when conversions were done.

Local context matters more often than teams expect. Agent-training data built from support queues needs it to model after-hours behavior; SLA breach labels depend on the customer's business calendar; shift-based workflows in field service and industrial logs depend on plant-local time. When you request enterprise workflow and task histories, say which of these questions you intend to ask so the supplier knows which zone fields to preserve.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "event_id": "evt_000184",
  "case_id": "case_7731",
  "activity": "ticket_reopened",
  "event_time_utc": "2026-03-09T18:05:12.481Z",
  "event_time_local_offset": "-05:00",
  "event_tz": "America/Chicago",
  "source_system": "helpdesk",
  "source_seq": 55120931,
  "time_precision": "ms",
  "time_kind": "event",
  "ingest_time_utc": "2026-03-09T18:05:14.020Z"
}

Note the separation of event_time_utc (when it happened) from ingest_time_utc (when the logging pipeline recorded it). Many exports only contain one of these and do not say which.

Event ordering when timestamps collide

When two events share a timestamp, order them by an explicit tie-breaker the supplier provides, never by file position or by a sort your engine happens to produce. Collisions are common: coarse precision, batch jobs that stamp many rows at once, and automated rules that fire within the same millisecond as the human action that triggered them. Unstable sorts in Spark, pandas and SQL engines do not guarantee the same order twice.

Ask the supplier for a total ordering key per source, such as (event_time_utc, source_system, source_seq, event_id), where source_seq is a monotonically increasing number from the originating system: a database log sequence, an audit-table identity column, or a per-case counter. Clock skew between systems means cross-source order is approximate even with good timestamps; the honest answer is to keep order strict within a source and treat cross-source order within a skew tolerance as uncertain.

Process-mining formats already encode these concerns. XES (IEEE 1849-2023) is an XML format for event logs and streams whose extensions give attributes such as timestamps defined meaning [5]. OCEL 2.0 defines an object-centric metamodel with SQLite, XML and JSON exchange formats, so one event can relate to several objects such as an order, an invoice and a shipment [4]. If your downstream tooling consumes either, request it directly; see process mining event logs for training and evaluating agents.

Time-correct splits and point-in-time joins

Timestamps also decide whether your evaluation is honest. In predictive process monitoring, random splits across cases leak information between training and test sets, and temporal splits are the standard fix [7]. That only works if every event has a trustworthy event time, not an export or ingest time.

The same applies when you join delivered logs with features or labels. Feature stores such as Feast treat an entity's event_timestamp as an inclusive upper bound, returning only feature values stamped at or before it [6]. Late-arriving corrections and backfilled rows need their own recorded_at field to avoid look-ahead; the guide to point-in-time correct training data covers as-of joins in detail.

De-identification and date shifting change intervals

De-identification can alter timestamps, and you need to know exactly how before you model durations. Timestamps contribute to the uniqueness of individual cases in event logs, which is a measured re-identification risk [8]. For health records, HIPAA's Safe Harbor method requires removing listed identifiers, including date elements more specific than the year, while Expert Determination can permit transformed dates under an expert's risk analysis [9].

Common transformations and their side effects:

• Per-subject consistent shift: preserves intervals within a case but breaks alignment across cases, day-of-week and seasonality.
• Generalization (to day, week or year): destroys intra-day ordering, so you need source_seq more than ever.
• Noise injection: can reorder close events and create negative durations.

Ask for the method, the shift granularity and whether ordering was preserved to be recorded in the dataset documentation. For datasets sourced through SourceX, personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect; health records require HIPAA de-identification by Safe Harbor or Expert Determination.

A timestamp acceptance checklist for event-log deliveries

Use this checklist in the data dictionary review and again on the first delivered sample.

Illustrative example: invented to show structure; it does not describe an available dataset.

Item 10 ties into handling schema changes across recurring deliveries; a supplier that switches from milliseconds to microseconds mid-series has made a breaking change even if column names stay the same.

How sourced event and workflow data fits this spec

SourceX sources operational datasets from US companies, including support and sales histories, engineering records and finance and legal workflows, on request rather than from stock, and a request does not guarantee a match. Every dataset is rights-reviewed and delivered under a license defining records, uses, term and delivery, and diligence materials covering source, rights, preparation and allowed use are prepared per dataset. That is the right point to attach a timestamp spec like the one above: describe the data and its time semantics when you submit a buyer request, and the requirement travels with the request.

Requesting event logs with a timestamp spec that holds

Describe the event or workflow data you need, including the timestamp, zone and ordering rules on this page, and SourceX looks for US businesses that hold it. Assessment covers the data and licensing permissions, nothing is contracted until a supplier agrees, and every release is approved by the supplying company. Start a buyer request at SourceX.