Audit Logging for Access to Licensed Datasets

Why access logs are the buyer's evidence, not the supplier's

Access logs are the only record in your environment that can show a licensor who touched their data, when, and for what purpose. Once a dataset leaves the supplier, the supplier cannot see your buckets, warehouse or GPU cluster; your logs are what you produce when a license asks you to demonstrate use within the permitted scope, or when your own counsel needs to answer "which jobs read this file?"

Logs answer a narrower question than lineage. They prove access events; they do not by themselves prove which model weights contain which records. Pair them with the job-to-model records described in tracing which models trained on which licensed dataset, and with the controls on who may read the data in the first place, covered in access controls for licensed training data after delivery.

License hygiene is a known weak point in the field. The Data Provenance Initiative's audit of more than 1,800 text datasets found license information omitted in over 70% of cases on popular hosting sites and error rates above 50% [4]. When the license itself is clear, a buyer who cannot show compliant access is in a weaker position than one who can.

Object-level logging is off by default on the major clouds

As of October 2026, the default audit configuration on AWS and Google Cloud records control-plane changes, not who read each object, so a licensed bucket with default settings produces no per-file read history. Plan the enablement and its cost before delivery lands.

• Amazon S3. CloudTrail trails and event data stores do not log data events by default, and data events are billed on top of management events; check current CloudTrail pricing. You can scope S3 data events to the specific buckets or prefixes holding licensed data rather than every bucket in the account. Data events never appear in the CloudTrail Event history console, so you read them from the trail's destination bucket or a CloudTrail Lake event data store. S3 server access logging is a separate, best-effort mechanism; CloudTrail data events are generally the stronger record because delivery is not best-effort and each event carries the full userIdentity context.
• Google Cloud Storage. Admin Activity audit logs are always on, but Data Access audit logs (DATA_READ, DATA_WRITE) are disabled by default and are not written until you enable them. Enable them through the project or folder IAM policy auditConfigs for storage.googleapis.com; note that a principal with only the basic Logs Viewer role cannot read Data Access entries.
• Azure Blob Storage. Read events similarly require a diagnostic setting that sends StorageRead logs to a Log Analytics workspace or storage account; verify the current category names against Microsoft's documentation before you rely on them. Confirm every default above against the provider's current documentation before publishing internal standards.
• Snowflake. SNOWFLAKE.ACCOUNT_USAGE.ACCESS_HISTORY records which objects and columns each query read or wrote [1], which is more granular than object storage logs. When data arrives through Secure Data Sharing, nothing is copied and the consumer's view is read-only [2], so your query-side history is the access record.

Volume is the main operational surprise. A busy bucket can generate thousands of data-event records in minutes, and DATA_READ on high-traffic Google Cloud resources raises Cloud Logging ingestion and storage costs in the same way. Scope logging to licensed prefixes and accept the cost; this is not a place to sample.

What each access record must contain

A useful record answers six questions: who, what, when, from where, under which job, and under which license. Native cloud logs cover the first four; the last two require your own tagging.

Illustrative example: invented to show structure; it does not describe an available dataset.

The two fields teams most often lose are job_id and license_id. Make training jobs assume a role with a session name or session tag that carries the run ID, so the identity in the log already points to the job. Keep a mapping table from bucket or prefix to license ID in your data catalog, and include the dataset version so a log line can be matched to the files listed in the delivery manifest and checksums.

Avoid shared identities. If every notebook reads through one long-lived access key, your logs show a single principal for every human and pipeline, which is close to useless to an auditor. Requiring authenticated requests on every path, as Requester Pays buckets do by design [5], is the baseline; named roles per job are the target.

Retention: cover the license term plus the audit window

Keep access logs at least as long as the license term plus any period after expiry during which the supplier may audit or raise a claim, and longer if the license or your own policy says so. Read the audit clause and the post-termination clause together; the retention period is the later of the two dates, not the term alone.

Default retention in most tools is far shorter than a multi-year license. CloudTrail's console Event history keeps only recent management events and excludes data events entirely, and Cloud Logging buckets have their own retention settings that you must extend or route to long-term storage. CI systems are similar: GitHub Actions deletes workflow logs and artifacts after 90 days by default, configurable up to 400 days on private repositories [3]. If your training jobs launch from CI, the job logs that tie a commit to a dataset read disappear long before most licenses end unless you export them.

Illustrative example: invented to show structure; it does not describe an available dataset.

Make the log store write-once and separate from the data

Store audit logs in a separate account or project, with write-once retention and no delete rights for the teams whose access is being logged. If the same administrators who run training can edit or expire the logs, the record proves little to a supplier or auditor.

Practical controls:

• Separate account or project. Deliver CloudTrail to a log-archive account and GCS audit logs to a sink in a dedicated logging project, so a compromised training account cannot rewrite history.
• Write-once retention. Use S3 Object Lock in compliance mode or a locked Cloud Storage retention policy on the archive bucket, with the retention period set from the license table above. Confirm the mode in your provider's current documentation, because governance-style modes can be bypassed by privileged users.
• Integrity checks. Turn on CloudTrail log file integrity validation, or hash daily exports and store the digests in a second location, so you can show a log file has not changed since it was written.
• Encryption and key custody. Encrypt the archive with a customer-managed key whose deletion is separately controlled; a deleted key is an effective deletion of the logs. See encrypting dataset deliveries for key handling patterns.
• Restricted readers. Grant read access to the security and legal teams that produce audit reports, not to data scientists.

One tension to settle early: if the license requires deletion of the data at term end, deletion of the dataset must not delete the logs. Keep logs that prove access and later deletion, and record destruction with a certificate of data destruction.

Turning raw logs into a supplier audit report

A supplier rarely wants raw CloudTrail JSON; they want a short report that answers whether use stayed within the license. Build the query once, at delivery, and rerun it on request.

Illustrative example: invented to show structure; it does not describe an available dataset.

Dataset access summary, license LIC-0142-v1, period 2026-07-01 to 2026-09-30

1. Licensed locations: s3://licensed-ds-0142/ (versions v2, v3); Snowflake share LIC0142_DB.
2. Distinct principals with read access: 4 roles, 0 human users with direct read.
3. Read events by principal and job: table of role, job ID, object count, first and last read.
4. Jobs and resulting artifacts: job ID to model checkpoint IDs, from the lineage register.
5. Reads from outside approved networks or regions: none, or listed with the incident ticket.
6. Gaps: periods when logging was disabled or delivery failed, with explanation.
7. Integrity: digest validation result for the archive covering this period.
8. Sign-off: security engineer and data owner, dated.

Two failure modes damage these reports. The first is a silent gap: a Terraform change drops the data-event selector, and nobody notices until an audit. Alert on changes to trail selectors, audit configs and diagnostic settings, and run a daily canary read of a test object that must appear in the logs. The second is copying data out of the logged location, such as a researcher syncing files to a local disk or an unlogged scratch bucket; once copied, later reads are invisible. Block egress paths through bucket policies and VPC endpoint conditions, and log the copies you do allow.

Fitting logs into the wider evidence pack

Access logs are one exhibit in a larger evidence pack that also includes the license, the delivery manifest, preparation notes and lineage records. The AI training data audit readiness guide covers that pack, and SourceX's notes on keeping a record of what you licensed and whether you can audit how data is used cover the record-keeping side.

When sourcing through SourceX, each dataset is rights-reviewed for ownership and consents and delivered under a license that defines the records, uses, term and delivery, and diligence materials on source, rights, preparation and allowed use are prepared per dataset. Delivery runs through private, access-controlled workflows after an executed agreement and supplier approval, never as email attachments. That gives your logging design fixed inputs: a defined term, defined uses and a known delivery location to instrument. For the delivery patterns themselves, start at the delivery and schemas hub or the cross-account guide on delivering licensed datasets to your cloud bucket. Buyers planning a request can describe the data they need at SourceX for buyers.

Setup checklist before the first delivery lands

Illustrative example: invented to show structure; it does not describe an available dataset.

• Licensed bucket or share created in an account dedicated to licensed data.
• Object-level read logging enabled (CloudTrail data events, GCS DATA_READ, Azure StorageRead, Snowflake access history export).
• Logs routed to a separate archive account or project with write-once retention set to license term plus audit window.
• Integrity validation or digest export enabled.
• Training roles carry job IDs in session names or tags; no shared access keys.
• Prefix-to-license-ID mapping recorded in the catalog with dataset version.
• Alerts on logging configuration changes; daily canary read verified.
• Saved report query producing the supplier summary above.
• Deletion runbook that removes data but preserves logs.

Source licensed data you can audit

SourceX sources operational datasets from US companies on request and manages the licensing process from Find and Assess through Agree, Transact and Manage, with every release approved by the supplying company. Nothing is contracted until a supplier agrees, and a request does not guarantee a match. Describe the data you need and your delivery and audit requirements at SourceX for buyers.