Skip to content

Question

What is RLHF data?

By SourceX Editorial · Updated

Short answer

RLHF data is the human input used in reinforcement learning from human feedback. It usually includes demonstrations (people writing ideal responses) and preference comparisons (people choosing the better of two model outputs). The preferences train a reward model, which then guides further model training. The method was described in research by Christiano et al. (2017) and applied to language models by Ouyang et al. (2022).

Part of: AI training data

Explanation#

RLHF data is produced against a model's own outputs, so it is usually commissioned rather than licensed from existing archives.

Existing business data can still support related steps, such as supervised fine-tuning with expert demonstrations.

Examples#

  • Prompt + two responses + which is better
  • Prompt + expert-written ideal response

Rights and privacy#

  • Annotator agreements
  • Confidential content in prompts

How licensing works#

A business describes its data, buyer demand is assessed, scope and permitted use are proposed, the data is prepared and de-identified, an agreement is signed, and the approved dataset is delivered. See data licensing for detail.

Relevant industries

Relevant data types

Related questions

Sources

  1. Christiano et al., Deep reinforcement learning from human preferences (2017)
  2. Ouyang et al., Training language models to follow instructions with human feedback (2022)

General information, not legal advice. Editorial policy.

Related resources

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies Talk to SourceX
See if your company qualifies