AILM-02

Is your LLM training data vetted, validated, and verified before training the solution's AI model?

Explanation

This question is asking whether your organization has a robust process for ensuring the quality, accuracy, and appropriateness of the data used to train your Large Language Model (LLM) before it's incorporated into the training dataset. In the context of a security assessment, this question addresses several key risks: 1. Data Poisoning: Malicious data could be introduced to manipulate model outputs or create backdoors 2. Bias and Fairness: Unvetted data may contain biases that could lead to discriminatory outputs 3. Accuracy and Quality: Poor quality training data leads to unreliable model outputs 4. Compliance: Using certain types of data may violate regulations or intellectual property rights 5. Security Vulnerabilities: Unvetted data might contain patterns that could be exploited by attackers The guidance specifically asks about: - Policies and procedures for data validation - Validation checks on training data - Multiple data labelers to ensure accurate labeling To best answer this question, you should describe your organization's specific processes for: - Data sourcing and provenance tracking - Content filtering and moderation - Quality assurance procedures - Bias detection and mitigation - Data labeling verification methods - Any automated and human review processes If you don't have robust vetting processes, it's better to acknowledge this gap and describe any compensating controls or plans to address it rather than providing a misleading answer.

Guidance

Looking for policies/procedures for validating and verifying any data used to train the model through validation checks and employing multiple data labelers to validate the accuracy of the data labeling.

Example Responses

Example Response 1

Yes, our LLM training data undergoes a comprehensive vetting, validation, and verification process We have a documented Data Governance Policy specifically for AI training data that outlines our multi-stage approach First, all data sources are evaluated for provenance, licensing compliance, and content appropriateness We maintain a data provenance tracking system that documents the origin, ownership, and permissions for all datasets Second, our automated data validation pipeline checks for quality issues such as duplicate content, formatting problems, and potentially harmful content using both rule-based filters and ML-based detection systems Third, we employ a team of 15 trained data labelers who work with clear annotation guidelines, and we implement a consensus model where at least three labelers must agree on sensitive content classifications All labelers undergo regular calibration exercises to ensure consistency Finally, we conduct statistical analysis on our datasets to identify potential biases and implement mitigation strategies when necessary Our Data Ethics Committee reviews summary reports of all training datasets before they are approved for model training.

Example Response 2

Yes, our training data undergoes rigorous vetting and validation We source our data from carefully selected providers with clear data rights and have developed a proprietary data validation framework called DataGuard This framework includes automated checks for data quality (detecting duplicates, malformed entries, etc.) and content safety (identifying harmful, illegal, or biased content) For data labeling, we use a two-tier approach: initial labeling is performed by our AI-assisted labeling system, followed by human review from our team of data specialists We maintain a 20% overlap in human review assignments to measure inter-annotator agreement, which must exceed 85% before a dataset is approved Any disagreements are resolved through team lead review We also maintain detailed documentation of our data processing pipeline, including transformation steps and filtering criteria Before any dataset is used for training, it undergoes a final review by our AI Safety team who verify compliance with our responsible AI guidelines and check for potential bias or representation issues.

Example Response 3

No, we currently do not have a comprehensive vetting process for our LLM training data While we do perform basic automated filtering to remove obviously harmful content and duplicates, we lack a formal validation framework with documented procedures Our data labeling is primarily done through a single-pass review process without multiple validators or consensus mechanisms We recognize this as a gap in our security posture and are actively developing a more robust data governance framework In the interim, we mitigate potential risks by: 1) sourcing data only from established, reputable datasets with clear licensing terms, 2) conducting post-training evaluations to identify any problematic model behaviors, and 3) implementing strict content filtering on model outputs We expect to have a comprehensive data validation process implemented within the next quarter, including formal procedures for data vetting and a multi-annotator validation system.

Context

Tab: AI
Category: AI Large Language Model (LLM)