Will institutional data be used or processed by any shared AI services?
Explanation
Shared AI services and institutional data intersect here, and the question is whether data belonging to the institution will be used or processed by any shared AI service. 'Shared AI services' typically refers to third-party AI platforms like OpenAI's ChatGPT, Google's Vertex AI, Microsoft Azure AI, or similar services where the AI models are hosted and maintained by another company.
The question is important for security assessments because when institutional data is processed by external AI services, there are several risks:
- Data privacy concerns: Institutional data may contain sensitive or personally identifiable information (PII) that requires protection
- Data retention policies: External AI services might store or learn from submitted data
- Regulatory compliance: Educational institutions must comply with regulations like FERPA, GDPR, or HIPAA
- Intellectual property concerns: Research data or proprietary information could be exposed
The guidance specifically asks you to detail:
- What types of data will be processed by the AI service
- Where this data comes from (the sources)
- Whether any data shared with the AI comes from outside the institution
To best answer this question, you should be transparent about any use of external AI services, specify exactly what institutional data might be processed by these services, explain your data handling practices, and describe any safeguards in place to protect sensitive information. If you don't use shared AI services with institutional data, clearly state that.
Guidance
Provide detailed response to the type of data needed for the AI service to function appropriately, the sources of the data, and whether any data shared with the AI service comes from data sources outside the institution.
Example Responses
Example Response 1
Yes, our learning management system uses Microsoft Azure OpenAI Service to provide automated essay feedback and plagiarism detection The system processes student essay submissions and assignment text The data sources are limited to the content students upload to our platform and the assignment descriptions created by instructors No data from sources outside the institution is used in this AI processing We have implemented the following safeguards: 1) All data is encrypted in transit and at rest, 2) We use Azure's private endpoints to ensure data doesn't traverse the public internet, 3) We have a data processing agreement with Microsoft that prohibits the use of institutional data for training their models, and 4) All processed data is deleted within 30 days of the end of the academic term.
Example Response 2
No, our solution does not use any shared AI services to process institutional data While our product does incorporate AI capabilities for predictive analytics on student performance, these AI models are proprietary and run entirely within our application's infrastructure that is deployed in the institution's environment All data processing occurs within the boundaries of the institution's network, and no institutional data is sent to external AI services for processing, analysis, or model training Our AI functionality is pre-trained on synthetic data and anonymized datasets that do not contain any institutional data from your or any other institution.
Example Response 3
Our product currently does not use shared AI services for processing institutional data, but we are planning to implement a feature in Q3 2023 that will use Google Cloud's Natural Language API to analyze sentiment in student feedback surveys This implementation will process de-identified student comments (with no personally identifiable information) to generate aggregate sentiment scores for courses We cannot fully meet this requirement at present because our planned implementation will send anonymized institutional data to a third-party AI service However, we are designing this feature with privacy by design principles, including data minimization and purpose limitation We would be happy to discuss our implementation plans and potential mitigations to address any concerns before this feature is deployed.
Context
- Tab
- Privacy
- Category
- Privacy and AI
Related questions
- Does your service use AI for the processing of institutional data?
- Is any institutional data retained in AI processing?
- Do you have agreements in place with third parties or subprocessors regarding the protection of customer data and use of AI?
- Will institutional data be processed through a third party or subprocessor that also uses AI?
- Is AI processing limited to fully licensed commercial enterprise AI services?
- Do you have safeguards in place to protect institutional data and data privacy from unintended AI queries or processing?
