DPAI-06

Will institutional data be used or processed by any shared AI services?

Explanation

This question is asking whether your product or service will use any institutional data (data belonging to the educational institution) with shared AI services. 'Shared AI services' typically refers to third-party AI platforms like OpenAI's ChatGPT, Google's Vertex AI, Microsoft Azure AI, or similar services where the AI models are hosted and maintained by another company. The question is important for security assessments because when institutional data is processed by external AI services, there are several risks: 1. Data privacy concerns: Institutional data may contain sensitive or personally identifiable information (PII) that requires protection 2. Data retention policies: External AI services might store or learn from submitted data 3. Regulatory compliance: Educational institutions must comply with regulations like FERPA, GDPR, or HIPAA 4. Intellectual property concerns: Research data or proprietary information could be exposed The guidance specifically asks you to detail: - What types of data will be processed by the AI service - Where this data comes from (the sources) - Whether any data shared with the AI comes from outside the institution To best answer this question, you should be transparent about any use of external AI services, specify exactly what institutional data might be processed by these services, explain your data handling practices, and describe any safeguards in place to protect sensitive information. If you don't use shared AI services with institutional data, clearly state that.

Guidance

Provide detailed response to the type of data needed for the AI service to function appropriately, the sources of the data, and whether any data shared with the AI service comes from data sources outside the institution.

Example Responses

Example Response 1

Yes, our learning management system uses Microsoft Azure OpenAI Service to provide automated essay feedback and plagiarism detection The system processes student essay submissions and assignment text The data sources are limited to the content students upload to our platform and the assignment descriptions created by instructors No data from sources outside the institution is used in this AI processing We have implemented the following safeguards: 1) All data is encrypted in transit and at rest, 2) We use Azure's private endpoints to ensure data doesn't traverse the public internet, 3) We have a data processing agreement with Microsoft that prohibits the use of institutional data for training their models, and 4) All processed data is deleted within 30 days of the end of the academic term.

Example Response 2

No, our solution does not use any shared AI services to process institutional data While our product does incorporate AI capabilities for predictive analytics on student performance, these AI models are proprietary and run entirely within our application's infrastructure that is deployed in the institution's environment All data processing occurs within the boundaries of the institution's network, and no institutional data is sent to external AI services for processing, analysis, or model training Our AI functionality is pre-trained on synthetic data and anonymized datasets that do not contain any institutional data from your or any other institution.

Example Response 3

Our product currently does not use shared AI services for processing institutional data, but we are planning to implement a feature in Q3 2023 that will use Google Cloud's Natural Language API to analyze sentiment in student feedback surveys This implementation will process de-identified student comments (with no personally identifiable information) to generate aggregate sentiment scores for courses We cannot fully meet this requirement at present because our planned implementation will send anonymized institutional data to a third-party AI service However, we are designing this feature with privacy by design principles, including data minimization and purpose limitation We would be happy to discuss our implementation plans and potential mitigations to address any concerns before this feature is deployed.

Context

Tab
Privacy
Category
Privacy and AI

ResponseHub is the product I wish I had when I was a CTO

Previously I was co-founder and CTO of Progression, a VC backed HR-tech startup used by some of the biggest names in tech.

As our sales grew, security questionnaires quickly became one of my biggest pain-points. They were confusing, hard to delegate and arrived like London busses - 3 at a time!

I'm building ResponseHub so that other teams don't have to go through this. Leave the security questionnaires to us so you can get back to closing deals, shipping product and building your team.

Signature
Neil Cameron
Founder, ResponseHub
Neil Cameron