Learn more about the questions and process used to evaluate the privacy and security practices of educational technology.
For the last two years, Common Sense has been working with school districts, vendors, privacy advocates, legal professionals, parents, security researchers, and other stakeholders on developing the core questions we will use to evaluate privacy practices of commonly used educational technology apps. Over that two-year time period, the question set has undergone multiple transformations and versions as we incorporated feedback from these multiple external stakeholders. We were also fortunate to get some incredibly helpful early feedback from Davis Polk on the questions and background.
In our evaluation, we use two main types of questions: transparency questions, designed to gauge what is and isn't covered in a vendor's policies, and qualitative questions, which highlight the content and obligations stated in the terms. Our questions are organized into categories derived from the Fair Information Practice Principles (FIPPs) that underlie privacy laws and regulations around the world. Our questions, and the categories that organize them, are all mapped to a range of legal and technical resources that provide background on why each question is relevant. These questions -- paired with the Information Security Primer -- provide the foundation and define the limits of our evaluation work.
When we evaluate privacy policies using these questions, the transparency questions allow us to get a sense of what is, or isn't, covered in a policy. Obviously, different applications will have different needs, but the transparency questions allow us to set a baseline.
The qualitative questions build on the foundation of the transparency evaluation. Where the transparency questions focus strictly on what is covered, the qualitative questions highlight what the terms actually mean. Between the transparency evaluation and the qualitative evaluation, we start to get a more accurate sense of the strengths and potential risks involved with using an application. By breaking down the contents of terms, and highlighting what is and isn't covered, we allow people to make informed decisions about the software they use. While our initial audience for the questions is school and district staff along with software vendors, the questions are also of immediate use to anyone who wants to understand privacy -- from teachers and students to parents, policymakers, and advocates.
The questions contain a lot of information. As we worked on developing the questions, one of the issues we wrestled with was how best to convey and share the questions with a broader audience. As part of our testing, we shared the full question set -- the categories, the transparency questions with legal rationale, and the qualitative questions with legal rationale -- with a small group of stakeholders. Uniformly, we received feedback that the amount of information was somewhere on the spectrum between confusing and overwhelming.
In response to that feedback, we are releasing the questions in smaller chunks to make them more accessible. For people interested in getting a quick sense about privacy and the various elements that need to be considered when evaluating policies, starting with either the categories or the qualitative questions provides a good initial overview.
For people interested in the range of elements that could be included in a privacy policy and terms of service, the transparency questions are a good resource. For vendors looking to do an initial evaluation of what their terms cover, and what their terms leave out, these questions can be a useful initial point of reference. For people who are reviewing the terms of service for applications, the transparency questions can be useful to help get a sense of potential areas of concern.
For people looking to dig deeper into the background behind the questions, the detailed category explanations, the full transparency questions, and the full qualitative questions provide a good starting point. These versions of the questions provide citations to federal statutes, Federal Trade Commission (FTC) guidelines, Privacy Technical Assistance Center (PTAC) resources, and other sources of information on data handling and privacy. We have already published the complete list of sources we consulted for this work.
Additionally, for people who want to browse by category, we set up this page - organized by category - to let you jump right in.
At the risk of stating the obvious, every question will not apply equally across every application. A graphing calculator, a student information system, and an adaptive learning platform all have very different levels of potential risk. These levels of risk will also vary depending on who is using the application -- for example, the needs of a third-grade student, an eighth-grade student, a high school junior, a teacher, and a district administrator will vary widely and have different levels of acceptable risk. Additionally, standards and concerns will vary across schools and districts. Ultimately, decision makers will need to make an informed choice for their specific context. Our question set -- freely and openly available and fully documented -- provides a consistent point of reference to support these informed decisions.
As is the case with everything we are doing on this work, we are constantly iterating, learning, and -- hopefully -- improving. The questions we are releasing today are a good representation of the questions we will be using through and after our launch, but ongoing revision and updates have always been part of our plan. Toward that end, if you want to be part of the cohort we consult for future feedback, please be in touch. If you are a school or district and want to be part of the consortium that is driving this work, please join us.
Image credit: "Takeoff," by Bill Fitzgerald, used under a Creative Commons Attribution Share-Alike 4.0 License.
The privacy evaluation questions are released under a Creative Commons Attribution Non-Commercial Share-Alike 4.0 License. If you use these questions in your non-commercial project, please credit "Common Sense Media" as the author and link back to the announcement post.
