Wednesday, 28 March 2018

What you measure - distorts and affects

With the concern over Cambridge Analytica's use of Facebook to profile and influence voters in many counties, closer scrutiny is being rightfully given to all aspects of data measurement and the effect this can have, both intended and unintended, internet or otherwise.
A good post by Ignatia de Waard draws four lessons from the CA/FB debacle, and drew my attention to a criticism of the OECD's PISA tests. Although this piece is concerned with correlations and how it seems to be used "successfully" in the recent profiling and even given as proof by the OECD, the truth is what you concentrate on, measure, report, and then - crucially - compare with other countries (or country divisions), this profoundly distorts and affects in the pursuit of short term fix-its.
The criticism (The Guardian: OECD and Pisa tests are damaging education worldwide - academics) dates from May 2014 but is, none-the-less, a very useful list of the pitfalls of these tests. Four paragraphs were of particular note:

  • As an organisation of economic development, OECD is naturally biased in favour of the economic role of public [state] schools. But preparing young men and women for gainful employment is not the only, and not even the main goal of public education, which has to prepare students for participation in democratic self-government, moral action and a life of personal development, growth and wellbeing.
  • Unlike United Nations (UN) organisations such as UNESCO or UNICEF that have clear and legitimate mandates to improve education and the lives of children around the world, OECD has no such mandate. Nor are there, at present, mechanisms of effective democratic participation in its education decision-making process.
  • To carry out Pisa and a host of follow-up services, OECD has embraced "public-private partnerships" and entered into alliances with multi-national for-profit companies, which stand to gain financially from any deficits—real or perceived—unearthed by Pisa. Some of these companies provide educational services to American schools and school districts on a massive, for-profit basis, while also pursuing plans to develop for-profit elementary education in Africa, where OECD is now planning to introduce the Pisa programme.
  • Finally, and most importantly: the new Pisa regime, with its continuous cycle of global testing, harms our children and impoverishes our classrooms, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted "vendor"-made lessons, and less autonomy for teachers. In this way Pisa has further increased the already high stress level in schools, which endangers the wellbeing of students and teachers.