Students are back in school and today we are pleased to feature a guest post from our consulting analyst Gemma Mangione. She offers insights into her teaching practice in the Arts Administration program at Columbia University.
Evaluation factors into the Association of Arts Administration Educators’ graduate and undergraduate curriculum guidelines across an array of areas including fundraising, strategic planning, and diversity, equity, and inclusion. This curricular emphasis underscores the increased importance of evaluation skills in arts administration more generally and, for educators like myself: a faculty member in arts administration who advises, teaches, and collaborates with future arts professionals. But deciding how best to teach such skills — often for professionals who may never conduct evaluation, but nevertheless need to know about it — is open to interpretation.
In spring 2018, I led my program’s first elective in arts evaluation, which I titled “Politics and Practice of Arts Assessment.” On the “practice” side, students learned how to identify double-barreled survey questions (bad), practice random sampling (good), and write evaluation proposals for funders (necessary). They debated whether and how “theories of change” were appropriate for most arts evaluation projects and reflected on the “positive evaluation phenomenon” in audience research: the idea that people often feel daunted to offer negative opinions on arts programming because of a perceived lack of expertise. We also had special guests from LaPlaca Cohen, Slover Linett, and our very own Stephanie Downey, all of whom introduced students to different types of evaluation services and data resources, discussed case studies, and facilitated useful exercises.
But I also wanted to teach my students something more. I wanted them to finish the course able to make informed decisions about evaluative practice, guided by an understanding of its philosophical underpinnings: that is, to be sensitive to evaluation’s “politics.” Here, for example, we talked about how evaluation — often presumed an objective measure, like a thermometer — can in fact change the very thing it ends up evaluating. To this end, we read Wendy Espeland and Michael Sauder’s book on law school rankings, which traces how U.S. News and World Report’s categories for excellence (such as alumni outreach) have, over time, come to shape where universities invest their money and resources. We also spent a week talking about some arts professionals’ resistance to evaluation — the idea that art is about something special that can’t be reduced to facts, figures, and dollars — and brainstormed how to respond to this challenge. And we did an extended session on ethics, identifying how and why norms of ethical practice vary among medical, social, and arts research and evaluation.
How best to teach evaluation to future arts administrators? My belief is that no one can learn the evaluation strategies necessary for completely independent internal evaluation in a single semester. But you can learn what a logic model is, and why it may (or may not be) useful to you; you can learn how to write an RFP that balances between what you think might be right and also asks for suggestions. Formal evaluation notwithstanding, you can build principles of action research and impact planning into your daily work, which allows you to assess whether you’re meeting your goals and how. And you can even come to understand and think through how to talk to both the people who think evaluation is only for the bean-counters and those who think art’s intrinsic, ineffable benefits are not worth measuring. All of this necessitates training students that evaluation is not just a set of skills, but a way of thinking: a politics, and a practice.