Thinking Critically about Evaluation

As a staff, we have noticed the slow by steady upswing in the number of museums doing and requesting evaluation over the years.  While evaluation was uncommon in the museum world 15 or 20 years ago, today many, many museum professionals are enthusiastic advocates for evaluation and view it as essential to their work.  Ultimately, we are thrilled about this trend because we truly believe that evaluation can be used as a learning tool to reflect on and improve practice.  This has to be good for the museum world, right?  But, there is a part of me that worries about this trend.   As someone who values quality, how can the field be sure the results produced are reliable and useful?  Just because someone says they are doing evaluation, should we take at face value that the evaluation they are doing is “good evaluation?”   No, like most things in the world, there is a continuum of quality when it comes to evaluation—there are ways of doing evaluation that will lead to results you can feel confident about and make decisions from and there are ways of doing evaluation that lack purpose and will result in piles of data that are meaningless and therefore never acted upon.

All of this hit home for me recently when I worked with a museum’s education department to build staff capacity for evaluation.  The education department in this museum had been doing evaluation on their own for years, and while much of it had been useful, they felt they were sometimes collecting data for the sake of collecting data and not quite able to make decisions about what to evaluate and what not to evaluate.  None of them are trained in evaluation, but they all have a great respect for it and wanted to learn how to do it better.  Thus, I step in.  Gulp.  I was honored that they wanted me to teach them about evaluation.  As a trained evaluator with many, many years of experience, it should be easy, right?  I quickly realized that teaching others how to do what you do is anything but easy.  And in the process of preparing my teaching materials I did something I hadn’t done in awhile.  I looked very critically at what I do as my chosen profession and asked myself how I do it—I broke down what I do everyday into pieces that I could explain and teach.  And in the process I came to a new appreciation for how hard it is to do evaluation well unless you truly have the training and the experience.  I have to admit, I feel really good about what I have been able to teach the education department of this museum about evaluation, but it hasn’t been easy by any means.

All this is to say, that we would like to start a conversation about how to conduct high-quality evaluation so that evaluation efforts will result in findings you can feel confident about and use.  High quality isn’t about size—good evaluation can be a very small evaluation study (with small sample sizes) done in-house or a large evaluation study with mixed methods and complex analysis strategies.  Quality evaluation hinges mostly on having a purpose when planning and staying true to that purpose when implementing a study.  As a result of all this thinking we have been doing, we are planning to host our first Twitter Chat, where we will invite museum professionals to think critically about evaluation through a series of questions we will pose. Stay tuned for more details!

1 Response

Leave a Reply