Cool Culture
Program Evaluation of Laboratory for New Audiences

Cool Culture contracted RK&A to evaluate the Community Cohort element of the Laboratory for New Audiences (referred to as the Lab).  The Lab addressed the question, “How can New York City (NYC) museums adapt to changing demographics through innovative education and community strategies to attract more diverse audiences and enrich the experiences of culturally diverse families?”  Working in small cross-institutional teams, museum educators applied design thinking to the development of prototypes to address the above core question.  The evaluation intended to determine the extent to which and how the Lab influenced participants and achieved participant outcomes.

How did we approach this study?

The evaluation was multi-phased and included developing an Outcomes Framework, which articulates the intended effects of the Lab project on participants; a pre- and post-assessment of participants’ learning using a standardized questionnaire; and case studies of three Lab teams.  The questionnaires helped us efficiently collect and compare information on participants’ beliefs and opinions before and after the program.  The small sample size precluded us from using inferential statistics to draw statistical relationships across groups; instead, RK&A used descriptive methods to analyze data.  To complement the questionnaire, we used case studies to examine the team experience of developing a prototype to solve a problem.  Case studies produce qualitative data and demonstrate the unique nature of the program at the team level as well as the range of possible outcomes.  Case studies included three interview phases—two interviews with teams and one one-on-one interviews with individuals.

What did we learn?

Findings indicate participants experienced substantial growth in several areas, including partnering with parents and design thinking.  As a result of the program, participants said they will likely use design thinking, such as working collaboratively and understanding audiences, as a foundation for planning and designing future projects.  Participants also valued the opportunity to elicit feedback from parents as they developed program prototypes in Lab sessions.  However, some case studies reveal that participants felt stifled by a [perceived] lack of influence within their own institutions in the development of their prototypes, meaning they did not feel they could be as innovative as they wanted to be.  Findings also suggest that many participants thought the Lab was too long and that they did not have clarity on the process required of them, particularly in terms of expectations for the time needed to develop a prototype.

What are the implications of the findings?

Findings highlight important program strengths, as well as several areas where the Lab could improve.  For example, considering the value participants placed on the ability to solicit feedback from parents, Cool Culture could further improve this experience by teaching participants how to balance the needs of parents with the expertise of professionals (themselves and experts in the field) when developing prototypes.  Such teachings might support achieving the outcome, “Participants will understand that museums can benefit by placing parents in an active role in early childhood learning.”

The program also helped participants recognize that Cool Culture is adept at providing advocacy and leadership training.  Participants noted that Cool Culture can serve as a valuable liaison between museums and communities, providing information about the changing demographics of NYC, current research and resources, and NYC’s political scene in relationship to culturally diverse families and early childhood education.

See all Cool Culture projects.

Program Evaluation Cool Culture