So many programs are virtual now. How can we advocate for their value?
Throughout the Covid-19 pandemic, local governments, nonprofit organizations, and their funders have had to make hard decisions about what defines an essential service. When push comes to shove, arts and humanities organizations are often left behind. We invited leaders from our original Arts Roundtable, as well as others from around the country, to talk about tracking impact in the virtual space.
To kick off the session, Dr. Janelle Junkin gave a short presentation about her consultation work with the Armed Services Arts Partnership, a D.C.-based organization that provides community building exercises and growth opportunities for veterans and families of the military. Over the past year and a half, Janelle designed a program impact evaluation for A.S.A.P.—then coronavirus happened. Please find an excerpt from the discussion below.
If you’d like to help us organize a roundtable on an issue impacting your city or cities around the country, please reach out to programs@tomtomfoundation.org.
Participants:
Chelsea Thaler, Director of Programming, Ocean City Film Festival (Ocean City, MD)
Janelle Junkin, Researcher, Consultant, and Music Therapist (Philadelphia, PA)
Ebony Lumumba, Associate Professor and Department Chair of English & Modern Languages, Tougaloo College (Jackson, MS)
Enjoli Moon, Founder and Creative Director, Afrikana Film Festival; Assistant Curator of Film, Institute for Contemporary Art at Virginia Commonwealth University (Richmond, VA)
Patti Pan, Chief Executive Officer and Founder, RevArt (Charlottesville, VA)
Dwandalyn Reece, Curator of Music and Performing Arts, Smithsonian Institution National Museum of African American History and Culture (Washington, DC)
Emily Smith, Executive Director, 1708 Gallery (Richmond, VA)
Marlon Torres, Executive Director, NC Arts in Action (Durham, NC)
Jim Walker, Chief Executive Officer, Big Car Collaborative (Indianapolis, IN)
Tom Tom Foundation
Rachel Baker, Summit Director
Ben Wilkes, Program Manager
Janelle Junkin (Researcher, Consultant, and Music Therapist): I started working with [the Armed Services Arts Partnership] in 2017 and I did a year and a half, mixed-methods program impact evaluation that we are actually getting ready to submit a public report about. We were in partnership with Western Carolina University, so it did go through an IRB approval, and it was specifically looking at the impact of participating in their comedy, improvisation, storytelling, and creative writing programs on veteran well-being. We looked at that from the veterans’ self-reporting and staff reporting, so talking with the teaching artists, the family members, and the caregivers as well as participants. A.S.A.P. is set up to do performances with audiences, so it was all in-person. We created an internal monitoring and evaluation system that's been in operation just about a year now. Then Covid-19 hit. So, in probably about one month’s time, we switched everything to accommodate all of the different avenues that they are now using, that they weren't before, in terms of all these virtual touch points. We wanted to really then hone in around the impact of Covid-19, trying to suss out why people are participating in the programming and the impact it’s having. We rebuilt the entire M&E system in a matter of weeks, to capture that data for everybody who’s going through it. In the psych world, we're talking about the psychological impact and the uptick in isolation, depression, and suicide that we're tracking in the field. This is a really great time for community arts and creative arts therapists to work together to say “how do we begin to track our impact on these areas that the psych world is concerned about?” Again, for an internal monitoring and evaluation system, we really switched over to measuring the impact of participation in these classes. The pivot that we made with Covid-19 was really just looking at isolation, belonging, and well-being, and specifically improving mood. We're tracking through polls, and then through post-surveys at the end of all of their points of contact, and the engagement, people self-reporting back about isolation, belonging, and mood. The other piece that we're also doing is just basic feedback around: Was this accessible? How accessible was it? What was the ease of using the online platform for delivery of the content? I will say the main concern that I have as an evaluator is survey fatigue. You really shouldn't have more than 5-10 questions maximum and ten is kind of pushing it. You want to make sure that your questions are clear and focused, so you don't want to ask three different things in one question.
Tom Tom: How can we sustain appetite for virtual events and virtual surveys? In what ways are we tracking impact? How do we advocate for the impact of the arts in this moment?
Ebony Lumumba (Tougaloo College): This is an opportunity for every industry, including our arts communities, to increase access to what it is that we produce because we know that it’s beneficial. No one should be isolated from artistic manifestations and productions. We've gotten to share some CARES grants through one of the boards that I serve on with a lot of museums, a lot of art spaces, and what I look for in a lot of those applications is how they intend on increasing their accessibility to the communities where they’re housed. I hope that this moment is not overlooked as sort of this sort of this arduous thing that we're doing, just to get through, and then we go back to business as usual, but we start to incorporate more voices and more communities in what we're producing and what we're doing.
Dwandalyn Reece (Smithsonian Institution National Museum of African American History and Culture): I think part of this is making things more widely accessible and showing the importance of doing so, not just doing it as an add-on. It really needs to be including the fabric of your daily work. To counter that, we've also had discussions about people who don't have digital access. Whether they don’t have internet connections, or they’re older, or maybe some younger individuals who don't do Zoom, or don't do all these things. So much of our audience falls into one of those categories—it's a unique dilemma. Part of the experience of the museum is being in that space, and no virtual platform or engagement is going to capture that. It's a pilgrimage. The space is part of the narrative that we tell. Healthcare, education, and businesses are all important issues, but what I see at jeopardy are the real ways of connecting: being face to face, touching people. Those are very powerful things that sustain us in all ways.
Enjoli Moon (Afrikana Film Festival, Institute for Contemporary Art): I agree with that but I’ll also add that I've seen communities come together in a really tremendous way during this. There have been really creative ways that people have decided to help each other, in ways that they weren't doing before this. They’ve figured out how to utilize technology in ways that have allowed us to keep some semblance of connectivity. If organizations and institutions can be creative enough to find ways to connect with people right now and help them ride that wave into like post-Covid reality, people are so starving for what you're saying, [Dwandalyn], that I think they're going to rush to the places that have intentionally created space.
Patti Pan (RevArt): We are thinking about these new curves and new initiatives a lot because we're a start-up, so I’ve been observing what people are doing right now. Many people spend a lot of time on Netflix, listen to their Spotify, play puzzle games. They just need some entertainment and they want to engage. People need to handle a lot of other things like health issues, family issues, jobs and a lot of things. They want something very simple and to feel distracted. Like yes, they're connected, and I believe that art and creativity actually can have some impact, but it should be easier and should request less energy from people.
Chelsea Thaler (Ocean City Film Festival): We're presenting everything on Facebook right now because we have such a large app gap in our ages that we provide content to. Our film festival was March 5th-7th, so it felt like we had the film festival, it was a huge success, and then all of a sudden the world was flipped over. Right now, we’re doing all of our programming pretty much for free because we know that it’s going to help our audiences, it’s gonna help their mental state. They’re gonna have a film to watch on Saturday. We’re getting like 600 people a night viewing our films on Facebook and for us as a small film festival, that's unprecedented. We had almost 700 people, in-person in March, to see our feature film. So just to give you a sense of our numbers and how we're tracking that. Facebook tracks everything for you. [Facebook isn’t] suited for every organization, but we found it to be really useful in terms of tracking our impact.
Tom Tom: I’m curious about everybody’s numbers. Of course, you don't have to have huge numbers to have an impact. That's so interesting, Chelsea, that your numbers are up. Is everyone else, as they pivot to virtual, feeling that?
Emily Smith (1708 Gallery): [Before Covid], we really didn’t actively use social media in a way, though it was a goal. So there’s some silver linings in all of this. We've been doing once a week studio visits where the artists just talk about what they’re working on, and people are commenting and asking questions in the stream, and then we're putting them on IGTV and we’ll have like 500-600 views of them over the course of the week. That's vastly different than the number of people walking into the galley on a regular Wednesday or a Thursday. In a lot of ways, it’s given us a platform that we didn't have. Even if [the videos] are kind of glitchy, and not perfect, they are really accessible and you have this chance to see a person. There's a human component to that, that's part of the reason why they've been so popular.
Janelle Junkin (Researcher, Consultant, and Music Therapist): I think it was Ebony who said it first—this pandemic has brought forth the inequalities and access to the arts. I’m wondering if maybe part of the reason that you’re getting higher numbers is that people who couldn't normally in-person access them are able to access them digitally.
Tom Tom: What are people hearing or feeling or getting on the connection between impact and funding?
Jim Walker (Big Car Collaborative): One of the things that we're trying to figure out is how we're moving our work out into public spaces and how you can do surveys and data collection out in public. We know that everybody out there may not have a smartphone to fill out a survey on their phone. You can't do what we normally do, which is just have people with clipboards just talking to people and having them answer, and then we just fill out their answers, so they don't even have to write anything. What we're trying to work on is using data to support our partners, as well as ourselves, in terms of funding potential.
Marlon Torres (NC Arts in Action): When things are normal, of course our funding is always tied to what we can demonstrate the impact of the program is, and we have to be able to track that somehow. We use surveys for that and we use a designed research program to measure our impact. Now that we have been teaching online, we haven't gotten to that point. We just think “okay let's measure our impact on how many children we can get online now” and look at it that way and have a goal of re-engaging more and more children as the weeks go by. That’s really where we stand but it does concern me that demonstrating impact so you can go to your funders and say "this is what we're doing” somehow either through numbers, or through your story. It’s going to become even more important now. So that's one of my concerns: how to measure this impact and how to put it in a narrative to make sure our funders say “yeah that's worth investing in.”
Dwandalyn Reece (Smithsonian Institution National Museum of African American History and Culture): One of our strategic initiatives right now is to reach a billion people. It’s easy to say you've reached X amount of people, but what kind of engagement are we having? In my experience, there hasn't been a lot of conversation about that. How do you measure the impact of a humanities program? We all know, but when you have to go to Congress and make your case, the stories do help, but they also look at the numbers, while the people on the ground are creating experiences that have real weight and really change communities. We know that's true. I think part of the challenge, that I think we still have—and you can talk about higher education and making a case for the humanities—is getting that message across and getting people to value it.
Ebony Lumumba (Tougaloo College): Folks get really excited aboutapplications that demonstrate this sortof interdisciplinary existence betweenthe arts and all of these other fields. Students get the mostexcited about our courses and curriculumwhen they can draw these connections, when they can talk about what they’reinterested in, in conjunction to whatthey're forced to take tofulfill institutional requirements. Some folks feel that thearts can be marginalizing and it onlymakes sense to a certain population offolks. We know that that's not thecase. There are folks that are curatingthose experiences in every community, inevery hood, all over the world, and soreally playing up how this should matterto you because this is what you’realready doing and this is what alreadyexists in your space.