Tuesday, August 4, 2015

New Economic Models for Field Wide Research

On my desk is a report from Reach Advisors’ Museums R+D membership program. It’s on museums and trust.

Before I opened the report, the latest data I had on this critical subject was a survey about public attitudes toward museums that AAM released back in 2001. That study (gratifyingly) showed that our sector was viewed as one of the most trustworthy sources of information. But a lot has changed in the last 14 years. People have higher expectations about transparency and ethical behavior (see TrendsWatch 2015). And our field has been buffeted by controversies over funding, conflict of interest and influence over content.

Museums R+D dug into this issue by:
  • Polling over 7000 members of the public about trustworthiness of information presented by institutions (including museums), media and researchers
  • Digging into the issue more deeply with a panel of 152 museum goers, asking why museums are trustworthy, and the role of curation in building and maintaining trust. They also asked the panel a very timely question—whether how money is raised, and from whom, affects the public’s trust of museums.

I’ll share some of the top line results at the bottom of this post, but first my main message: MUSEUMS NEED MORE RESEARCH LIKE THIS. (No apologies for all caps. I was indeed raising my digital voice.)

Back in 2010 Betty Farrell, who led the U. Chicago team that prepared the CFM report on the growing demographic disconnect between museums & the American public, shared her thoughts here on the blog about the “outdated and narrow” data available on race & ethnicity among museum audiences. She threw down a challenge, to individual museums and to our field to:
  • Make better use of existing data
  • Mine data from other sources
  • Pressure existing research projects to capture more information about museums, and
  • Share the knowledge. (Emphasis added.)
  • Too much valuable data is locked away in proprietary studies, in the form of market research or evaluation studies,” Betty observed, “but never shared beyond the walls of the museum that commissioned the research. Museums need to develop a shared expectation that the knowledge that what they collect as individual organizations will be shared with the field unless there is a compelling reason for it to remain confidential. There are many models of data sharing on a local or discipline-specific scale.

To Betty’s imperatives I now add another: find and join research coalitions—they may well be the future of sustainable field-wide research.

Economic and cultural trends have made it massively harder for traditional large scale research to thrive. The last time AAM conducted the field-wide Museum Financial survey was 2009. Why? First, people aren’t willing to fill out long, complicated surveys the way they used to. (This isn’t just my experience—every researcher I’ve talked to agrees this is a broad shift in behavior and expectations.) Second, the old financial model supporting research doesn’t work. While museums say they want on-demand benchmarking data, AAM’s experiment offering an online, real-time data service—what we called Museum Benchmarking Online—failed because people weren’t actually willing to pay a subscription fee that covered our costs. (Or to put data in before they actually needed to get data out. Usually the night before a board meeting.)

Another example of the “good old days” being done and gone: the fabulous Listening Post Project run by the Center for Civil Society Studies at Johns Hopkins University, which enlisted representative institutions from a range of nonprofits (museums, theater, family and child services, aging) to act as wind socks detecting trends shaping our sector. After about a decade of doing great work and issuing “soundings” on topics that included nonprofit values, innovation and performance measurement and the recession effect on nonprofit jobs, the Center concluded they couldn’t keep the program running on philanthropic funding and the intermediary partners were not willing to pick up the costs.

A number of experiments are popping up to fill the resulting research gap. Some people who run or work in research firms (frustrated because, working for individual clients, they don’t get the chance to ask big, interesting questions about museums as a whole) have created non-profit affiliates (that can receive direct grant funding) such as The White Oak Institute, a spin-off of John Jacobsen and Jeannie Stahl’s White Oak Associates. Others are finding ways to compile and share non-proprietary studies and research. For example, CultureLab, a “social library” built by WolfBrown to facilitate knowledge sharing, now includes digests of arts and culture research conducted by Createquity*.

But grants and foundation support can only go so far. Hence, a niche for “subscription” or membership-based research groups. These kinds of research coalitions can play an important part in gathering data of wider importance to the field. They provide a sustainable model bridging the gap between big, grant funded projects (increasingly rare), and bespoke research for individual museums (often not shared at all or not having wider significance). And because research coalitions are financially supported by diverse clients, they are guided by good practical feedback about what museums, collectively, need to know.

Personally, I think Reach is doing great work with Museums R+D. While the museums supporting the work get the first and highest benefit (as is only fair), in the end the research will benefit the field as a whole. I bet there are other, similar coalitions out there, and am hoping you will weigh into the comments section, below, to share opportunities to engage in those.

OK, I promised you I would get to the good stuff, so here are some nuggets from Reach’s recent report:

On a scale from 1-10 of public perceptions of trustworthiness, museums scored a 6.4. Corporate researchers came in at the bottom, with a 3.6, Fox News clocked in at 4.7, NPR at 5.0 and Wikipedia at 5.7. (Researchers collectively, corporate, government and nonprofit scored at the bottom of the heap, which I found profoundly depressing. Also, somewhat ironic, given the source of this information. #Meta.)

History museums/historic sites score 6.7 while science centers earned a 5.2. (Lower than Wikipedia and as Reach notes “only marginally better than the U.S. Government.” Troubling.) Reach speculates that this is because science museums are tackling highly politicized issues like global warming, evolution and vaccination--i.e., doing their jobs.  However, drawing on previous research, they note that core visitors to science centers rank them a 9.4 on trustworthiness—suggesting that there is a huge gap between those who go to science centers and those who don’t. (Which may say more about self-identification and pre-selection than about impact—Reach notes the gap is important and worth further investigation.)

Regarding whether a museum should accept funding from a source with vested interests that might be at odds with a museum’s mission and values, I was interested to see that the museum panel split in their opinion (as are museum staff, in my experience). Some see this as the first step on a slippery slope, while others pragmatically say "take the money, just make sure there are no strings attached." The key to maintaining trust while accepting funding like this, all agree, is transparency, and to this end the panel made three suggestions and one request:

  • Make all gift agreements and communications with the donor public
  • Make transparent how the content presented was researched, sourced or otherwise created, so visitors could see it was not biased by the donor
  • Make every effort to balance the controversial donor with matching gifts from donors who hold opposing viewpoints, neutralizing the influence.

And the suggestion? Reach summarizes it as “Trust us as smart individuals to make up our minds about your credibility, and to vote with our feet if you make the wrong choice.”

Sounds like good advice (and a just a little bit of a warning) to me.

*This post was corrected on 8-11-15 to accurately reflect the relationship between CultureLab and Createquity, and credit the support of WolfBrown.


James Chung said...

A belated note for the shout out. We ended up pursuing this model for field-wide research since it's become almost impossible for us stand on the sidelines when there's a lot more stuff screaming for deeper examination and stronger action. While there are some museums willing to pursue the level of depth necessary to drive towards a greater level of impact, we've found that it's not really a very big universe of museums willing to do that. And when they are willing, it's typically to address a specific challenge of their institution, rather than designed for the entire field.

To help bridge the gap, we launched Museums R+D, designed to tackle the deepest issues field-wide concern as well as for our participating museums. In a way, it disrupts the core business model of Reach Advisors since the fees we collect from Museums R+D are far lower than for our custom strategy engagements. But it's allowed more museums to join in this quest to tackle and address some of the biggest issues out there. It's been a fascinating journey as we've been able to build a much deeper understanding of museum goers and the general public, and we look forward to continuing making more of this kind of work more accessible to more museums.

As you well know at CFM, while some of what's ahead presents significant challenges for museums, it also presents significant opportunities. So let's all keep pushing for more ways to uncover how museums can have even greater impact as the future unfolds!
- James Chung, Reach Advisors

Georgianna said...

Elizabeth -
AAM's field-wide financial survey was of great use to the field. I think it is significant that the reluctance of museums to fill out lengthy financial documents coincided with new IRS 990 reporting requirements which took effect in 2009. Why not collect 990's from AAM members, mine these forms for data, and fund related costs through a nominal membership surcharge?

Just a thought -

Georgianna de la Torre

The Alliance's Center for the Future of Museums said...

Hi Georgianna, thanks for weighing in. We did look into this, but after consulting a committee of museum folks, found that the 990s do not contain a lot of the information that museums want to know. The IRS data categories simply don't map to much of the benchmarking data museums want to access (like expenditures on collections care), and related data like facility size that enable us to calculate key metrics like cost per square foot. The other challenge is that a big percentage of the field don't file separate 990s. Over a third are part of a larger parent organization (for example, a university, local state or federal government). For these reasons, we feel that the cost of 990 mining would not be justified by the incomplete and not-representative data that resulted from the work. That said, my personal suspicion is that the future of research lies in mining of a number of sources of open data, including data museums will increasingly agree to put online themselves. In this universe, 990 data (if it is posted in a usable, readable, format) could play an important role.

Brian Hoshowski said...


This implies a very costly tradeoff when macroeconomic and financial stability goals are in conflict.