Q&A Session with Ben Morgan by Natalia Grincheva

Questions from Anna Molnar

molnar85 — 2018-04-09 07:43:55 UTC

How do you evaluate the impact of something that has more of a symbolic value, like a song, than actual instrumental change e.g. economic or social?

What qualitative methods are useful in this area for you?

A: Evaluating 'impact' is first about defining what that 'impact' actually is. What do you want to measure? What are your indicators and methods would then follow from that. You could interview many people about how songs make them feel, that would be a more standard way of doing it. Daniel Lewitin is a neuroscientist at McGill who tries to measure the effect of music on brain activity. Spotify's algorithmic ecosystem is based on machine learning evaluation of song files to create metric indicators of musicological qualities (key, tempo, timbre, rhythm, etc. but very comprehensively). There are many different approaches to how to measure meaning. Symbolic meaning sounds like humans creating meaning, so I would say talking to people is probably still the best way to do that. But you could also scrape social media, blogs, etc. to see what people are saying about an artist or a song. I think most of us still react to how journalists, bloggers, critics, and other fans react to popular music. Cultural intermediaries are still involved in shaping the meaning of popular music. Thanks for your question -Ben

p.s. - 'evaluating impact' is associated with instrumental use. I don't think I've heard of evaluating the intrinsic impact before. That would be more about trying to capture and evaluate value and meaning, and less about 'impact.'

Mark Thomson

fitzroy66 — 2018-04-09 07:47:32 UTC

Could you compare results to similar programs ( if they exist) in other African countries?

A: If you are referring to the STYL evaluation by IPA, I believe there are mass studies of

Thanks for your question -Ben

caoydan0311 — 2018-04-09 07:54:30 UTC

Heyuan *

HeyuanXing — 2018-04-09 08:14:17 UTC

Art therapy can't bring instant impacts and the public may question and challenge this method. Have you meet some critiques?

A: Art therapy is not an area I am familar with. There are many questions about art therapy on this padlet, so I wonder if it was not communicated properly. The STYL program involved cognitive-behavioural therapy (psychologyical technique) and Song For Hawa was Education Entertainment, or a behavioral change intervention delivered through a song vehicle.

Regarding critiques, the donors who fund cultural projects like Song For Hawa like to see reports say they were effective, but these are rarely rigorous. The idea that these instrumental cultural interventions are effective uses of funds to improve the lives of people in low-income countries is questioned constantly, especially when there is a competition for donor funds to pay for interventions. Someone who wants funds for their project to build sanitation systems is going to look at something like Song For Hawa as a waste of money if they see donor funds as a zero sum game. As I like to mention, the idea that the arts are transformative and good for people is an ideological belief that many of us share and it is difficult to prove in a metric fashion, but it can certainly be done. It takes a lot resources: time and money, plus often exogenous variables (i.e. Ebola) can come along and mess up your data.

Thanks for your question! -Ben

J. Avery *

jcavery — 2018-04-09 08:15:21 UTC

Could you speak to the relationship between the spotify/music on demand "datafication of listening" and that qualitative data was pretty much all that you could obtain in regards to the rap song discussed earlier.

Couldn't these methods of analysing data be altered to be applicable to the qualitative data you mentioned, such as that the song was popular on the radio?

A: In Liberia there was/is no sort of system for evaluating music usage. The internet is coming to Africa fast, but while I was there it was only enjoyed by elites.

But your question is a great one. Metrics can tell us a lot of things about how people are using music. How meaning is shaped is a rather human story, but Spotify, for example, tries to quantify meaning through machine learning about the song file and scraping the internet/social media to see what people are saying about it and what other information is out there. If you are really interested, have a look at one of the founders of the Echo Nest's PhD Thesis https://dspace.mit.edu/handle/1721.1/32500

Thanks for your question -Ben

Daniela Gutierrez

Gutierrez84 — 2018-04-09 08:19:21 UTC

In regards to the project in Liberia, is there any government support to get data so as to measure the impact of the organisations´ efforts through songs and visual arts to attack social issues? In essence, what is the government´s role in supporting the organisations to fulfill their goals?

A: Some perspective on a country in the least-developed bracket is helpful. They don't have very good data on a lot of economic priorities like who has access to health care, electricity, etc., and what does exist is created by international institutions (CIA Worldbook, UN, etc.) They don't have a lot of things we take for granted. The early part of the lecture pointed out how in 2015 more of their budget consisted of NGO projects and other 'off-budget' support than actual tax revenue. They have a Ministry of Culture, and Ministry of Commerce, etc., but they have a lot more pressing priorities than cultural consumption or other sorts of data. These things will appear, they just aren't priorities yet. For those sorts of secific cultural projects, data really needs to be created. In the case of the STYL project it was done rigorously, at great cost. Its mainly a matter of priority of funds. The government is going to have a hard time defending a decision to spend budget on data collection when people don't have access to infrastructure, health care, good education, etc.

Thanks for your question! -Ben

Mark Thomson

fitzroy66 — 2018-04-09 08:20:02 UTC

Why doesn't Spotify want to use people to choose the songs for Mix tapes?

A: Most of the streaming sites allow you to create your own playlists. Not quite sure what you mean exactly, but Spotify has lots of pre-made playlists on offer for you to enjoy. But anyone can create their own and make them publicly accessible to other users.
Thanks for your question -Ben

Olga Yakubenko

olga_yakubenko — 2018-04-09 08:20:04 UTC

How did you choose the focus group? Does the number of people in the focus group depend on the number of population? E.g. any certain share of the population to make the experiment representative?

A: Focus Groups were not used in the interventions I talked about. The STYL behavioural change intervention for ex-combatants had research participants who received the cognitive behavioural treatment. This is different than a focus group, which is generally just having a chat about opinions in a group setting.

The intake/recruitment for the STYL study was complex and you can read more about it in the paper , but I know they identified around 1,500 potential participants and then recruited them. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868

Thanks for your question -Ben

Wanying Yang

wanyingy — 2018-04-09 08:21:44 UTC

Is there any rule of choosing focus group in those programs?
In terms of music therapy, do you think indicators are the most effective solution?

A: Focus Groups were not used in the interventions I talked about. The STYL behavioural change intervention for ex-combatants had research participants who received the cognitive behavioural treatment. This is different than a focus group, which is generally just having a chat about opinions in a group setting.

The intake/recruitment for the STYL study was complex and you can read more about it in the paper , but I know they identified around 1,500 potential participants and then recruited them. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868

Indicators are necessary to measure anything, so if you want to evaluate any kind of therapy, you would need indicators to measure and evaluate it, so they are not just 'the most effective solution' you actually NEED them.

Thanks for your question! -Ben

Yingni Chen

Yingni_CHEN — 2018-04-09 08:23:51 UTC

Is this kind of 'Music Therapy ' you just mentioned also effective in other countries,
or, are the results different when this way introduced in different countries( e.g. developed and developing countries)

A: I've seen several questions here on Padlet about 'music therapy', which is something different than the interventions I looked at.

The musical approaches to behavioural change (i.e. Education Entertainment) like Song For Hawa are often not well-studied in their own country and I am not aware of any comparative studies. I can tell you that different cultural environments will certainly have different ways, but local artists probably are well-equipped to create pieces based on their context. You can look at the PCI-Media Impact website to see all the projects like this they have around the world, just don't go looking for rigorous evaluations because they don't seem to prioritize them.
http://mediaimpact.org/
There are many international NGOs using cultural tools to try and change behaviour like they are. How effective they are? That I am not so sure of. The impact of these sorts of projects are often not very well researched.

Thanks for your question -Ben

Do you think it is a good thing to collect data for both indicator and people?

Li945936 — 2018-04-09 08:24:13 UTC

For example, you mentioned in the lecture, people are more concerned about music instead of lyrics. So, it won't be an efficient way to change the situation. (Na LI)

A: Not sure I understand the question. Collecting data depends on what you want to learn, and indicators are for measuring data. Evaluating the impact of something is going to depend on what you are trying to measure. Music vs. lyrics are not really things you are going to want to evaluate in a rigorous evidence-based analysis in the same way that you would a policy, or an intervention in a low-income country, or in a behavioural change scenario.

Thanks for your question, sorry if I could not understand what you meant. -Ben

Olga Yakubenko

olga_yakubenko — 2018-04-09 08:24:47 UTC

Do the NGO's provide any methodology on how to do this kind of surveys?

A: You can read about the methods of that evaluation in https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868 It is a whole other field of research so there are certain to be methodology touchstones. There are many economists working in this field and arguing over methods. I was not properly trained in behavioural economics so I can't comment on what the best sources are at the moment, I'm sorry, but I know that Esther Duflo and Chris Blattman are two of the main academics five years ago whom I associated with Innovations for Poverty Action (IPA) and who focus on that approach. As far as I know, they are still thought leaders in this area. Thank you for your question! -Ben

Jingyi Li *

2018-04-09 08:25:44 UTC

(1)* In the pictures or music you showed before, some of them express the ideas in straight and strong ways, and it is easy for people to understand, for example, it will tell people, stop doing something, do not do something. However, I know that some of them express the ideas in creative or soft ways. It will let people to consider and conclude the idears by themselves. In your opinion, which one is more effective?
(2) Is it necessary for students to learn music with strong educational expressions in school music lessons?

A: 1) This 'pedantic' style of instructional/educational music is going to be great in some situations, while in others it could completely backfire. I think part of what I wrote about Takun J is that it was a better of example of soft power / cultural diplomacy (in the non-state actor sense of one value system to another) than many examples of 'informational diplomacy' (e.g. the murals saying don't rape, the song saying 'small girls say no to sex'). I think the more sophisticated audience, the more you have to be interesting and compelling and more than just pedantic, but that is really just my opinion. I am not aware of much good research on how effective these kinds of interventions are. Part of the point of the lecture is that they are not well studied, at least in international development.

2)I think this is a very different question that I can't really answer, sorry. I think it is easy to see that plenty of people love music who don't receive any kind of education about it, though.
Thanks for your question -Ben

Qazim

AbdulKarim — 2018-04-09 08:25:44 UTC

The surveys that were done took place over different time periods - 12 months, 13 months; how do you determine this and why so?

Also, were all 999 men involved throughout the whole duration - did any of them leave the study halfway and hence, disrupting the eventual, potential results?

A: I know there is specific information about methods in the paper at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868

From what I recall, I am pretty sure that several participants were not located for subsequent surveys, and the researchers worked very hard to keep these to a minimum. The periodic time periods are likely a combination of strategy but also the logistics of running a study in Liberia where rainy season makes it very difficult to travel during certain periods. Also locating funding to run more cohorts was also a variable and the study shut down and restarted when new funds were found to run more trainings. Those 999 men were over several cohorts, I am sorry I don't know the number. Both questions are probably examples of: you plan a study certain way, and not everything goes right. It is unlikely with a study this large that you will get 100% of the participants to complete all the surveys. Statisticians have ways of dealing with these kinds of scenarios.

Thanks for your question -Ben

Lingxuan Zhang

zlx_1995 — 2018-04-09 08:27:29 UTC

Do you think the art therapy can apply in developed countries?

A: I know art therapy is used in a wide range of situations, including post-conflict states like Liberia. If you even just google 'arts therapy in post-conflict state' you will find things to investigate. There is certainly both activity and research into it. Peace building and post-conflict trauma treatment are entire fields of research that likely have looked into it. Just to be clear, the rigorous study conducted by Innovations for Poverty Action on the STYL intervention that we looked at before we talked about music was conducted using cognitive-behavioral therapy, which is a psychological technique that is not art therapy.

Thanks for your question! -Ben

Mark Thomson

fitzroy66 — 2018-04-09 08:28:27 UTC

Given that influencers are all the range, why does Spotify use algorithms instead of people to choose content?

A: Sort of. Brands, labels, people, all sorts of entities create Spotify playlists, but Apple Music focusses on having celebrity curators, and Spotify focusses on having lots of algorithmic features. Apple uses algorithms too. I think you are getting confused by some foci, or possibly by how Pandora's algorithm uses humans listening to songs, whereas Spotify uses machine learning. Spotify has plenty of humans (and brands and companies) creating playlists.

Thanks for your question! -Ben

Jie Li

barurujaney — 2018-04-09 08:28:40 UTC

I also believe in power of arts, but how do you evaluate the impact of one single song? Could you tell the indicators in evaluating a specific song like ?

A: I think you might have misunderstood the points I was trying to point out about that specific Song For Hawa case, in which there was no evaluation strategy or indicators to look at. How do you evaluate the impact of one single song? Well, it would depend what the 'impact' is, but the point is that it is very tricky to do in a metric style. To measure behaviour change effectively like in the STYL case, it took 8 years, hundreds of thousands of dollars, and hundreds of indicators.

But just to throw ideas out, some indicators you could measure over time through surveys or interviews would be: Is rape a taboo subject in general discussion? Would you feel comfortable if someone in your family was raped? Have you helped someone who was raped go to the hospital and the police?

There are probably many other methods and indicators you could use. The point is for that example, there was nothing from the NGO, and finding them/constructing them without a strategy in place from the beginning is very difficult.

Thanks for your question! -Ben

Yin Huang

yinh6 — 2018-04-09 08:28:42 UTC

what's the tangible change in behaviour after cultural texts such as the anti-rape song be popularize in Liberia?

A: This is precisely the question that was not possible to answer given the data I was able to find. The evidence I was able to use to try and answer this was mainly aesthetical and anecdotal stories, so it is not very strong evidence. One of the main takeaways is that you need to collect data yourself in Liberia, as there was very little public information. But it applies everywhere: to do a good evaluation you need to take baseline data and have a strategy (indicators and methods). when you start

Thanks for your question -Ben

Chen Quan

mgong1 — 2018-04-09 08:29:33 UTC

Is there any ethic problem when you make the evaluation, for example follow up the condition of patients?

I am not 100% certain, but since the STYL evaluation was academic research, it likely involved an ethics oversight at the University level. There are many ethical concerns in international development and in particular with this kind of intense intervention. Most NGO interventions do not involve academic research like this, so I would guess this sort of evaluation involves more academic oversight. Their methods are covered in detail in the publication at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868 Thanks for your question! -Ben

Yangyang Lin

Lin66 — 2018-04-09 08:30:45 UTC

Any specific findings in terms of gender

A: The STYL program was an intervention for young men only. There isn't much data on the Song For Hawa to look at.

Thanks for your question! -Ben

Yuhang Du

yuhangd — 2018-04-09 08:31:23 UTC

The art is absolutely important and have influence to people, but i guess the social and cultural impact of it is still hard to be evaluate, especially when such evaluation happened in a very particular form, for example for a particular song (in your case) or a movie/ painting in other cases. can you give more explain about how to take such issues?

A: There are lots of ways to evaluate policy, or a project. Natalia went over a lot of the different kinds of things to measure in a broad sense. The best thing you can do is to look at an impact report from an organisation to see how they argue for their impact. Here's just one example https://www.melbourne.vic.gov.au/SiteCollectionDocuments/economic-impact-melbourne-arts.pdf

Thanks for your question! - Ben

Nan Qin

nanq — 2018-04-09 08:31:42 UTC

As you mentioned in the part of music softwares. They tend to collect data for the exact recommendation to users and classify people. Do you agree that music will be more and more commercial with the "help" of data and unique music can hardly survive?

A: Not at all. For one thing, music will NEVER die. The idea that a technology can 'kill' music or make it somehow homogenous is a rhetorical idea that is usually pushed by groups who have power who are threatened by technologies. Or perhaps by artists who are worried about changing tastes and styles. At least in the Western countries, we have seen popular music change dramatically over the last century. Music changes, but it will take a lot more than technology.

Another perspective: if commercial systems and recommendations get better at making music more commercial / effective at targetting listeners, there will be opportunities for musicians to create music outside of that system to offer an alternative.

Technology rarely controls us the way we think, and a lot of innovation (including file sharing) comes from unintended use. People find ways to make things interesting and use technology in the ways they want, not in the ways that companies always want them to.

I think datafication and metrics will change what music is popular, but I don't think that what is popular ever stopped artists from doing things that they wanted to. That's part of why music is so interesting - it can be very surprising and unpredictable. Even with all these tools, major record labels still can't really predict what is going to be huge and what is going to fail. There are sites claiming to be able to do this, but they aren't terribly effective at predicting. They can just track things really well.
i.e. https://www.nextbigsound.com/

Thanks for your question! -Ben

Xiaotong XU

XIAOTONGXU — 2018-04-09 08:32:54 UTC

Just for curious, why you choose this country as the example?

A: I was talking about things from my own personal experience. I worked on the IPA evaluation of STYL in 2011/2012, and I was a fan of Takun J and wrote a paper on Song For Hawa in 2015. My interest in Liberia comes from living there. The country chose me!

Thanks for your question -Ben

From Haitao Li

278054354 — 2018-04-09 08:33:35 UTC

How to evaluate the influence of culture is difficult as culture is an abstract object. Therefore, in oder to gain specific data used in analyzing cultural impact, some academic research methods can be adopted. In view of choosing analysis methods, what does an art company or organisation need to consider?

A: I would say any data you collect is to answer a question. If the organisation wants to know: "Do people enjoy what we are offering?" that might involve talking to people. If you want to know if it has an effect on the community, that is something different. Natalia's lecture covered a lot of different kinds of indicators, but the art of focussing on what indicators will best answer certain questions is evolving. People are still finding new methods and indicators to try to answer age-old questions like: "Is our program appealing to everyone, or just a certain demographic?"

Thanks for your question! -Ben

Xiaoshu Wu

xiaoshuWU — 2018-04-09 08:33:39 UTC

How do you seek and select the target group? Did you try to find other groups paying attention to other perspective rather than just critical social issues.

A: The STYL program targeted criminals and young men who were not part of normal society. You can read more about recruitment and methods in the paper if you are interested at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868

Thanks for your question -Ben

tingtingz6 — 2018-04-09 08:38:57 UTC

Tingting Zhang
I am impressed by the positive influence by Art. The analysis of culture is valuable. And I think Maybe we
could add psychologists to this projects because they can get more and deeper information from statistics.

A: Psychologists were a big part of STYL because it was cognitive behavioural therapy. But that was not an arts/culture intervention.
Sure, more experts from different fields can make research stronger. It also makes it more expensive. Psychologists are involved in certain kinds of evaluations, but project and policy evaluation is a different sort of skill than being able to treat mental conditions.
Thanks for your question -Ben

YUchao WANG

yuchaow2 — 2018-04-09 08:39:08 UTC

With regard to the arts therapy, how does it influence the people in a specific way. What I meant is just what is the correlation of the people and arts therapy

A: Arts therapy is a different kind of intervention. If you mena the Song For Hawa, I was unable to prove anything about behaviour change with the limited data.

If you mean the STYL intervention of cognitive-behavioural therapy (not arts therapy) then you can read more in the paper at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2594868

or here is a blog style report about it:
https://www.washingtonpost.com/news/monkey-cage/wp/2015/04/15/jobs-and-jail-might-not-keep-young-men-out-of-crime-but-how-about-therapy/?utm_term=.93a5f124d7cb

Remember, 'therapy' in that case does not mean arts therapy, it means psychological treatment. That was a very different kind of intervention.

Thanks for your question -Ben

Muyun Zhang

MuyunZhang — 2018-04-09 08:40:44 UTC

I want to know more music therapy application examples of other countries, especially developed countries.

A: I am really only familiar with Liberia, but I know that arts therapy is being used in many scenarios, and is probably more likely to be well-researched in developed countries. That would be Kim Dunphy's area. I went into more detail about this in a similar question.
Thanks for your question -Ben

Xiangyu Xu

barurujaney — 2018-04-09 08:43:03 UTC

To be honest, when I listen to the for the first time, I may be attracted by its happy melody rather than its serious lyrics. The social problems these songs reflect should be paid attention to, but there’s risk that people may ignore its lyrics, do you think it is detrimental for its promotion and expression?

A: One thing important to keep in mind is that different cultures (and people) create meaning and value around music in different ways. What might sound happy to you might not have the same effect on others. But even if the song sounds 'happy', there is still the question of: how do you try to talk about something so uncomfortable in a song? Would making it sad or angry work differently to get people to pay attention? Perhaps, but it is really more of an artistic creative strategy than something we can try to standardize. Music works in mysterious ways, as they say.... Trying to get a very serious and difficult topic across in song is probably going to be hard no matter what you do. We can't really be sure how ultimately effective it was at changing behaviour, which is why the aesthetic evaluation is all we have to go on. I bet it would be interesting to ask Takun J if he wanted the song to sound happy. Thanks for your question! -Ben

Yitian Chen

yitianc — 2018-04-09 09:04:33 UTC

Do you think this is the most successful example in music therapy ?

A: -Music therapy is something a bit different than this example. I called this 'cultural diplomacy' from one value system to another when I wrote the paper in 2015, but to use the NGO terms it is 'Entertainment-Education' - it is a social change intervention. Music therapy is not my area, but I think of thereapeutic treatment as a different process than a mass media delivery of a message like 'Song For Hawa'. Whether Song For Hawa can even be considered 'successful' is open to debate and difficult to provide strong evidence for, outside of an aesthetic narrative. Thanks for your question! -Ben