# data[].attributes.body
How likely are BigData rich institutions to audit & be critical of WMD developers?

# data[].attributes.body
The question of security with these automated systems is a topic I’d be interested in exploring. Is big data easier to hack and control through adversarial means? Is this an area of research? What kind of power could hacking utilize in an increasingly automated world? Are there examples of sidestepping an algorithm meant to mediate access/approval? This could be more than signal jamming, as we saw adversarial research targeting image-recognition networks, hackers can create desirable certainties in an AI’s eyes.

# data[].attributes.body
1. O’Neil suggests that “models that have a significant impact on our lives, including credit scores and e-scores, should be open and available to the public. Ideally, we could navigate them at the level of an app on our phones.” [6, p. 180] But Langdon Winner debates on this statement that we cannot expect information technologies act as “great equalizers” because it requires “people’s willingness to act together in pursuit of their common ends” [7, p. 111]. How could we extend O’Neil’s claims of equality and trust without falling into Mythinformation?

2. O’Neil described a project to create accountability of big data in the city of New York. The government of the city did not welcome the results and demanded the results take down [6, p. 182]. What do you think a research can do to translate (in terms of Latour) this kind of research and find better results?

# data[].attributes.body
O’Neill mentions using the threat of brand damage to make companies assume pro-social accountability processes, like letting an algorithm inspect their supply chain for exploitative labor. How has this approach to corporate accountability worked out for us so far? Cancel, boycott?

https://foundation.mozilla.org/en/blog/congratulations-youtube-now-show-your-work/

# data[].attributes.body
1. I liked the interventions that O’Neil laid out in her conclusion. How do we actually go about implementing them, particularly when WMDs accumulate power at a faster rate than we have been able to grapple with them?
2. O’Neil’s suggestion that we embed values into our algorithms is a nice one, but also technically fraught. Who gets to decide which values are embedded, and what does it really mean to “embed” an abstract idea? We have a burgeoning field of AI fairness, but it is focused around very literal mathematical constructs of fairness, and doesn’t feel like it has the broad imaginative power that O’Neil is looking for.

# data[].attributes.body
1. O'Neill says, “As human beings learn and adapt, we change, and so do our processes. Automated systems, by contrast, stay stuck in time until engineers dive in to change them.” However, this statement seems to suggest that humans will evolve in ways that increase equality and liberty for others rather than the opposite. We also know that humans don’t change unless their environment changes, so it seems less an issue of ability to change as it is an issue of scale. Technologies have codified the perspective of a network of individuals at a large scale that then persists throughout time usually due to market forces and the economic inefficiency of dismantling those systems.

Where do resources need to be put in our society in order for us to responsibly innovate?

2. Does the diversity issue in tech mean that MORE voices must enter into the conversation or that some voices need to replace others? What is the ideal balance of ideologies and methods on a Big Data team?

# data[].attributes.body
Q2. There is a huge gulf between institutions when it comes to accessing Big Data resources. But the Matthew effect, as Merton puts it, has been a contentious aspect of scientific research practice for a long time. Is there a possibility of the Open Science movement narrowing such divides in the context of accessing Big Data resources? Or is it too difficult to get the centers of knowledge production on board?
Q3. How do we include more critical voices in the development of predictive algorithms? What are the structural limitations that need to be overcome?

# data[].attributes.body
1. I’m curious how much money is made by the police with nuisance crimes, either that which comes from bail, fines or other sources. Wouldn’t the context of police department short funding be both the reasons for using predictive policing AND for especially targeting nuisance crimes? This might be a bit cynical, but, I doubt “high-level” crimes pay the police department bills the way “low-level” crimes do...
2. If data scientists are so sure of the factors that increase the likelihood that someone will commit a crime (poverty, unemployment, lack of high school degree, friends have been criminalized), why don’t they focus more attention on changing those factors, rather than targeting folks who live with those circumstances as criminals?
3. It’s surprising that for all of O’Neill’s analysis, the other side to this analysis (that police officers historically have taken on more and more work that previously was the purvey of civic agents such as mental health care or social workers) is not taken up ... O’Neill addresses this in part with “We need to think about how we assign blame in modern life and how models exacerbate this cycle.”
4. How can the “hidden costs” of WMDs be measured and accounted for? These hidden costs seem even harder to qualify and quantify than environmental and social devastation from market driven operations involving mining, oil production-sale-consumption, and a long list of other capitalist ventures...

# data[].attributes.body
Boyd/Crawford:
Throughout the paper, the authors illustrate the impact of the tool or machine on the worker, on the industry, and on the larger idea of labor. How can we take back agency in our work while using computational tools? Or do we accept that our tools will always shape our work, and we must observe and record those influences?

O'Neil:
We as society are used to black boxes, and accepting of their presence and impact. How can we communicate the input, output, and inner workings of black boxes without overwhelming people? Is it reasonable to trust in certain black boxes?

# data[].attributes.body
1. Both boyd and Crawford point to the obfuscating character of Big Data. AIs hide the biases inherent in their models by appealing to mathematics and an apparent objectivity without considering the flaws inherent in these machines or in the data sets. Further, biases in who gets access to the data are critically important. Citing Derrida, they point to the fact that democratization requires, essentially, “participation in and access to the archive, its constitution, and its interpretation.” (674) But is this in principle a possibility? The point of artificial intelligences, their mystique and attraction, relies on the fact that the decision-making algorithms themselves are too complex for humans to understand. Can meaningful participation and interpretation in this sort of archive be democratized? If not, should (and can) we do away with Big Data in general, even modified (more humane and ethical) versions of it?

2. In the 9 years since the boyd and Crawford wrote their article, the panorama may have changed, somewhat. For example, the 2019 AI Now report (of which Crawford is a notable contributor), ethical concerns, as well as the acknowledgement of bias in data sets seem to be much more common – perhaps diminishing (if only in certain sectors of society) the mythological character of which their article speaks. But can this mythological character be fully eliminated, given the size of the data sets and the complexity of the algorithms?

3. O’Neil argues that to dismantle WMDs, we should regulate the mathematical models that inform them, and that human values, such as success, or fairness, need to be reexamined and be made part of the process. But, as she also claims, “Big Data processes codify the past. They do not invent the future” (169), and values change – and rapidly. How could we account for the change of values in a society without needing to rebuild these systems from scratch at ever-increasing speeds? Further, values also change from place to place, and a great number of WMDs operate globally. How would we regulate them in such a way that we are not imposing values on other societies which are also subject to these machines?