# data[].attributes.body
2016

# data[].attributes.headline
Lo and Behold: Reveries of the Connected World

# data[].attributes.subject
Lo and Behold: Reveries of the Connected World

# data[].attributes.body
Documentary by Werner Herzog 2016

# data[].attributes.attachment_link.provider_name
Search for screenings / showtimes and book tickets for Lo and Behold, Reveries of the Connected World. See the release date and trailer. The Official Showtimes Destination brought to you by Dogwoof Ltd #LoandBehold

# data[].attributes.attachment_link.keywords[]
lo and behold reveries of the connected world

# data[].attributes.attachment_link.title
Lo and Behold, Reveries of the Connected World: Cinema Screenings & Ticket Booking - The Official Showtimes Destination

# data[].attributes.attachment_link.description
Search for screenings / showtimes and book tickets for Lo and Behold, Reveries of the Connected World. See the release date and trailer. The Official Showtimes Destination brought to you by Dogwoof Ltd #LoandBehold

# data[].attributes.headline
Vilém Flusser, Interview about technical revolution, 1988

# data[].attributes.subject
Vilém Flusser, Interview about technical revolution, 1988

# data[].attributes.attachment_link.title
Vilém Flusser - 1988 interview about technical revolution (intellectual level is lowering)

# data[].attributes.attachment_link.readable_text
When we look at the rise of artificial intelligence, it's easy to get carried away with dystopian visions of sentient machines that rebel against their human creators. Fictional baddies such as the Terminator's Skynet or Hal from 2001: A Space Odyssey have a lot to answer for.

However, the real risk posed by AI - at least in the near term - is much more insidious. It's far more likely that robots would inadvertently harm or frustrate humans while carrying out our orders than they would become conscious and rise up against us. In recognition of this, the University of California, Berkeley has this week launched a center to focus on building people-pleasing AIs.

The Center for Human-Compatible Artificial Intelligence, launched this week with $5.5m in funding from the Open Philanthropy Project, is led by computer science professor and artificial intelligence pioneer Stuart Russell. He's quick to dispel any "unreasonable and melodramatic" comparisons to the threats posed in science fiction.

"The risk doesn't come from machines suddenly developing spontaneous malevolent consciousness," he said. "It's important that we're not trying to prevent that from happening because there's absolutely no understanding of consciousness whatsoever."

Russell is well known in the artificial intelligence community and in 2015 penned an open letter calling for researchers to look beyond the goal of simply making AI more capable and powerful to think about maximizing its social benefit. The letter has been signed by more than 8,000 scientists and entrepreneurs including physicist Stephen Hawking, entrepreneur Elon Musk and Apple co-founder Steve Wozniak.

"The potential benefits [of AI research] are huge, since everything that civilization has to offer is a product of human intelligence; we cannot predict what we might achieve when this intelligence is magnified by the tools AI may provide, but the eradication of disease and poverty are not unfathomable," the letter reads.

"Because of the great potential of AI, it is important to research how to reap its benefits while avoiding potential pitfalls."

It's precisely this thinking that underpins the new center.

Up until now, AI has primarily been applied to very limited contexts such as playing Chess or Go or recognizing objects in images, where there isn't much scope for the system to do much damage. As they start to make decisions on our behalf within the real world, the stakes are much higher.

"As soon as you put things in the real world, with self-driving cars, digital assistants ... as soon as they buy things on your behalf, turn down appointments, then they have to align with human values," Russell said.

He uses autonomous vehicles to illustrate the type of problem the center will try to solve. Someone building a self-driving car might instruct it never to go through a red light, but the machine might then hack into the traffic light control system so that all of the lights are changed to green. In this case the car would be obeying orders but in a way that humans didn't expect or intend. Similarly, an artificially intelligent hedge fund designed to maximize the value of its portfolio could be incentivized to short consumer stocks, buy long on defence stocks and then start a war - as suggested by Elon Musk in Werner Herzog's latest documentary.

"Even when you think you've put fences around what an AI system can do it will tend to find loopholes just as we do with our tax laws. You want an AI system that isn't motivated to find loopholes," Russell said.

"The problem isn't consciousness, but competence. You make machines that are incredibly competent at achieving objectives and they will cause accidents in trying to achieve those objectives."

To address this, Russell and his colleagues at the center propose making AI systems that observe human behavior and try to work out what the human's objective is, then behave accordingly and learn from mistakes. So instead of trying to give the machine a long list of rules to follow, the machine is told that its main objective is to do what the human wants them to do.

It sounds simple, but it's not how engineers have been building systems for the past 50 years.

But if AI systems can be designed to learn from humans in this way, it should ensure that they remain under human control even when they develop capabilities that exceed our own.

In addition to watching humans directly using cameras and other sensors, robots can learn about us by reading history books, legal documents, novels, newspaper stories as well as by watching videos and movies. From this they can start to build up an understanding of human values.

It won't be easy for machines. "People are irrational, inconsistent, weak-willed, computationally limited, heterogenous and sometimes downright evil," Russell said.

"Some are vegetarians and some really like a nice juicy steak. And the fact that we don't behave anything close to perfectly is a serious difficulty."

# data[].attributes.attachment_link.title
The rise of robots: forget evil AI - the real risk is far more insidious

# data[].attributes.attachment_link.description
When we look at the rise of artificial intelligence, it's easy to get carried away with dystopian visions of sentient machines that rebel against their human creators. Fictional baddies such as the Terminator's Skynet or Hal from 2001: A Space Odyssey have a lot to answer for.

# data[].attributes.attachment_link.readable_text
SAN FRANCISCO (Reuters) - Amazon.com Inc's machine-learning specialists uncovered a big problem: their new recruiting engine did not like women.

The team had been building computer programs since 2014 to review job applicants' resumes with the aim of mechanizing the search for top talent, five people familiar with the effort told Reuters.

Automation has been key to Amazon's e-commerce dominance, be it inside warehouses or driving pricing decisions. The company's experimental hiring tool used artificial intelligence to give job candidates scores ranging from one to five stars - much like shoppers rate products on Amazon, some of the people said.

"Everyone wanted this holy grail," one of the people said. "They literally wanted it to be an engine where I'm going to give you 100 resumes, it will spit out the top five, and we'll hire those."

But by 2015, the company realized its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way.

That is because Amazon's computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry. (For a graphic on gender breakdowns in tech, see: tmsnrt.rs/2OfPWoD)

In effect, Amazon's system taught itself that male candidates were preferable. It penalized resumes that included the word "women's," as in "women's chess club captain." And it downgraded graduates of two all-women's colleges, according to people familiar with the matter. They did not specify the names of the schools.

Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said.

The Seattle company ultimately disbanded the team by the start of last year because executives lost hope for the project, according to the people, who spoke on condition of anonymity. Amazon's recruiters looked at the recommendations generated by the tool when searching for new hires, but never relied solely on those rankings, they said.

Amazon declined to comment on the technology's challenges, but said the tool "was never used by Amazon recruiters to evaluate candidates." The company did not elaborate further. It did not dispute that recruiters looked at the recommendations generated by the recruiting engine.

The company's experiment, which Reuters is first to report, offers a case study in the limitations of machine learning. It also serves as a lesson to the growing list of large companies including Hilton Worldwide Holdings Inc and Goldman Sachs Group Inc that are looking to automate portions of the hiring process.

Some 55 percent of U.S. human resources managers said artificial intelligence, or AI, would be a regular part of their work within the next five years, according to a 2017 survey by talent software firm CareerBuilder.

Employers have long dreamed of harnessing technology to widen the hiring net and reduce reliance on subjective opinions of human recruiters. But computer scientists such as Nihar Shah, who teaches machine learning at Carnegie Mellon University, say there is still much work to do.

"How to ensure that the algorithm is fair, how to make sure the algorithm is really interpretable and explainable - that's still quite far off," he said.

MASCULINE LANGUAGE

Amazon's experiment began at a pivotal moment for the world's largest online retailer. Machine learning was gaining traction in the technology world, thanks to a surge in low-cost computing power. And Amazon's Human Resources department was about to embark on a hiring spree: Since June 2015, the company's global headcount has more than tripled to 575,700 workers, regulatory filings show.

The logo of Amazon is seen at the company logistics centre in Boves, France, August 8, 2018. REUTERS/Pascal Rossignol/Files

So it set up a team in Amazon's Edinburgh engineering hub that grew to around a dozen people. Their goal was to develop AI that could rapidly crawl the web and spot candidates worth recruiting, the people familiar with the matter said.

The group created 500 computer models focused on specific job functions and locations. They taught each to recognise some 50,000 terms that showed up on past candidates' resumes. The algorithms learned to assign little significance to skills that were common across IT applicants, such as the ability to write various computer codes, the people said.

Instead, the technology favoured candidates who described themselves using verbs more commonly found on male engineers' resumes, such as "executed" and "captured," one person said.

Gender bias was not the only issue. Problems with the data that underpinned the models' judgments meant that unqualified candidates were often recommended for all manner of jobs, the people said. With the technology returning results almost at random, Amazon shut down the project, they said.

THE PROBLEM, OR THE CURE?

Other companies are forging ahead, underscoring the eagerness of employers to harness AI for hiring.

Kevin Parker, chief executive of HireVue, a startup near Salt Lake City, said automation is helping firms look beyond the same recruiting networks upon which they have long relied. His firm analyses candidates' speech and facial expressions in video interviews to reduce reliance on resumes.

"You weren't going back to the same old places; you weren't going back to just Ivy League schools," Parker said. His company's customers include Unilever PLC and Hilton.

Goldman Sachs has created its own resume analysis tool that tries to match candidates with the division where they would be the "best fit," the company said.

Microsoft Corp's LinkedIn, the world's largest professional network, has gone further. It offers employers algorithmic rankings of candidates based on their fit for job postings on its site.

Still, John Jersin, vice president of LinkedIn Talent Solutions, said the service is not a replacement for traditional recruiters.

"I certainly would not trust any AI system today to make a hiring decision on its own," he said. "The technology is just not ready yet."

Some activists say they are concerned about transparency in AI. The American Civil Liberties Union is currently challenging a law that allows criminal prosecution of researchers and journalists who test hiring websites' algorithms for discrimination.

"We are increasingly focusing on algorithmic fairness as an issue," said Rachel Goodman, a staff attorney with the Racial Justice Program at the ACLU.

Still, Goodman and other critics of AI acknowledged it could be exceedingly difficult to sue an employer over automated hiring: Job candidates might never know it was being used.

As for Amazon, the company managed to salvage some of what it learned from its failed AI experiment. It now uses a "much-watered down version" of the recruiting engine to help with some rudimentary chores, including culling duplicate candidate profiles from databases, one of the people familiar with the project said.

Another said a new team in Edinburgh has been formed to give automated employment screening another try, this time with a focus on diversity.

Reporting By Jeffrey Dastin in San Francisco; Editing by Jonathan Weber and Marla Dickerson

Our Standards: The Thomson Reuters Trust Principles.

# data[].attributes.attachment_link.title
Insight - Amazon scraps secret AI recruiting tool that showed bias against women

# data[].attributes.attachment_link.description
Amazon.com Inc's <AMZN.O> machine-learning specialists uncovered a big problem: their new recruiting engine did not like women.

# data[].attributes.attachment_link.title
'Internet meme ruined my career' BBC News

# data[].attributes.headline
Crowd takes selfies with victim of lynch mob

# data[].attributes.subject
Crowd takes selfies with victim of lynch mob

# data[].attributes.attachment_link.readable_text
Police in the south Indian state of Kerala have arrested two people after a video of a mob lynching a man accused of theft went viral.

Footage of the incident horrified Indians, along with images of people taking selfies while watching the man, who was tied up, being killed.

The victim was identified as a tribesman who lived in the area.

Police told the BBC that they were actively searching for other people involved in the murder.

Senior police official Prateesh Kumar said a team had rushed to the spot in the state's Palakkad district after receiving information that a man was being attacked.

They managed to halt the attack and rushed the man, named Madhu, to hospital. However, he died before he could receive any medical attention, Mr Kumar added.

There was widespread horror in Kerala when graphic footage of the incident emerged, and some of it was broadcast on TV.

Popular Kerala actor Mammooty made an impassioned statement on Facebook about the incident.

"Madhu's death is because our system is responsible... a system which has ordered mob justice and a person who attacks another is no human. How can we declare ourselves modern and progressive?" his post read.

State chief minister Pinarayi Vijayan said that "strict action" would be taken against the attackers and that the incident was a "blot" on Kerala's progressive society.

"The worst thing that one person can do to another is to take a selfie in such a situation," tribal activist Dhanya Raman told BBC Hindi.

# data[].attributes.attachment_link.title
Indian lynch mob take selfies with victim

# data[].attributes.attachment_link.description
Police in the south Indian state of Kerala have arrested two people after a video of a mob lynching a man accused of theft went viral. Footage of the incident horrified Indians, along with images of people taking selfies while watching the man, who was tied up, being killed.

# data[].attributes.headline
Rise of societal public shaming through modern technology and image sharing

# data[].attributes.subject
Rise of societal public shaming through modern technology and image sharing

# data[].attributes.attachment_link.title
Public shaming, social media, ruined lives - CNN

# data[].attributes.attachment_link.description
Do you believe in forgiveness? Do you believe in second chances? Maybe in person -- but in the Internet era, it's so easy to pile on and destroy people.

# data[].attributes.headline
Internetting with Amanda Hess - New York Times Series

# data[].attributes.subject
Internetting with Amanda Hess - New York Times Series

# data[].attributes.attachment_link.description
White supremacists used to hide their faces under hoods. But today's internet-savvy racists are shielding themselves behind the plausible deniability of meme culture.

# data[].attributes.headline
Power of too much pathos in Visual Culture: ASPCA commercials

# data[].attributes.subject
Power of too much pathos in Visual Culture: ASPCA commercials

# data[].attributes.attachment_link.readable_text
Even Sarah McLachlan changes the channel when her notoriously gut-wrenching ASPCA commercials come on.

The famed singer has lent her voice to numerous commercials sponsored by the American Society For The Prevention Of Cruelty to Animals that feature sad-looking animals desperate for a home. The PSAs have taken on a life of their own because of how hard they can be to watch. When she spoke with HuffPost Live's Caitlyn Becker about the release of her new album, "Shine On," McLachlan added that she too avoids the commercials.

"I change the channel. I can't take it," McLachlan said. "I can't even look at it. It's just so depressing."

She added that even though these ads generated more than $30 million for the ASCPA, she doesn't want her legacy to only include these tear-jerking commercials.

"Do I want to be remembered just for that? Absolutely not," McLachlan said. "I mean, I love animals absolutely; but, if I want to think about what my legacy would be, it would be more about kids and music, which is sort of my true passion."

Watch the full HuffPost Live interview with Sarah McLachlan here.

# data[].attributes.attachment_link.title
Sarah McLachlan Admits Even She Can't Watch Her ASPCA Sad Dog Commercials

# data[].attributes.attachment_link.description
Even Sarah McLachlan changes the channel when her notoriously gut-wrenching ASPCA commercials come on. The famed singer has lent her voice to numerous commercials sponsored by the American Society For The Prevention Of Cruelty to Animals that feature sad-looking animals desperate for a home.

# data[].attributes.attachment_link.readable_text
On a spring afternoon in 2014, Brisha Borden was running late to pick up her god-sister from school when she spotted an unlocked kid's blue Huffy bicycle and a silver Razor scooter. Borden and a friend grabbed the bike and scooter and tried to ride them down the street in the Fort Lauderdale suburb of Coral Springs.

Just as the 18-year-old girls were realizing they were too big for the tiny conveyances - which belonged to a 6-year-old boy - a woman came running after them saying, "That's my kid's stuff." Borden and her friend immediately dropped the bike and scooter and walked away.

But it was too late - a neighbor who witnessed the heist had already called the police. Borden and her friend were arrested and charged with burglary and petty theft for the items, which were valued at a total of $80.

Subscribe to the Series

Machine Bias: Investigating the algorithms that control our lives.

Compare their crime with a similar one: The previous summer, 41-year-old Vernon Prater was picked up for shoplifting $86.35 worth of tools from a nearby Home Depot store.

Prater was the more seasoned criminal. He had already been convicted of armed robbery and attempted armed robbery, for which he served five years in prison, in addition to another armed robbery charge. Borden had a record, too, but it was for misdemeanors committed when she was a juvenile.

Yet something odd happened when Borden and Prater were booked into jail: A computer program spat out a score predicting the likelihood of each committing a future crime. Borden - who is black - was rated a high risk. Prater - who is white - was rated a low risk.

Two years later, we know the computer algorithm got it exactly backward. Borden has not been charged with any new crimes. Prater is serving an eight-year prison term for subsequently breaking into a warehouse and stealing thousands of dollars' worth of electronics.

Scores like this - known as risk assessments - are increasingly common in courtrooms across the nation. They are used to inform decisions about who can be set free at every stage of the criminal justice system, from assigning bond amounts - as is the case in Fort Lauderdale - to even more fundamental decisions about defendants' freedom. In Arizona, Colorado, Delaware, Kentucky, Louisiana, Oklahoma, Virginia, Washington and Wisconsin, the results of such assessments are given to judges during criminal sentencing.

Rating a defendant's risk of future crime is often done in conjunction with an evaluation of a defendant's rehabilitation needs. The Justice Department's National Institute of Corrections now encourages the use of such combined assessments at every stage of the criminal justice process. And a landmark sentencing reform bill currently pending in Congress would mandate the use of such assessments in federal prisons.

In 2014, then U.S. Attorney General Eric Holder warned that the risk scores might be injecting bias into the courts. He called for the U.S. Sentencing Commission to study their use. "Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice," he said, adding, "they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society."

The sentencing commission did not, however, launch a study of risk scores. So ProPublica did, as part of a larger examination of the powerful, largely hidden effect of algorithms in American life.

We obtained the risk scores assigned to more than 7,000 people arrested in Broward County, Florida, in 2013 and 2014 and checked to see how many were charged with new crimes over the next two years, the same benchmark used by the creators of the algorithm.

The score proved remarkably unreliable in forecasting violent crime: Only 20 percent of the people predicted to commit violent crimes actually went on to do so.

When a full range of crimes were taken into account - including misdemeanors such as driving with an expired license - the algorithm was somewhat more accurate than a coin flip. Of those deemed likely to re-offend, 61 percent were arrested for any subsequent crimes within two years.

We also turned up significant racial disparities, just as Holder feared. In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways.

The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants.

White defendants were mislabeled as low risk more often than black defendants.

Could this disparity be explained by defendants' prior crimes or the type of crimes they were arrested for? No. We ran a statistical test that isolated the effect of race from criminal history and recidivism, as well as from defendants' age and gender. Black defendants were still 77 percent more likely to be pegged as at higher risk of committing a future violent crime and 45 percent more likely to be predicted to commit a future crime of any kind. ( Read our analysis.)

The algorithm used to create the Florida risk scores is a product of a for-profit company, Northpointe. The company disputes our analysis.

In a letter, it criticized ProPublica's methodology and defended the accuracy of its test: "Northpointe does not agree that the results of your analysis, or the claims being made based upon that analysis, are correct or that they accurately reflect the outcomes from the application of the model."

Northpointe's software is among the most widely used assessment tools in the country. The company does not publicly disclose the calculations used to arrive at defendants' risk scores, so it is not possible for either defendants or the public to see what might be driving the disparity. (On Sunday, Northpointe gave ProPublica the basics of its future-crime formula - which includes factors such as education levels, and whether a defendant has a job. It did not share the specific calculations, which it said are proprietary.)

Northpointe's core product is a set of scores derived from 137 questions that are either answered by defendants or pulled from criminal records. Race is not one of the questions. The survey asks defendants such things as: "Was one of your parents ever sent to jail or prison?" "How many of your friends/acquaintances are taking drugs illegally?" and "How often did you get in fights while at school?" The questionnaire also asks people to agree or disagree with statements such as "A hungry person has a right to steal" and "If people make me angry or lose my temper, I can be dangerous."

The appeal of risk scores is obvious: The United States locks up far more people than any other country, a disproportionate number of them black. For more than two centuries, the key decisions in the legal process, from pretrial release to sentencing to parole, have been in the hands of human beings guided by their instincts and personal biases.

If computers could accurately predict which defendants were likely to commit new crimes, the criminal justice system could be fairer and more selective about who is incarcerated and for how long. The trick, of course, is to make sure the computer gets it right. If it's wrong in one direction, a dangerous criminal could go free. If it's wrong in another direction, it could result in someone unfairly receiving a harsher sentence or waiting longer for parole than is appropriate.

The first time Paul Zilly heard of his score - and realized how much was riding on it - was during his sentencing hearing on Feb. 15, 2013, in court in Barron County, Wisconsin. Zilly had been convicted of stealing a push lawnmower and some tools. The prosecutor recommended a year in county jail and follow-up supervision that could help Zilly with "staying on the right path." His lawyer agreed to a plea deal.

But Judge James Babler had seen Zilly's scores. Northpointe's software had rated Zilly as a high risk for future violent crime and a medium risk for general recidivism. "When I look at the risk assessment," Babler said in court, "it is about as bad as it could be."

Then Babler overturned the plea deal that had been agreed on by the prosecution and defense and imposed two years in state prison and three years of supervision.

Criminologists have long tried to predict which criminals are more dangerous before deciding whether they should be released. Race, nationality and skin color were often used in making such predictions until about the 1970s, when it became politically unacceptable, according to a survey of risk assessment tools by Columbia University law professor Bernard Harcourt.

In the 1980s, as a crime wave engulfed the nation, lawmakers made it much harder for judges and parole boards to exercise discretion in making such decisions. States and the federal government began instituting mandatory sentences and, in some cases, abolished parole, making it less important to evaluate individual offenders.

But as states struggle to pay for swelling prison and jail populations, forecasting criminal risk has made a comeback.

Dozens of risk assessments are being used across the nation - some created by for-profit companies such as Northpointe and others by nonprofit organizations. (One tool being used in states including Kentucky and Arizona, called the Public Safety Assessment, was developed by the Laura and John Arnold Foundation, which also is a funder of ProPublica.)

There have been few independent studies of these criminal risk assessments. In 2013, researchers Sarah Desmarais and Jay Singh examined 19 different risk methodologies used in the United States and found that "in most cases, validity had only been examined in one or two studies" and that "frequently, those investigations were completed by the same people who developed the instrument."

Their analysis of the research through 2012 found that the tools "were moderate at best in terms of predictive validity," Desmarais said in an interview. And she could not find any substantial set of studies conducted in the United States that examined whether risk scores were racially biased. "The data do not exist," she said.

Since then, there have been some attempts to explore racial disparities in risk scores. One 2016 study examined the validity of a risk assessment tool, not Northpointe's, used to make probation decisions for about 35,000 federal convicts. The researchers, Jennifer Skeem at University of California, Berkeley, and Christopher T. Lowenkamp from the Administrative Office of the U.S. Courts, found that blacks did get a higher average score but concluded the differences were not attributable to bias.

The increasing use of risk scores is controversial and has garnered media coverage, including articles by the Associated Press, and the Marshall Project and FiveThirtyEight last year.

Most modern risk tools were originally designed to provide judges with insight into the types of treatment that an individual might need - from drug treatment to mental health counseling.

"What it tells the judge is that if I put you on probation, I'm going to need to give you a lot of services or you're probably going to fail," said Edward Latessa, a University of Cincinnati professor who is the author of a risk assessment tool that is used in Ohio and several other states.

But being judged ineligible for alternative treatment - particularly during a sentencing hearing - can translate into incarceration. Defendants rarely have an opportunity to challenge their assessments. The results are usually shared with the defendant's attorney, but the calculations that transformed the underlying data into a score are rarely revealed.

"Risk assessments should be impermissible unless both parties get to see all the data that go into them," said Christopher Slobogin, director of the criminal justice program at Vanderbilt Law School. "It should be an open, full-court adversarial proceeding."

Black Defendants' Risk Scores

White Defendants' Risk Scores

These charts show that scores for white defendants were skewed toward lower-risk categories. Scores for black defendants were not. (Source: ProPublica analysis of data from Broward County, Fla.)

Proponents of risk scores argue they can be used to reduce the rate of incarceration. In 2002, Virginia became one of the first states to begin using a risk assessment tool in the sentencing of nonviolent felony offenders statewide. In 2014, Virginia judges using the tool sent nearly half of those defendants to alternatives to prison, according to a state sentencing commission report. Since 2005, the state's prison population growth has slowed to 5 percent from a rate of 31 percent the previous decade.

In some jurisdictions, such as Napa County, California, the probation department uses risk assessments to suggest to the judge an appropriate probation or treatment plan for individuals being sentenced. Napa County Superior Court Judge Mark Boessenecker said he finds the recommendations helpful. "We have a dearth of good treatment programs, so filling a slot in a program with someone who doesn't need it is foolish," he said.

However, Boessenecker, who trains other judges around the state in evidence-based sentencing, cautions his colleagues that the score doesn't necessarily reveal whether a person is dangerous or if they should go to prison.

"A guy who has molested a small child every day for a year could still come out as a low risk because he probably has a job," Boessenecker said. "Meanwhile, a drunk guy will look high risk because he's homeless. These risk factors don't tell you whether the guy ought to go to prison or not; the risk factors tell you more about what the probation conditions ought to be."

"I'm surprised [my risk score] is so low. I spent five years in state prison in Massachusetts." (Josh Ritchie for ProPublica)

Sometimes, the scores make little sense even to defendants.

James Rivelli, a 54-year old Hollywood, Florida, man, was arrested two years ago for shoplifting seven boxes of Crest Whitestrips from a CVS drugstore. Despite a criminal record that included aggravated assault, multiple thefts and felony drug trafficking, the Northpointe algorithm classified him as being at a low risk of reoffending.

"I am surprised it is so low," Rivelli said when told by a reporter he had been rated a 3 out of a possible 10. "I spent five years in state prison in Massachusetts. But I guess they don't count that here in Broward County." In fact, criminal records from across the nation are supposed to be included in risk assessments.

Less than a year later, he was charged with two felony counts for shoplifting about $1,000 worth of tools from Home Depot. He said his crimes were fueled by drug addiction and that he is now sober.

Northpointe was founded in 1989 by Tim Brennan, then a professor of statistics at the University of Colorado, and Dave Wells, who was running a corrections program in Traverse City, Michigan.

Wells had built a prisoner classification system for his jail. "It was a beautiful piece of work," Brennan said in an interview conducted before ProPublica had completed its analysis. Brennan and Wells shared a love for what Brennan called "quantitative taxonomy" - the measurement of personality traits such as intelligence, extroversion and introversion. The two decided to build a risk assessment score for the corrections industry.

Brennan wanted to improve on a leading risk assessment score, the LSI, or Level of Service Inventory, which had been developed in Canada. "I found a fair amount of weakness in the LSI," Brennan said. He wanted a tool that addressed the major theories about the causes of crime.

Brennan and Wells named their product the Correctional Offender Management Profiling for Alternative Sanctions, or COMPAS. It assesses not just risk but also nearly two dozen so-called "criminogenic needs" that relate to the major theories of criminality, including "criminal personality," "social isolation," "substance abuse" and "residence/stability." Defendants are ranked low, medium or high risk in each category.

As often happens with risk assessment tools, many jurisdictions have adopted Northpointe's software before rigorously testing whether it works. New York State, for instance, started using the tool to assess people on probation in a pilot project in 2001 and rolled it out to the rest of the state's probation departments - except New York City - by 2010. The state didn't publish a comprehensive statistical evaluation of the tool until 2012. The study of more than 16,000 probationers found the tool was 71 percent accurate, but it did not evaluate racial differences.

A spokeswoman for the New York state division of criminal justice services said the study did not examine race because it only sought to test whether the tool had been properly calibrated to fit New York's probation population. She also said judges in nearly all New York counties are given defendants' Northpointe assessments during sentencing.

In 2009, Brennan and two colleagues published a validation study that found that Northpointe's risk of recidivism score had an accuracy rate of 68 percent in a sample of 2,328 people. Their study also found that the score was slightly less predictive for black men than white men - 67 percent versus 69 percent. It did not examine racial disparities beyond that, including whether some groups were more likely to be wrongly labeled higher risk.

Brennan said it is difficult to construct a score that doesn't include items that can be correlated with race - such as poverty, joblessness and social marginalization. "If those are omitted from your risk assessment, accuracy goes down," he said.

In 2011, Brennan and Wells sold Northpointe to Toronto-based conglomerate Constellation Software for an undisclosed sum.

Wisconsin has been among the most eager and expansive users of Northpointe's risk assessment tool in sentencing decisions. In 2012, the Wisconsin Department of Corrections launched the use of the software throughout the state. It is used at each step in the prison system, from sentencing to parole.

In a 2012 presentation, corrections official Jared Hoy described the system as a " giant correctional pinball machine " in which correctional officers could use the scores at every "decision point."

Wisconsin has not yet completed a statistical validation study of the tool and has not said when one might be released. State corrections officials declined repeated requests to comment for this article.

Some Wisconsin counties use other risk assessment tools at arrest to determine if a defendant is too risky for pretrial release. Once a defendant is convicted of a felony anywhere in the state, the Department of Corrections attaches Northpointe's assessment to the confidential presentence report given to judges, according to Hoy's presentation.

In theory, judges are not supposed to give longer sentences to defendants with higher risk scores. Rather, they are supposed to use the tests primarily to determine which defendants are eligible for probation or treatment programs.

Prediction Fails Differently for Black Defendants

Overall, Northpointe's assessment tool correctly predicts recidivism 61 percent of the time. But blacks are almost twice as likely as whites to be labeled a higher risk but not actually re-offend. It makes the opposite mistake among whites: They are much more likely than blacks to be labeled lower risk but go on to commit other crimes. (Source: ProPublica analysis of data from Broward County, Fla.)

But judges have cited scores in their sentencing decisions. In August 2013, Judge Scott Horne in La Crosse County, Wisconsin, declared that defendant Eric Loomis had been "identified, through the COMPAS assessment, as an individual who is at high risk to the community." The judge then imposed a sentence of eight years and six months in prison.

Loomis, who was charged with driving a stolen vehicle and fleeing from police, is challenging the use of the score at sentencing as a violation of his due process rights. The state has defended Horne's use of the score with the argument that judges can consider the score in addition to other factors. It has also stopped including scores in presentencing reports until the state Supreme Court decides the case.

"The risk score alone should not determine the sentence of an offender," Wisconsin Assistant Attorney General Christine Remington said last month during state Supreme Court arguments in the Loomis case. "We don't want courts to say, this person in front of me is a 10 on COMPAS as far as risk, and therefore I'm going to give him the maximum sentence."

That is almost exactly what happened to Zilly, the 48-year-old construction worker sent to prison for stealing a push lawnmower and some tools he intended to sell for parts. Zilly has long struggled with a meth habit. In 2012, he had been working toward recovery with the help of a Christian pastor when he relapsed and committed the thefts.

After Zilly was scored as a high risk for violent recidivism and sent to prison, a public defender appealed the sentence and called the score's creator, Brennan, as a witness.

Brennan testified that he didn't design his software to be used in sentencing. "I wanted to stay away from the courts," Brennan said, explaining that his focus was on reducing crime rather than punishment. "But as time went on I started realizing that so many decisions are made, you know, in the courts. So I gradually softened on whether this could be used in the courts or not."

"Not that I'm innocent, but I just believe people do change." (Stephen Maturen for ProPublica)

Still, Brennan testified, "I don't like the idea myself of COMPAS being the sole evidence that a decision would be based upon."

After Brennan's testimony, Judge Babler reduced Zilly's sentence, from two years in prison to 18 months. "Had I not had the COMPAS, I believe it would likely be that I would have given one year, six months," the judge said at an appeals hearing on Nov. 14, 2013.

Zilly said the score didn't take into account all the changes he was making in his life - his conversion to Christianity, his struggle to quit using drugs and his efforts to be more available for his son. "Not that I'm innocent, but I just believe people do change."

Florida's Broward County, where Brisha Borden stole the Huffy bike and was scored as high risk, does not use risk assessments in sentencing. "We don't think the [risk assessment] factors have any bearing on a sentence," said David Scharf, executive director of community programs for the Broward County Sheriff's Office in Fort Lauderdale.

Broward County has, however, adopted the score in pretrial hearings, in the hope of addressing jail overcrowding. A court-appointed monitor has overseen Broward County's jails since 1994 as a result of the settlement of a lawsuit brought by inmates in the 1970s. Even now, years later, the Broward County jail system is often more than 85 percent full, Scharf said.

In 2008, the sheriff's office decided that instead of building another jail, it would begin using Northpointe's risk scores to help identify which defendants were low risk enough to be released on bail pending trial. Since then, nearly everyone arrested in Broward has been scored soon after being booked. (People charged with murder and other capital crimes are not scored because they are not eligible for pretrial release.)

The scores are provided to the judges who decide which defendants can be released from jail. "My feeling is that if they don't need them to be in jail, let's get them out of there," Scharf said.

Scharf said the county chose Northpointe's software over other tools because it was easy to use and produced "simple yet effective charts and graphs for judicial review." He said the system costs about $22,000 a year.

In 2010, researchers at Florida State University examined the use of Northpointe's system in Broward County over a 12-month period and concluded that its predictive accuracy was "equivalent" in assessing defendants of different races. Like others, they did not examine whether different races were classified differently as low or high risk.

Scharf said the county would review ProPublica's findings. "We'll really look at them up close," he said.

Broward County Judge John Hurley, who oversees most of the pretrial release hearings, said the scores were helpful when he was a new judge, but now that he has experience he prefers to rely on his own judgment. "I haven't relied on COMPAS in a couple years," he said.

Hurley said he relies on factors including a person's prior criminal record, the type of crime committed, ties to the community, and their history of failing to appear at court proceedings.

ProPublica's analysis reveals that higher Northpointe scores are slightly correlated with longer pretrial incarceration in Broward County. But there are many reasons that could be true other than judges being swayed by the scores - people with higher risk scores may also be poorer and have difficulty paying bond, for example.

Most crimes are presented to the judge with a recommended bond amount, but he or she can adjust the amount. Hurley said he often releases first-time or low-level offenders without any bond at all.

However, in the case of Borden and her friend Sade Jones, the teenage girls who stole a kid's bike and scooter, Hurley raised the bond amount for each girl from the recommended $0 to $1,000 each.

Hurley said he has no recollection of the case and cannot recall if the scores influenced his decision.

Sade Jones, who had never been arrested before, was rated a medium risk. (Josh Ritchie for ProPublica)

The girls spent two nights in jail before being released on bond.

"We literally sat there and cried" the whole time they were in jail, Jones recalled. The girls were kept in the same cell. Otherwise, Jones said, "I would have gone crazy." Borden declined repeated requests to comment for this article.

Jones, who had never been arrested before, was rated a medium risk. She completed probation and got the felony burglary charge reduced to misdemeanor trespassing, but she has still struggled to find work.

"I went to McDonald's and a dollar store, and they all said no because of my background," she said. "It's all kind of difficult and unnecessary."

Julia Angwin is a senior reporter at ProPublica. From 2000 to 2013, she was a reporter at The Wall Street Journal, where she led a privacy investigative team that was a finalist for a Pulitzer Prize in Explanatory Reporting in 2011 and won a Gerald Loeb Award in 2010.

Lauren Kirchner is a senior reporting fellow at ProPublica. Surya Mattu is a contributing researcher. Design and production by Rob Weychert and David Sleight.

# data[].attributes.attachment_link.description
There's software used across the country to predict future criminals. And it's biased against blacks.

# data[].attributes.body
Dziga Vertov, 1929

# data[].attributes.body
Leni Riefenstahl, 19

# data[].attributes.attachment_link.title
Triumph of the Will / Triumph des Willens (1935)

# data[].attributes.headline
Facebook Doesn't Tell Users What it Really Knows About Them | ProPublica

# data[].attributes.subject
Facebook Doesn't Tell Users What it Really Knows About Them | ProPublica

# data[].attributes.attachment_link.readable_text
Facebook has long let users see all sorts of things the site knows about them, like whether they enjoy soccer, have recently moved, or like Melania Trump.

But the tech giant gives users little indication that it buys far more sensitive data about them, including their income, the types of restaurants they frequent and even how many credit cards are in their wallets.

Since September, ProPublica has been encouraging Facebook users to share the categories of interest that the site has assigned to them. Users showed us everything from "Pretending to Text in Awkward Situations" to "Breastfeeding in Public." In total, we collected more than 52,000 unique attributes that Facebook has used to classify users.

Facebook's page explaining "what influences the ads you see" says the company gets the information about its users "from a few different sources."

What the page doesn't say is that those sources include detailed dossiers obtained from commercial data brokers about users' offline lives. Nor does Facebook show users any of the often remarkably detailed information it gets from those brokers.

"They are not being honest," said Jeffrey Chester, executive director of the Center for Digital Democracy. "Facebook is bundling a dozen different data companies to target an individual customer, and an individual should have access to that bundle as well."

When asked this week about the lack of disclosure, Facebook responded that users can discern the use of third-party data if they know where to look. Each time an ad appears using such data, Facebook says, users can click a button on the ad revealing that fact. Users can still not see what specific information about their lives is being used.

The company said it does not disclose the use of third-party data on its general page about ad targeting because the data is widely available and was not collected by Facebook.

"Our approach to controls for third-party categories is somewhat different than our approach for Facebook-specific categories," said Steve Satterfield, a Facebook manager of privacy and public policy. "This is because the data providers we work with generally make their categories available across many different ad platforms, not just on Facebook."

Satterfield said users who don't want that information to be available to Facebook should contact the data brokers directly. He said users can visit a page in Facebook's help center, which provides links to the opt-outs for six data brokers that sell personal data to Facebook.

Limiting commercial data brokers' distribution of your personal information is no simple matter. For instance, opting out of Oracle's Datalogix, which provides about 350 types of data to Facebook according to our analysis, requires " sending a written request, along with a copy of government-issued identification" in postal mail to Oracle's chief privacy officer.

Users can ask data brokers to show them the information stored about them. But that can also be complicated. One Facebook broker, Acxiom, requires people to send the last four digits of their social security number to obtain their data. Facebook changes its providers from time to time so members would have to regularly visit the help center page to protect their privacy.

One of us actually tried to do what Facebook suggests. While writing a book about privacy in 2013, reporter Julia Angwin tried to opt out from as many data brokers as she could. Of the 92 brokers she identified that accepted opt-outs, 65 of them required her to submit a form of identification such as a driver's license. In the end, she could not remove her data from the majority of providers.

ProPublica's experiment to gather Facebook's ad categories from readers was part of our Black Box series, which explores the power of algorithms in our lives. Facebook uses algorithms not only to determine the news and advertisements that it displays to users, but also to categorize its users in tens of thousands of micro-targetable groups.

Our crowd-sourced data showed us that Facebook's categories range from innocuous groupings of people who like southern food to sensitive categories such as "Ethnic Affinity" which categorizes people based on their affinity for African-Americans, Hispanics and other ethnic groups. Advertisers can target ads toward a group - or exclude ads from being shown to a particular group.

Last month, after ProPublica bought a Facebook ad in its housing categories that excluded African-Americans, Hispanics and Asian-Americans, the company said it would build an automated system to help it spot ads that illegally discriminate.

We live in an era of increasing automation. But as machines make more decisions for us, it is increasingly important to understand the algorithms that produce their judgments. Read the story.

There's software used across the country to predict future criminals. And it's biased against blacks. Read the story.

Facebook has been working with data brokers since 2012 when it signed a deal with Datalogix. This prompted Chester, the privacy advocate at the Center for Digital Democracy, to file a complaint with the Federal Trade Commission alleging that Facebook had violated a consent decree with the agency on privacy issues. The FTC has never publicly responded to that complaint and Facebook subsequently signed deals with five other data brokers.

To find out exactly what type of data Facebook buys from brokers, we downloaded a list of 29,000 categories that the site provides to ad buyers. Nearly 600 of the categories were described as being provided by third-party data brokers. (Most categories were described as being generated by clicking pages or ads on Facebook.)

The categories from commercial data brokers were largely financial, such as "total liquid investible assets $1-$24,999," "People in households that have an estimated household income of between $100K and $125K," or even "Individuals that are frequent transactor at lower cost department or dollar stores."

We compared the data broker categories with the crowd-sourced list of what Facebook tells users about themselves. We found none of the data broker information on any of the tens of the thousands of "interests" that Facebook showed users.

Our tool also allowed users to react to the categories they were placed in as being "wrong," "creepy" or "spot on." The category that received the most votes for "wrong" was "Farmville slots." The category that got the most votes for "creepy" was "Away from family." And the category that was rated most "spot on" was "NPR."

Clarification, Jan. 4, 2017: We've added details about what Facebook tells users regarding third-party data. Specifically, each time an ad appears using such information, Facebook says, users can click a button on the ad revealing the use of third-party data.

Julia Angwin is a senior reporter at ProPublica. From 2000 to 2013, she was a reporter at The Wall Street Journal, where she led a privacy investigative team that was a finalist for a Pulitzer Prize in Explanatory Reporting in 2011 and won a Gerald Loeb Award in 2010.

Surya Mattu is a contributing researcher at ProPublica.

Terry is ProPublica's deputy editor, engagement. Prior to joining ProPublica, he led digital production and engagement at WDET 101.9 FM, NPR's affiliate in Detroit.

# data[].attributes.attachment_link.keywords[]
doesn’t reveal the data it is buying about their offline

# data[].attributes.attachment_link.authors[].name
Julia Angwin,Surya Mattu,Terry Parris Jr.

# data[].attributes.attachment_link.title
Facebook Doesn't Tell Users Everything It Really Knows About Them - ProPublica

# data[].attributes.attachment_link.description
Facebook has long let users see all sorts of things the site knows about them, like whether they enjoy soccer, have recently moved, or like Melania Trump. But the tech giant gives users little indication that it buys far more sensitive data about them, including their income, the types of restaurants they frequent and even how many credit cards are in their wallets.

# data[].attributes.headline
Data and Goliath: The Hidden Battles to Collect Your Data and Control your World | Bruce Schneier

# data[].attributes.subject
Data and Goliath: The Hidden Battles to Collect Your Data and Control your World | Bruce Schneier

# data[].attributes.attachment_link.title
Bruce Schneier: "Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World"

# data[].attributes.headline
Click Here to Kill Everybody | Bruce Schneier

# data[].attributes.subject
Click Here to Kill Everybody | Bruce Schneier

# data[].attributes.attachment_link.title
Bruce Schneier: "Click Here to Kill Everybody" | Talks at Google

# data[].attributes.headline
Problems with Machine Learning | James Mickens

# data[].attributes.subject
Problems with Machine Learning | James Mickens

# data[].attributes.attachment_link.description
Upload and share videos instantly. It's free and simple. No signup required.

# data[].attributes.headline
Cambridge Analytica is only the Beginning | Computational Privacy Group

# data[].attributes.subject
Cambridge Analytica is only the Beginning | Computational Privacy Group

# data[].attributes.attachment_link.readable_text
Recent revelations from Cambridge Analytica and the Trump campaign show how vulnerable our privacy is to innocuous apps installed by our friends. In an new preprint, we model how our privacy is impacted by the people we interact with. Our findings show that node-based intrusions, attacks on our privacy through our friends, is becoming one of the main privacy risk in today's networked societies.
In the span of a week, Cambridge Analytica turned from yet another Big Data analytics company to being in the media spotlight and the focus of attention from regulators. The reason for such a sudden interest? A whistleblower's revelations that the company obtained private data on 30 to 50 million Americans, and used this information to assist the Trump campaign through a large scale, micro-targeted, Facebook ad campaign.

All of this started in 2014 when a Cambridge University researcher, Aleksandr Kogan, developed a personality test app, that, in addition to collecting your own personal data, collected data about your friends. This app was installed by over 270,000 unsuspecting Americans (some of whom were recruited through MTurk). While the users gave "informed consent" for the app to collect the data, few of them realized that the app would harvest information not only about themselves, but also about their entire social circle. This data was then acquired by Cambridge Analytica to help the Trump campaign analyze people's political preferences.

These revelations have come as a shock to many. Besides the concerns of large-scale manipulation, few realized how easy it had been for Kogan to harvest their data. The fact that your friend's farming simulation game might be actively spying on you is an unsettling fact that most of us were completely unaware of.

This risk to your privacy, incurred by the privacy choice of those you interact with (as well as Facebook's settings), is what we call group privacy. We are convinced that group privacy, attacks on your privacy through your friends, is becoming one of the main privacy risk in today's networked societies. Our latest research on this topic is currently under review but, given the recent revelations, we felt it was important for our results to be available to the public. We posted today the preprint of our manuscript, "Quantifying Surveillance in the Networked Age: Node-based Intrusions and Group Privacy", on arXiv.

In Quantifying Surveillance in the Networked Age, we investigate and quantify how the privacy settings of the people we interact with affect our privacy. we consider various context where people interact with one another along a network: all the people we friended on Facebook, the people around us on the street, or the people we call and text on our phone. More precisely, we model the risk of node-based intrusions: if an attacker compromise some nodes (people, phones, etc) in the network, e.g. through a Facebook or smartphone app, how does it impact everyone else's privacy? How many people's data does he have access to and is the fact that we live in an increasingly connected society (where we connected to anyone else on earth by only six degrees of separation making the matter worse?

To answer those questions and quantify the reach of the Cambridge Analytica leak, we introduced the concept of node-observability, the number of people (fraction of the nodes in the graph) that are observed when I compromised nodes in the graph. Using this notion, we were able to independently confirm that Cambridge Analytica was likely to have collected data about 46.6M Americans and that the Obama 2012 campaign had likely access to data about 102M Americans (his campaign staff reported to have "ingested the entire U.S. social graph". We then showed that similar attacks are likely to happen in other settings e.g. using smartphone apps to collect location data. Group privacy, distributed attacks on our privacy through our friends, are much more likely than previously believed and, in our opinion, one of the main privacy risk moving forward.

In the paper, we first prove that all we need to estimate the 1-hop node-observability, the number of profiles I can access if 10.000 people install my app, is the degree distribution of the graph: how many users there are on Facebook, and how many friends they have. We extracted the former (214M) from Statistica and the latter from a paper published by FB researchers. The degree distribution (Fig. 1) shows that, in 2011, people on FB had 214 friends on average and couldn't have more 5000 friends.

Using these two pieces of information, our model allows us to estimate Facebook's node-observability, the number of profiles that are accessible if 10 000, 100 000 or 1M people install an app like Kogan's. Qz reported that 270 000 installed Kogan's app allowing him and the Trump campaign to access the profiles of 30M to 50M Americans. Knowing only the number of people who install the app, our model allows us to independently validate that the Trump campaign had likely access to the profile of 46.6M Americans which they then used to target voters (Fig. 1).

The Trump campaign was not the first one to realize the power of social media. In fact, the Obama campaign relied on a similar, albeit much more transparent, mechanism in 2012. The Washington Post reported that roughly 1M people installed their Obama 2012 Facebook app allowing them, according to our model, to use data about 100M Americans to recommend to app users who they should reach out to (Fig. 1). This contrast with claims from his staff at time who said they had "ingested the entire U.S. social graph"

The fact that Kogan was able to extract so much data with Facebook having no actual technical control over the information is what lead some to call it a data breach. Indeed, as Jason Koebler said to Motherboard: "If your data has already been taken, Facebook has no mechanism and no power to make people delete it.. If your data was taken, it has very likely been sold, laundered, and put back into Facebook". Furthermore, research (incl. ours) showed that big data cannot - be - effectively - anonymized.

While our analysis shows that both the Trump and Obama campaign use modern privacy's network effects to gain access to data about millions of Americans, it could have been even worse. Facebook, at the time of the Cambridge Analytica data collection, only let apps access data from your friends (what we call 1-hop). Their previous privacy settings were a lot more loose, letting you share data with friends of friends (2-hops). While this may sound like a small difference, it is not. Our analysis shows that such settings would have allowed Cambridge Analytica to collect a striking 210M profiles, 98.5% of the US Facebook population!

If you have been following the surveillance debate, this might not be the first time you are hearing about "hops". A similar idea has indeed been built into US regulation allowing intelligence agencies not only to collect phone records of suspects, but also to collect the phone records of their contacts (1 hop), or the contacts of their contacts (2 hops). Until the Snowden revelations, even 3-hop contacts could be collected. Using our metric and a real-world large-scale mobile phone dataset, we showed that all it would take to observe 86.6% of all communications occuring in the US through this 3-hops policy is to convince a judge that 0.01% of the nodes are "suspicious". Under the new, 2-hops, policy this would still allow an attacker to observe 18.6% of all the communication.

Node-observability attacks apply beyond social networks and can, for example, be used to collect large-scale location data through the phone of people around you. The culprit here is a tiny, convenient, feature of your phone that you probably have never really thought about: your phone will automatically connect to Wi-Fi networks you have connected to before. To be more precise, your phone will attempt to connect to Wi-Fi that have the same name as Wi-Fi you have connected to before (and will connect to them if the original Wi-Fi was not encrypted). Using this, I would first embed a piece of code in a couple of apps on the Play or App store, like what UK startup Tamoco is already doing. Then, I would have the app create fake hotspots with popular names (xfinity-wifi, etc), allowing me to record the identifier of phones who connect to my hotspot and the GPS position of the phone. Using a real mobility dataset, we show this attack to be highly effective: compromising as few as 1% of the phones in London (Tamoco reports to have their code installed on 12M people's phone in the UK) would allow an attacker to track the location (in 1km²) of about 54% of the city's population.

In our modern networked societies, privacy is a shared responsibility. The people we interact with, be it at home, at work, or on the streets, impact our privacy. In our paper, we call this group privacy, and we study it through node-based intrusions. Our results show that the privacy risks incurred by the (poor) privacy settings of the people around us are much more important than previously believed, and that network effects makes us highly vulnerable to node-based intrusions compromising even a small fraction of the network. The Cambridge Analytica story is one of the first large scale examples of such group privacy attacks, and it is unlikely to be the last.

Methodology

1. We extracted the distribution of degrees for the United States Facebook graph from this paper.

2. Proposition 3.3 from our paper shows that we can compute 1-hop global node-observability using only the list of degrees from the graph, through an expression of the type (for a graph $(V,E), n=|V|$ and $n_c$ compromised nodes): $\rho_v^* = \frac{n_c}{n}+ \frac{n-n_c}{n} \cdot \sum_{v \in V} f\left(degree(v)\right)$. We have the probabilistic distribution $T_D$, and if we assume that nodes are sampled from it independently (by definition of $T_D$), then we compute global node observability as: $\rho_v^* = \sum_{d=1}^n T_D(d) f(d) = {E}(f(D))$

3. For the 2-hop global node-observability, we sample a graph with a large number of nodes that follows the same distribution as Facebook's graph, using the configuration model. We studied how the number of nodes influences the observability (Fig. 2), and conclude that taking $N = 250,000$ nodes provides stable enough results to extend to 214 millions (Fig. 2). We then compute 2-hop global node-observability using the code from our paper, which will be made available on publication.

# data[].attributes.attachment_link.title
Computational Privacy Group - Imperial College London

# data[].attributes.attachment_link.description
Recent revelations from Cambridge Analytica and the Trump campaign show how vulnerable our privacy is to innocuous apps installed by our friends. In an new preprint, we model how our privacy is impacted by the people we interact with.

# data[].attributes.headline
Wimbledon used IBM's Watson (AI) to celebrate its 150th anniversary

# data[].attributes.subject
Wimbledon used IBM's Watson (AI) to celebrate its 150th anniversary

# data[].attributes.attachment_link.readable_text
The All England Lawn Tennis Club (AELTC) - the organisation behind the annual Wimbledon tournament - has further strengthened its relationship with IBM by allowing the tech firm's AI to art direct the poster celebrating its 150-year anniversary.

The final composition is a mosaic of the famous courts composed of 9,000 separate images from the Wimbledon photo archive. After a brief 'teaching' session, IBM Watson's visual recognition interface combed through three terabytes of imagery - more than 300,000 separate images - to select the best photographs to feature in the poster.

IBM created another piece of software to stitch the pictures together into the final image.

Additionally, archive photos were matched specifically to their corresponding visuals in the main image - for instance, photos of ball girls were used to illustrate the ball girls in the poster. Watson was also taught to find elements within larger images that could be cropped out and included where necessary.

The AELTC hopes the cutting-edge art direction will demonstrate "how Wimbledon is never afraid to innovate in [its] own pursuit of greatness'.

IBM has been working with Wimbledon since 1990 in order to improve the experience of players, visitors and those who watch at home. Last year saw the tennis brand embrace video technology in order to take over from the BBC as the tournament's 'lead broadcaster', as well as launch its own smart assistant dubbed Fred.

# data[].attributes.attachment_link.title
IBM's Watson 'paints' poster for Wimbledon championships

# data[].attributes.attachment_link.description
The All England Lawn Tennis Club (AELTC) - the organisation behind the annual Wimbledon tournament - has further strengthened its relationship with IBM by allowing the tech firm's AI to art direct the poster celebrating its 150-year anniversary. The final composition is a mosaic of the famous courts composed of 9,000 separate images from the Wimbledon photo archive.

# data[].attributes.attachment_link.readable_text
Artificial intelligence could erase many practical advantages of democracy, and erode the ideals of liberty and equality. It will further concentrate power among a small elite if we don't take steps to stop it.

Yoshi Sodeoka

Editor's Note: This article is part of a series that attempts to answer the question: Is democracy dying?

I. The Growing Fear of Irrelevance

There is nothing inevitable about democracy. For all the success that democracies have had over the past century or more, they are blips in history. Monarchies, oligarchies, and other forms of authoritarian rule have been far more common modes of human governance.

The emergence of liberal democracies is associated with ideals of liberty and equality that may seem self-evident and irreversible. But these ideals are far more fragile than we believe. Their success in the 20th century depended on unique technological conditions that may prove ephemeral.

In the second decade of the 21st century, liberalism has begun to lose credibility. Questions about the ability of liberal democracy to provide for the middle class have grown louder; politics have grown more tribal; and in more and more countries, leaders are showing a penchant for demagoguery and autocracy. The causes of this political shift are complex, but they appear to be intertwined with current technological developments. The technology that favored democracy is changing, and as artificial intelligence develops, it might change further.

Information technology is continuing to leap forward; biotechnology is beginning to provide a window into our inner lives-our emotions, thoughts, and choices. Together, infotech and biotech will create unprecedented upheavals in human society, eroding human agency and, possibly, subverting human desires. Under such conditions, liberal democracy and free-market economics might become obsolete.

From Our October 2018 Issue

Subscribe to The Atlantic and support 160 years of independent journalism
Subscribe
Ordinary people may not understand artificial intelligence and biotechnology in any detail, but they can sense that the future is passing them by. In 1938 the common man's condition in the Soviet Union, Germany, or the United States may have been grim, but he was constantly told that he was the most important thing in the world, and that he was the future (provided, of course, that he was an "ordinary man," rather than, say, a Jew or a woman). He looked at the propaganda posters-which typically depicted coal miners and steelworkers in heroic poses-and saw himself there: "I am in that poster! I am the hero of the future!"

In 2018 the common person feels increasingly irrelevant. Lots of mysterious terms are bandied about excitedly in ted Talks, at government think tanks, and at high-tech conferences- globalization, blockchain, genetic engineering, AI, machine learning -and common people, both men and women, may well suspect that none of these terms is about them.

In the 20th century, the masses revolted against exploitation and sought to translate their vital role in the economy into political power. Now the masses fear irrelevance, and they are frantic to use their remaining political power before it is too late. Brexit and the rise of Donald Trump may therefore demonstrate a trajectory opposite to that of traditional socialist revolutions. The Russian, Chinese, and Cuban revolutions were made by people who were vital to the economy but lacked political power; in 2016, Trump and Brexit were supported by many people who still enjoyed political power but feared they were losing their economic worth. Perhaps in the 21st century, populist revolts will be staged not against an economic elite that exploits people but against an economic elite that does not need them anymore. This may well be a losing battle. It is much harder to struggle against irrelevance than against exploitation.

The revolutions in information technology and biotechnology are still in their infancy, and the extent to which they are responsible for the current crisis of liberalism is debatable. Most people in Birmingham, Istanbul, St. Petersburg, and Mumbai are only dimly aware, if they are aware at all, of the rise of AI and its potential impact on their lives. It is undoubtable, however, that the technological revolutions now gathering momentum will in the next few decades confront humankind with the hardest trials it has yet encountered.

II. A New Useless Class?

Let's start with jobs and incomes, because whatever liberal democracy's philosophical appeal, it has gained strength in no small part thanks to a practical advantage: The decentralized approach to decision making that is characteristic of liberalism-in both politics and economics-has allowed liberal democracies to outcompete other states, and to deliver rising affluence to their people.

Liberalism reconciled the proletariat with the bourgeoisie, the faithful with atheists, natives with immigrants, and Europeans with Asians by promising everybody a larger slice of the pie. With a constantly growing pie, that was possible. And the pie may well keep growing. However, economic growth may not solve social problems that are now being created by technological disruption, because such growth is increasingly predicated on the invention of more and more disruptive technologies.

Fears of machines pushing people out of the job market are, of course, nothing new, and in the past such fears proved to be unfounded. But artificial intelligence is different from the old machines. In the past, machines competed with humans mainly in manual skills. Now they are beginning to compete with us in cognitive skills. And we don't know of any third kind of skill-beyond the manual and the cognitive-in which humans will always have an edge.

At least for a few more decades, human intelligence is likely to far exceed computer intelligence in numerous fields. Hence as computers take over more routine cognitive jobs, new creative jobs for humans will continue to appear. Many of these new jobs will probably depend on cooperation rather than competition between humans and AI. Human-AI teams will likely prove superior not just to humans, but also to computers working on their own.

However, most of the new jobs will presumably demand high levels of expertise and ingenuity, and therefore may not provide an answer to the problem of unemployed unskilled laborers, or workers employable only at extremely low wages. Moreover, as AI continues to improve, even jobs that demand high intelligence and creativity might gradually disappear. The world of chess serves as an example of where things might be heading. For several years after IBM's computer Deep Blue defeated Garry Kasparov in 1997, human chess players still flourished; AI was used to train human prodigies, and teams composed of humans plus computers proved superior to computers playing alone.

Yet in recent years, computers have become so good at playing chess that their human collaborators have lost their value and might soon become entirely irrelevant. On December 6, 2017, another crucial milestone was reached when Google's AlphaZero program defeated the Stockfish 8 program. Stockfish 8 had won a world computer chess championship in 2016. It had access to centuries of accumulated human experience in chess, as well as decades of computer experience. By contrast, AlphaZero had not been taught any chess strategies by its human creators-not even standard openings. Rather, it used the latest machine-learning principles to teach itself chess by playing against itself. Nevertheless, out of 100 games that the novice AlphaZero played against Stockfish 8, AlphaZero won 28 and tied 72-it didn't lose once. Since AlphaZero had learned nothing from any human, many of its winning moves and strategies seemed unconventional to the human eye. They could be described as creative, if not downright genius.

Can you guess how long AlphaZero spent learning chess from scratch, preparing for the match against Stockfish 8, and developing its genius instincts? Four hours. For centuries, chess was considered one of the crowning glories of human intelligence. AlphaZero went from utter ignorance to creative mastery in four hours, without the help of any human guide.

AlphaZero is not the only imaginative software out there. One of the ways to catch cheaters in chess tournaments today is to monitor the level of originality that players exhibit. If they play an exceptionally creative move, the judges will often suspect that it could not possibly be a human move-it must be a computer move. At least in chess, creativity is already considered to be the trademark of computers rather than humans! So if chess is our canary in the coal mine, we have been duly warned that the canary is dying. What is happening today to human-AI teams in chess might happen down the road to human-AI teams in policing, medicine, banking, and many other fields.

What's more, AI enjoys uniquely nonhuman abilities, which makes the difference between AI and a human worker one of kind rather than merely of degree. Two particularly important nonhuman abilities that AI possesses are connectivity and updatability.

For example, many drivers are unfamiliar with all the changing traffic regulations on the roads they drive, and they often violate them. In addition, since every driver is a singular entity, when two vehicles approach the same intersection, the drivers sometimes miscommunicate their intentions and collide. Self-driving cars, by contrast, will know all the traffic regulations and never disobey them on purpose, and they could all be connected to one another. When two such vehicles approach the same junction, they won't really be two separate entities, but part of a single algorithm. The chances that they might miscommunicate and collide will therefore be far smaller.

Similarly, if the World Health Organization identifies a new disease, or if a laboratory produces a new medicine, it can't immediately update all the human doctors in the world. Yet even if you had billions of AI doctors in the world-each monitoring the health of a single human being-you could still update all of them within a split second, and they could all communicate to one another their assessments of the new disease or medicine. These potential advantages of connectivity and updatability are so huge that at least in some lines of work, it might make sense to replace all humans with computers, even if individually some humans still do a better job than the machines.

All of this leads to one very important conclusion: The automation revolution will not consist of a single watershed event, after which the job market will settle into some new equilibrium. Rather, it will be a cascade of ever bigger disruptions. Old jobs will disappear and new jobs will emerge, but the new jobs will also rapidly change and vanish. People will need to retrain and reinvent themselves not just once, but many times.

from the
Just as in the 20th century governments established massive education systems for young people, in the 21st century they will need to establish massive reeducation systems for adults. But will that be enough? Change is always stressful, and the hectic world of the early 21st century has produced a global epidemic of stress. As job volatility increases, will people be able to cope? By 2050, a useless class might emerge, the result not only of a shortage of jobs or a lack of relevant education but also of insufficient mental stamina to continue learning new skills.

III. The Rise of Digital Dictatorships

As many people lose their economic value, they might also come to lose their political power. The same technologies that might make billions of people economically irrelevant might also make them easier to monitor and control.

AI frightens many people because they don't trust it to remain obedient. Science fiction makes much of the possibility that computers or robots will develop consciousness-and shortly thereafter will try to kill all humans. But there is no particular reason to believe that AI will develop consciousness as it becomes more intelligent. We should instead fear AI because it will probably always obey its human masters, and never rebel. AI is a tool and a weapon unlike any other that human beings have developed; it will almost certainly allow the already powerful to consolidate their power further.
atlantic archives
"IN ALL THIS high-tech commercial world there is nothing that looks particularly democratic. It lends itself to surveillance as well as liberty, to new forms of manipulation and covert control as well as new kinds of participation, to skewed, unjust market outcomes as well as greater productivity. The consumer society and the open society are not quite synonymous. Capitalism and democracy have a relationship, but it is something less than a marriage." Read more

Matt Huynh

Consider surveillance. Numerous countries around the world, including several democracies, are busy building unprecedented systems of surveillance. For example, Israel is a leader in the field of surveillance technology, and has created in the occupied West Bank a working prototype for a total-surveillance regime. Already today whenever Palestinians make a phone call, post something on Facebook, or travel from one city to another, they are likely to be monitored by Israeli microphones, cameras, drones, or spy software. Algorithms analyze the gathered data, helping the Israeli security forces pinpoint and neutralize what they consider to be potential threats. The Palestinians may administer some towns and villages in the West Bank, but the Israelis command the sky, the airwaves, and cyberspace. It therefore takes surprisingly few Israeli soldiers to effectively control the roughly 2.5 million Palestinians who live in the West Bank.

In one incident in October 2017, a Palestinian laborer posted to his private Facebook account a picture of himself in his workplace, alongside a bulldozer. Adjacent to the image he wrote, "Good morning!" A Facebook translation algorithm made a small error when transliterating the Arabic letters. Instead of Ysabechhum (which means "Good morning"), the algorithm identified the letters as Ydbachhum (which means "Hurt them"). Suspecting that the man might be a terrorist intending to use a bulldozer to run people over, Israeli security forces swiftly arrested him. They released him after they realized that the algorithm had made a mistake. Even so, the offending Facebook post was taken down-you can never be too careful. What Palestinians are experiencing today in the West Bank may be just a primitive preview of what billions of people will eventually experience all over the planet.

Imagine, for instance, that the current regime in North Korea gained a more advanced version of this sort of technology in the future. North Koreans might be required to wear a biometric bracelet that monitors everything they do and say, as well as their blood pressure and brain activity. Using the growing understanding of the human brain and drawing on the immense powers of machine learning, the North Korean government might eventually be able to gauge what each and every citizen is thinking at each and every moment. If a North Korean looked at a picture of Kim Jong Un and the biometric sensors picked up telltale signs of anger (higher blood pressure, increased activity in the amygdala), that person could be in the gulag the next day.

And yet such hard-edged tactics may not prove necessary, at least much of the time. A facade of free choice and free voting may remain in place in some countries, even as the public exerts less and less actual control. To be sure, attempts to manipulate voters' feelings are not new. But once somebody (whether in San Francisco or Beijing or Moscow) gains the technological ability to manipulate the human heart-reliably, cheaply, and at scale-democratic politics will mutate into an emotional puppet show.

We are unlikely to face a rebellion of sentient machines in the coming decades, but we might have to deal with hordes of bots that know how to press our emotional buttons better than our mother does and that use this uncanny ability, at the behest of a human elite, to try to sell us something-be it a car, a politician, or an entire ideology. The bots might identify our deepest fears, hatreds, and cravings and use them against us. We have already been given a foretaste of this in recent elections and referendums across the world, when hackers learned how to manipulate individual voters by analyzing data about them and exploiting their prejudices. While science-fiction thrillers are drawn to dramatic apocalypses of fire and smoke, in reality we may be facing a banal apocalypse by clicking.

The biggest and most frightening impact of the AI revolution might be on the relative efficiency of democracies and dictatorships. Historically, autocracies have faced crippling handicaps in regard to innovation and economic growth. In the late 20th century, democracies usually outperformed dictatorships, because they were far better at processing information. We tend to think about the conflict between democracy and dictatorship as a conflict between two different ethical systems, but it is actually a conflict between two different data-processing systems. Democracy distributes the power to process information and make decisions among many people and institutions, whereas dictatorship concentrates information and power in one place. Given 20th-century technology, it was inefficient to concentrate too much information and power in one place. Nobody had the ability to process all available information fast enough and make the right decisions. This is one reason the Soviet Union made far worse decisions than the United States, and why the Soviet economy lagged far behind the American economy.

Related Stories

However, artificial intelligence may soon swing the pendulum in the opposite direction. AI makes it possible to process enormous amounts of information centrally. In fact, it might make centralized systems far more efficient than diffuse systems, because machine learning works better when the machine has more information to analyze. If you disregard all privacy concerns and concentrate all the information relating to a billion people in one database, you'll wind up with much better algorithms than if you respect individual privacy and have in your database only partial information on a million people. An authoritarian government that orders all its citizens to have their DNA sequenced and to share their medical data with some central authority would gain an immense advantage in genetics and medical research over societies in which medical data are strictly private. The main handicap of authoritarian regimes in the 20th century-the desire to concentrate all information and power in one place-may become their decisive advantage in the 21st century.

Yoshi Sodeoka

New technologies will continue to emerge, of course, and some of them may encourage the distribution rather than the concentration of information and power. Blockchain technology, and the use of cryptocurrencies enabled by it, is currently touted as a possible counterweight to centralized power. But blockchain technology is still in the embryonic stage, and we don't yet know whether it will indeed counterbalance the centralizing tendencies of AI. Remember that the Internet, too, was hyped in its early days as a libertarian panacea that would free people from all centralized systems-but is now poised to make centralized authority more powerful than ever.

IV. The Transfer of Authority to Machines

Even if some societies remain ostensibly democratic, the increasing efficiency of algorithms will still shift more and more authority from individual humans to networked machines. We might willingly give up more and more authority over our lives because we will learn from experience to trust the algorithms more than our own feelings, eventually losing our ability to make many decisions for ourselves. Just think of the way that, within a mere two decades, billions of people have come to entrust Google's search algorithm with one of the most important tasks of all: finding relevant and trustworthy information. As we rely more on Google for answers, our ability to locate information independently diminishes. Already today, "truth" is defined by the top results of a Google search. This process has likewise affected our physical abilities, such as navigating space. People ask Google not just to find information but also to guide them around. Self-driving cars and AI physicians would represent further erosion: While these innovations would put truckers and human doctors out of work, their larger import lies in the continuing transfer of authority and responsibility to machines.

Humans are used to thinking about life as a drama of decision making. Liberal democracy and free-market capitalism see the individual as an autonomous agent constantly making choices about the world. Works of art-be they Shakespeare plays, Jane Austen novels, or cheesy Hollywood comedies-usually revolve around the hero having to make some crucial decision. To be or not to be? To listen to my wife and kill King Duncan, or listen to my conscience and spare him? To marry Mr. Collins or Mr. Darcy? Christian and Muslim theology similarly focus on the drama of decision making, arguing that everlasting salvation depends on making the right choice.

What will happen to this view of life as we rely on AI to make ever more decisions for us? Even now we trust Netflix to recommend movies and Spotify to pick music we'll like. But why should AI's helpfulness stop there?

Every year millions of college students need to decide what to study. This is a very important and difficult decision, made under pressure from parents, friends, and professors who have varying interests and opinions. It is also influenced by students' own individual fears and fantasies, which are themselves shaped by movies, novels, and advertising campaigns. Complicating matters, a given student does not really know what it takes to succeed in a given profession, and doesn't necessarily have a realistic sense of his or her own strengths and weaknesses.

It's not so hard to see how AI could one day make better decisions than we do about careers, and perhaps even about relationships. But once we begin to count on AI to decide what to study, where to work, and whom to date or even marry, human life will cease to be a drama of decision making, and our conception of life will need to change. Democratic elections and free markets might cease to make sense. So might most religions and works of art. Imagine Anna Karenina taking out her smartphone and asking Siri whether she should stay married to Karenin or elope with the dashing Count Vronsky. Or imagine your favorite Shakespeare play with all the crucial decisions made by a Google algorithm. Hamlet and Macbeth would have much more comfortable lives, but what kind of lives would those be? Do we have models for making sense of such lives?

Can parliaments and political parties overcome these challenges and forestall the darker scenarios? At the current moment this does not seem likely. Technological disruption is not even a leading item on the political agenda. During the 2016 U.S. presidential race, the main reference to disruptive technology concerned Hillary Clinton's email debacle, and despite all the talk about job loss, neither candidate directly addressed the potential impact of automation. Donald Trump warned voters that Mexicans would take their jobs, and that the U.S. should therefore build a wall on its southern border. He never warned voters that algorithms would take their jobs, nor did he suggest building a firewall around California.

So what should we do?

For starters, we need to place a much higher priority on understanding how the human mind works-particularly how our own wisdom and compassion can be cultivated. If we invest too much in AI and too little in developing the human mind, the very sophisticated artificial intelligence of computers might serve only to empower the natural stupidity of humans, and to nurture our worst (but also, perhaps, most powerful) impulses, among them greed and hatred. To avoid such an outcome, for every dollar and every minute we invest in improving AI, we would be wise to invest a dollar and a minute in exploring and developing human consciousness.

More practically, and more immediately, if we want to prevent the concentration of all wealth and power in the hands of a small elite, we must regulate the ownership of data. In ancient times, land was the most important asset, so politics was a struggle to control land. In the modern era, machines and factories became more important than land, so political struggles focused on controlling these vital means of production. In the 21st century, data will eclipse both land and machinery as the most important asset, so politics will be a struggle to control data's flow.

Unfortunately, we don't have much experience in regulating the ownership of data, which is inherently a far more difficult task than regulating land or machines. Data are everywhere and nowhere at the same time, they can move at the speed of light, and you can create as many copies of them as you want. Do the data collected about my DNA, my brain, and my life belong to me, or to the government, or to a corporation, or to the human collective?

The race to accumulate data is already on, and is currently headed by giants such as Google and Facebook and, in China, Baidu and Tencent. So far, many of these companies have acted as "attention merchants"-they capture our attention by providing us with free information, services, and entertainment, and then they resell our attention to advertisers. Yet their true business isn't merely selling ads. Rather, by capturing our attention they manage to accumulate immense amounts of data about us, which are worth more than any advertising revenue. We aren't their customers-we are their product.

Ordinary people will find it very difficult to resist this process. At present, many of us are happy to give away our most valuable asset-our personal data-in exchange for free email services and funny cat videos. But if, later on, ordinary people decide to try to block the flow of data, they are likely to have trouble doing so, especially as they may have come to rely on the network to help them make decisions, and even for their health and physical survival.

Nationalization of data by governments could offer one solution; it would certainly curb the power of big corporations. But history suggests that we are not necessarily better off in the hands of overmighty governments. So we had better call upon our scientists, our philosophers, our lawyers, and even our poets to turn their attention to this big question: How do you regulate the ownership of data?

Currently, humans risk becoming similar to domesticated animals. We have bred docile cows that produce enormous amounts of milk but are otherwise far inferior to their wild ancestors. They are less agile, less curious, and less resourceful. We are now creating tame humans who produce enormous amounts of data and function as efficient chips in a huge data-processing mechanism, but they hardly maximize their human potential. If we are not careful, we will end up with downgraded humans misusing upgraded computers to wreak havoc on themselves and on the world.

If you find these prospects alarming-if you dislike the idea of living in a digital dictatorship or some similarly degraded form of society-then the most important contribution you can make is to find ways to prevent too much data from being concentrated in too few hands, and also find ways to keep distributed data processing more efficient than centralized data processing. These will not be easy tasks. But achieving them may be the best safeguard of democracy.
This article has been adapted from Yuval Noah Harari's book, 21 Lessons for the 21st Century.

# data[].attributes.attachment_link.description
Artificial intelligence could erase many practical advantages of democracy, and erode the ideals of liberty and equality. It will further concentrate power among a small elite if we don't take steps to stop it. Editor's Note: This article is part of a series that attempts to answer the question: Is democracy dying?

# data[].attributes.headline
Google's reCaptcha used to train Pentagon's Drones

# data[].attributes.subject
Google's reCaptcha used to train Pentagon's Drones

# data[].attributes.attachment_link.readable_text
Are you using Google's reCAPTCHA? Then you might unknowingly be training Pentagon's killer drones.

With the rise of blogs, web 2.0, and user-generated content, came a torrent of SPAM. Automated computer programs, or bots, were flooding the internet with comment SPAM, more often than not drowning out actual discussion. In an effort to deal with the problem, the good guys on the internet took advantage of the fact that computers used to be quite bad at recognizing objects in an image. And thus the "Completely Automated Public Turing test to tell Computers and Humans Apart", or CAPTCHA was born.

The first CAPTCHA implementations were dead simple. To be allowed to write a comment on a web site, the user had to recognize the letters in an image, and then write those exact letters in a text box. If the letters in the image matched the ones provided by the user, the site knew user was human, and was thus allowed to comment. Recognizing the letters in the image was an easy task for most people, but the bots didn't stand a chance.

But as image recognition software became more advanced, the bots started to solve the first CAPTCHA implementations. The dam was about to break, and to prevent this, the internet created more advanced CAPTCHAs. Some of these second generation CAPTCHA implementations were terrible, like the one in the image to the left. Sure, they were impossible for bots to solve, but they were also very hard for a human being to figure out. It happened more than once that I simply gave up filling out an online form because of an unsolvable CAPTCHA.

Then Google came along with their silver bullet: reCAPTCHA.

reCAPTCHA

The current version of Google's reCAPTCHA is different from most other CAPTCHA implementations. Instead of numbers and letters, the user is required to identify everyday objects. In the example to the right, the user is asked to identify all the images containing cars. Very easy for a human being, not that easy for your average computer recognition software.

Google is providing reCAPTCHA for free, but as we all know, there is no such thing as a free lunch. Millions of reCAPTCHAs are solved every day, and Google is using the technology to digitize text, annotate images, create better maps - and solve hard AI problems. The use of the data from our reCAPTCHA example with the cars above is obvious: It will help improve the algorithms used in Google's self driving vehicles.

And that's just fantastic, right? There is less SPAM on the internet, and self driving cars will be less likely to crash into things. Everybody wins, yes? The problem is that you can't be certain if the data you just provided will be used to help a self driving car identify another car to avoid it, or to help a military drone identify a car to blow it up.

Autonomous killers

"That can't be!", you might hear yourself screaming. Google is the Don't Be Evil-company! But no. Gizmondo recently published a story how Google is "helping the Pentagon build AI for drones". The story has since been picked up by other news and technology sites, like The Intercept and The Register.

Google is providing Pentagon with access to TensorFlow, an open-source software library for machine learning applications such as neural networks. At Pentagon, TensorFlow is being used to help military analysts detect objects in images.

You might argue that the Pentagon are the good guys. And Google's reCAPTCHA isn't showing pictures from Syria, asking you to identify Russian fighter jets, and al-Assad's tanks. At least not yet. But in my simple mind, you're on a terribly slippery slope as soon as you start getting directly involved with the military in any way, shape, or form.

Back to basic

I've been using Google's reCAPTCHA on this site for years. It's terrible effective for stopping SPAM. But I can't take part in a program that might potentially be used for military purposes. That means that I will replace reCAPTCHA with another CAPTCHA implementation as soon as time permits. Will this result in more comment SPAM? Perhaps. Will it make all military drones fall out of the sky? Certainly not.

But in a world where a single consumer's voice doesn't matter dick, it's about as much as I can do. If you, too, are using reCAPTCHA on your site, I urge you to reconsider.

# data[].attributes.attachment_link.title
You Might Be Helping Pentagon Train Killer Drones. | Vegard Skjefstad

# data[].attributes.attachment_link.description
Are you using Google's reCAPTCHA? Then you might unknowingly be training Pentagon's killer drones. With the rise of blogs, web 2.0, and user-generated content, came a torrent of SPAM. Automated computer programs, or bots, were flooding the internet with comment SPAM, more often than not drowning out actual discussion.

# data[].attributes.headline
To Save Everything Click Here: The Folly of Technological Solutionism  | Evgeny Morozov

# data[].attributes.subject
To Save Everything Click Here: The Folly of Technological Solutionism  | Evgeny Morozov

# data[].attributes.attachment_link.title
To Save Everything Click Here:  The Folly of Technological Solutionism

# data[].attributes.headline
Privacy Expert Quits Google's Smart City Project | The Guardian

# data[].attributes.subject
Privacy Expert Quits Google's Smart City Project | The Guardian

# data[].attributes.attachment_link.title
'City of surveillance': privacy expert quits Toronto's smart-city project

# data[].attributes.attachment_link.description
Wired neighborhood planned by Google sister company has raised questions over data protection

# data[].attributes.headline
Das leben der Anderen (The Lives of Others)

# data[].attributes.subject
Das leben der Anderen (The Lives of Others)

# data[].attributes.headline
Don’t worry about AI going bad – the minds behind it are the danger | John Naughton

# data[].attributes.subject
Don’t worry about AI going bad – the minds behind it are the danger | John Naughton

# data[].attributes.attachment_link.readable_text
As the science fiction novelist William Gibson famously observed: "The future is already here - it's just not very evenly distributed." I wish people would pay more attention to that adage whenever the subject of artificial intelligence (AI) comes up. Public discourse about it invariably focuses on the threat (or promise, depending on your point of view) of "superintelligent" machines, ie ones that display human-level general intelligence, even though such devices have been 20 to 50 years away ever since we first started worrying about them. The likelihood (or mirage) of such machines still remains a distant prospect, a point made by the leading AI researcher Andrew Ng, who said that he worries about superintelligence in the same way that he frets about overpopulation on Mars.

That seems about right to me. If one were a conspiracy theorist, one might ask if our obsession with a highly speculative future has been deliberately orchestrated to divert attention from the fact - pace Mr Gibson - that lower-level but exceedingly powerful AI is already here and playing an ever-expanding role in shaping our economies, societies and politics. This technology is a combination of machine learning and big data and it's everywhere, controlled and deployed by a handful of powerful corporations, with occasional walk-on parts assigned to national security agencies.

These corporations regard this version of "weak" AI as the biggest thing since sliced bread. The CEO of Google burbles about "AI everywhere" in his company's offerings. Same goes for the other digital giants. In the face of this hype onslaught, it takes a certain amount of courage to stand up and ask awkward questions. If this stuff is so powerful, then surely we ought to be looking at how it is being used, asking whether it's legal, ethical and good for society - and thinking about what will happen when it gets into the hands of people who are even worse than the folks who run the big tech corporations. Because it will.

Fortunately, there are scholars who have started to ask these awkward questions. There are, for example, the researchers who work at AI Now, a research institute at New York University focused on the social implications of AI. Their 2017 report makes interesting reading. Last week saw the publication of more in the same vein - a new critique of the technology by 26 experts from six major universities, plus a number of independent thinktanks and NGOs.

The bots and fake Facebook accounts that currently pollute our public sphere will look awfully amateurish in a couple of years

Its title - The Malicious Use of Artificial Intelligence: Forecasting, Prevention and Mitigation - says it all. The report fills a serious gap in our thinking about this stuff. We've heard the hype, corporate and governmental, about the wonderful things AI can supposedly do and we've begun to pay attention to the unintentional downsides of legitimate applications of the technology. Now the time has come to pay attention to the really malign things bad actors could do with it.

The report looks at three main "domains" in which we can expect problems. One is digital security. The use of AI to automate tasks involved in carrying out cyber-attacks will alleviate the existing trade-off between the scale and efficacy of attacks. We can also expect attacks that exploit human vulnerabilities (for example, through the use of speech synthesis for impersonation), existing software vulnerabilities (through automated hacking) or the vulnerabilities of legitimate AI systems (through corruption of the data streams on which machine learning depends).

A second threat domain is physical security - attacks with drones and autonomous weapons systems. (Think v2.0 of the hobbyist drones that Isis deployed, but this time with face-recognition technology on board.) We can also expect new kinds of attacks that subvert physical systems - causing autonomous vehicles to crash, say - or ones deploying physical systems that would be impossible to remotely control from a distance: a thousand-strong swarm of micro-drones, for example.

Finally, there's what the authors call "political security" - using AI to automate tasks involved in surveillance, persuasion (creating targeted propaganda) and deception ( eg, manipulating videos). We can also expect new kinds of attack based on machine-learning's capability to infer human behaviours, moods and beliefs from available data. This technology will obviously be welcomed by authoritarian states, but it will also further undermine the ability of democracies to sustain truthful public debates. The bots and fake Facebook accounts that currently pollute our public sphere will look awfully amateurish in a couple of years.

The report is available as a free download and is worth reading in full. If it were about the dangers of future or speculative technologies, then it might be reasonable to dismiss it as academic scare-mongering. The alarming thing is most of the problematic capabilities that its authors envisage are already available and in many cases are currently embedded in many of the networked services that we use every day. William Gibson was right: the future has already arrived.
Topics

# data[].attributes.attachment_link.title
Don't worry about AI going bad - the minds behind it are the danger | John Naughton

# data[].attributes.attachment_link.description
As the science fiction novelist William Gibson famously observed: "The future is already here - it's just not very evenly distributed." I wish people would pay more attention to that adage whenever the subject of artificial intelligence (AI) comes up.

# data[].attributes.attachment_link.readable_text
This story is part of "Smart Creativity," a series exploring the intersection between high-concept design and advanced technology.

It's 2018 and cyborgs live among us. But these human-machine hybrids don't look the way movies might have you believe. Most blend into the crowd, their mechanical elements hidden from view.

Such is the case for Catalan avant-garde artist and cyborg activist Moon Ribas, who has implants connected to online seismographs in her feet. Any time there's an earthquake somewhere on the planet, vibrations course through her body and the data is recorded online.

Ribas then transforms that data into dance or music, often incorporating elements of spontaneity and uncertainty. For example, the movements in the dance "Waiting for Earthquakes," in which the artist stands perfectly still until seismic activity occurs, can take many shapes.

"I'm a dancer and a choreographer, so I wanted to experience movement in a deeper way," she explains. "Whenever there is an earthquake, I move according to the intensity of the earthquake. It's a bit like a duet between the earth and myself. Earth is actually the choreographer of the piece and I'm just imitating the data that she gives."

These implants aren't Ribas' first time using body hacks to create avant-garde art.

Previously, she's worn earrings that vibrated to reflect the speed of those walking around her (apparently pedestrians in London and Stockholm are particularly fast), and kaleidoscopic glasses that distorted her vision and color perception.

With her partner, Neil Harbisson, and their Cyborg Arts organization (an offshoot of the Cyborg Foundation they also founded), Ribas hopes to empower artists around the world to embrace new forms of technology to enhance their bodies and their work, and discover new ways to communicate phenomena that we cannot perceive with our senses.

"If you use technology, you can reveal this reality and get a deeper experience of the planet," she says.
Watch the video above to find out more about Moon Ribas and how technology informs her practice.

# data[].attributes.attachment_link.keywords[]
moon ribas: the cyborg dancer who can detect earthquakes - cnn

# data[].attributes.attachment_link.title
The cyborg dancer who can detect earthquakes

# data[].attributes.attachment_link.description
This story is part of "Smart Creativity," a series exploring the intersection between high-concept design and advanced technology. It's 2018 and cyborgs live among us. But these human-machine hybrids don't look the way movies might have you believe. Most blend into the crowd, their mechanical elements hidden from view.

# data[].attributes.attachment_link.description
Human innovation has transformed the way we live, often for the better. But as our technologies grow more powerful, so do their consequences. This hour, TED speakers explore technology's dark side.

# data[].attributes.attachment_link.description
Human innovation has transformed the way we live, often for the better. But as our technologies grow more powerful, so do their consequences. This hour, TED speakers explore technology's dark side.