# data[].attributes.body
Group Show Title:
AI Disaster : Terminator in Human Workplace
Group show link:
https://youtu.be/sP1y8haQZnQ

Group members:
Chang Shan 11292000
Ho Chiu Lam 11290985
Leung Ching Yi 11240124
Liu Chun Ming 11283750

Group show date:
2020.11.23

Time:
30 minutes

# data[].attributes.attachment_link.readable_text
時代變化、科技日新月異。近年AI人工智能技術越嚟越普及，就連女團成員都有AI角色。近日韓國電視台就推出首位AI人工智能女 主播 ，引起熱議。因為AI主播同真人幾乎無分別，只從外表睇根本分辨唔出。

韓國娛樂公司SM近日就推出新女團「 aespa 」，當中4位成員都各有一個AI人工智能角色，但AI同真人仍然有好大嘅差異。

韓國收費電視台MBN最近亦有一位AI人工智能主播，引起大眾熱議。雖然同為AI，但呢個AI主播明顯比女團aespa嘅AI偶像更令人震驚。

AI主播係以電視台新聞女主播金柱夏（김주하）為原型，同樣取名為「金柱夏（김주하）」，AI金柱夏報導新聞過後，即引發討論，因為AI同真人金柱夏太過相似，甚至可以話係一模一樣！

兩者只睇外貌根本分辨唔出邊個係真人，邊個係AI。朗讀新聞稿方面，AI雖然感情仍有不足，但發音、語氣都同真人差唔多。

AI金柱夏係整合咗真人金柱夏嘅外貌、動作同聲音後誕生，最快可以喺1分鐘內快速生成1千個字報道。

AI主播由MBN同人工智能開發公司共同研發，可以24小時唔休息，無間斷工作。希望發生災難時，AI主播快可以快速報導突發新聞，而且AI主播仲可以節省人力、時間同成本製作新節目。

對於AI主播，真人金柱夏表示擔心AI技術有一日會威脅到佢嘅地位：「我會盡力傳達AI暫時無法畀到人嘅溫暖感覺。」

Text: HolidaySmart Editorial

（小編公告：如果讀者有任何關於美食、旅遊、玩樂、或者任何新奇有趣事物既資訊，都歡迎話比小編們知～大家可以透過Facebook inbox、或者send e-mail到holiday@presslogic.com，直接將資料傳比小編）

# data[].attributes.attachment_link.title
韓國首位AI人工智能主播登場！可24小時工作 與真人相似度極高難分真假！ | HolidaySmart 假期日常

# data[].attributes.attachment_link.description
時代變化、科技日新月異。近年AI人工智能技術越嚟越普及，就連女團成員都有AI角色。近日韓國電視台就推出首位AI人工智能女 主播 ，引起熱議。因為AI主播同真人幾乎無分別，只從外表睇根本分辨唔出。 韓國娛樂公司SM近日就推出新女團「 aespa 」，當中4位成員都各有一個AI人工智能角色，但AI同真人仍然有好大嘅差異。 ...

# data[].attributes.headline
“AlphaZero學習國際象棋、日本將棋和圍棋，AlphaZero自我訓練國際象棋4小時後，便擊敗世界冠軍AI Stockfish(96%勝場)、日本將棋則訓練2小時，便擊敗世界冠軍AI Elmo(91%勝場)；圍棋則需30小時訓練，才能戰勝前代AlphaGo(61%勝場)。”

# data[].attributes.subject
“AlphaZero學習國際象棋、日本將棋和圍棋，AlphaZero自我訓練國際象棋4小時後，便擊敗世界冠軍AI Stockfish(96%勝場)、日本將棋則訓練2小時，便擊敗世界冠軍AI Elmo(91%勝場)；圍棋則需30小時訓練，才能戰勝前代AlphaGo(61%勝場)。”

# data[].attributes.body
https://www2.hkej.com/instantnews/international/article/2010239/AlphaZero%E9%9D%A0%E8%87%AA%E5%AD%B8%E4%B8%89%E6%A3%8B%E6%93%8A%E6%95%97%E5%85%B6%E4%BB%96AI

# data[].attributes.headline
日本「3DCG人造」美少女 Saya加入AI人工智慧 會揮手也會害羞

# data[].attributes.subject
日本「3DCG人造」美少女 Saya加入AI人工智慧 會揮手也會害羞

# data[].attributes.attachment_link.title
日本「3DCG 人造」美少女 Saya，加入 AI 人工智慧，會揮手也會害羞

# data[].attributes.headline
“2016年3月，李世乭對戰Google旗下子公司DeepMind開發的AI電腦圍棋系統AlphaGo，吸引全世界關注。

# data[].attributes.subject
“2016年3月，李世乭對戰Google旗下子公司DeepMind開發的AI電腦圍棋系統AlphaGo，吸引全世界關注。

# data[].attributes.body
李世乭與AlphaGo大戰5盤，雖然最終以1比4落敗。”

https://news.ltn.com.tw/news/world/breakingnews/2983229

# data[].attributes.headline
"報告從感測、規劃、執行三層面闡述自駕車技術的最新發展狀況，涉及感測器、資料處理、機器學習、SLAM 與感測器融合、路徑規劃等多領域，同時提供多家自動駕駛公司的案例，包括特斯拉、Volvo、Waymo 等。"

# data[].attributes.subject
"報告從感測、規劃、執行三層面闡述自駕車技術的最新發展狀況，涉及感測器、資料處理、機器學習、SLAM 與感測器融合、路徑規劃等多領域，同時提供多家自動駕駛公司的案例，包括特斯拉、Volvo、Waymo 等。"

# data[].attributes.body
https://technews.tw/2020/03/27/wevolver-2020-autonomous-vehicle-technology-report/

# data[].attributes.body
Japan the emerging country in Artificial intelligence

# data[].attributes.attachment_link.readable_text
All the things are becoming automated nowadays due to this to make the work less for human robots are introduced. Earlier the robots were given the specific task only and that specific task is done by those robots only but about year ago only we have got the first human-robot named Sophia which has almost all human emotions and after interacting in some interview people came to know that the robot era is not so far now and the technology that works behind the robots is artificial intelligence.

Many non-technical people wonder what exactly artificial intelligence? well, it is basically transferring or making any machine that has the human thought process and works accordingly and the field of Artificial Intelligence is getting wider and growing day by day.

Japan is the country which always impresses the world by the innovation and thus they are currently working majorly on the artificial intelligence based projects and in fact, they have started using many different kinds of robots in their daily routines such as Google home and the Amazon Alexa which helps in normal reminders, playing music, googling and the to-do list on the daily basis, while others are explained below and some other consequences are explained in further ways.

Why Japan is emerging in AI and robots?

In fact, one of the facts that Japan is leading in the robotics field is they have started making robots in ending of the 19th century and using them in either or other way and thus in the field of robotics and its innovations no one can beat Japan. They have developed robots which can interact and communicate in Japanese for them, they also have Robot restaurants and hotels where the whole hotel is run by the robots only even from the receptionist to the waiters for the serving purpose. Even Japan has more patents in artificial intelligence than other countries over the world this shows how the fastly the field is growing towards the robotics, firstly the idea of robots was developed to do the task that risks human life, the task that is time-consuming but can be done in less time if robots involved in that task.

where will be the robots can be used in future?

Nowadays we have robots not only for critical tasks but also to make life easier more and more, robots for the surgery assistance, for the hospitality,humanoid robot that exactly understands human language and accordingly communicates even it also gives the answer that you ask for, it can be your reminder for the events and your to-do list, in future we will also get the customize robots that can understand what you want even advice you in your critical situation apart from that it can also help and maintain your healthy lifestyle including your food.

We will also have cooking robots and household robots soon that will help to maintain your home, even in the office work you will be helped by the robots, in the medical surgeries you doctors will be assisted by the robots.

Not only that but Japan is introducing the sex robots which will be used for the sex purpose by the humans, this sex robot can be customized according to the owners choice and can also change the setting. In the recent news it is said that Japan is also planning to have the robots for the hospitality purpose in the Olympics 2020 where the sports person will enjoy the services and hospitality of the robots it can also be the test for the robots that how well they can do work for us, the whole world is looking forward to this.

Why robots should be manufactured with limitations?

Many people also questions that is it good to have robots for everything as there will be some cons side too, it's ok to have robots for the critical tasks which risk the human lives but to do every task is not a good idea as this will lead to unemployment more we human will have to be dependent on robots at one point of time also not everybody can afford a robot at their home this can lead to create more gap line between the high class and middle class societies and even people may faced the diseases due to the less workload and laziness and due to the loneliness in people it may turn up into depression which can cause the life loss of the humans.

So if the interaction with the people should also be maintained and with the robots equally the balancing will be needed to avoid problems such as life loss. In 2017 we have heard about the news that 4 robots killed 29 Japanese scientists this incidence has shocked the world of robotics and it is the major concern that what if they kill all humans they can be the reason of the human destruction in the world.

why robots can be harmful?

One incidence has been reported by One lady named Linda Moulton Howe which is Journalist and conspiracy theorist by profession claimed in 2018 at Conscious Life Expo in Los Angeles that 4 robots killed 29 Japanese scientists, it was a laboratory where the military purpose related robots are in the manufacturing mode and during this the shooting takes place and these killed the 29 Japanese scientists. so it is important to take care of some major issues like this so it doesn't destroy humanity in the world and disturb the world peace.

In a nutshell, it can be said that everything has pros and cons, and thus robots also have it. Japan will be the one country which will be giving us the majority of the robots in the near future and they will not increase its value but the market value of Japan will also be increased in the world. but several majors should also be taken care of so that this type of incident doesn't take place in the future. This automation is good in some aspects but there should be some restrictions added to it so that destruction should not take place.

# data[].attributes.attachment_link.title
Japan the emerging country in Artificial intelligence - Techiexpert.com

# data[].attributes.attachment_link.description
All the things are becoming automated nowadays due to this to make the work less for human robots are introduced.

# data[].attributes.headline
在家工作同時未雨綢繆　應對將來會被AI淘汰的工種及時裝備自己

# data[].attributes.subject
在家工作同時未雨綢繆　應對將來會被AI淘汰的工種及時裝備自己

# data[].attributes.attachment_link.readable_text
圖片來源：Unsplash@mann_pantoja

在今天人工智能（AI）已經非常普及，我們隨時隨地也體會到，上網、電話都充斥著，之前Pew Research就有研究指出，在發達地區有很多本來由人類擔任的工種在50年內會因為電腦而消失，而牛津大學亦有研究指出，未來10年內有47%的工作可能被機器取代。當中主要是生產、接待服務及交通運輸範疇，例如工廠工人及農民，其實早在19世紀工業革命時期，英國就已經有一班紡織業工人抗議工作被機器取代，他們被稱為「Luddites」。而司機也因為自動駕駛技術而被淘汰，Tesla已經在發展。當整合了交通系統，沒有人的因素在內甚至因此而解決了不少交通問題，同理之下，連運輸及快遞也可能因為自動駕駛及無人機的普及而失去人去操控的價值。至於接待員，遠端的如電話或網上平台，在這一刻已經在用上AI去處理了，解決了第一重的分流程序或處理簡單的查詢，至於實體的接待員機器人也是同理，應用起來人們甚至可能會覺得比真人更具新鮮感。這些都是非常之大的就業比例，還有軍人、保安這些工種更因為可以由機械人去取代而減輕了人身安全問題。還有一樣也是已經普及起來的，就是營銷，雖然人性對於營商而言非常重要，但AI的分析早已在進行，我們上網就能感受到，而主動的推銷如電話行銷也不需要人類去做。

至於難以被取代的工種，不難想到就是涉及創意及人性、需要溝通交流的工作。典型的創作者如作家、畫家，還有設計師，設計師最頭痛的工作相信並非創作，而是與客戶來來回回的斡旋，除了需要解決問題的智慧，還有交際手腕與溝通的能力。除了設計師還有另一個向來悲哀的工作就是軟件工程師，User Interface及User Experience的建構都要由人性角度去出發的，正所謂以人為本，而且重點也是要去處理難纏的客戶及很多了解不足而提出的要求。看來這些悲哀的工作是不會滅絕的。人才管理及人力資源也是同理，玩過三國志都知道，發現人才需要魅力值，而解決人與人之間的問題，解鈴還需繫鈴人，還是交由人類去處理吧。還有很多本質上是溝通的工作，如律師、社工、心理及精神病醫生等，當然，神父也是，因為難以想象AI會有堅定的信仰。

在這個WFH的狀態，正好反思一下自身的工種有沒有這個危機，如果有更多可運用的時間，不妨裝備一下自己。AI的年代不會在明早就降臨，即便AI可以處理大量有用的數據，但在這階段還是要由人的角度去分析和統籌，所以掌握數據運用非常重要。這十年的關鍵詞Big Data，其實本來是沒有意義的、浮遊充斥在大氣之中，一顆塵埃都可以有數據，而懂得梳理、篩選中間有用的數據為己所用才是道理。此外，掌握技術的朋友更要提高技術能力，因為有些人會被機器取代，但大量的機器也會由少數的人去控制或居中協調。最後，當然是保持、發展人類獨有的特性，創造力、應變能力、以及同理心，都不能被冰冷的機器取代，別太依賴系統而是運用系統，採取體驗式教育及學習，解決問題不只有單一角度，靈活運用及享受作為人的藝術吧。

# data[].attributes.attachment_link.title
在家工作同時未雨綢繆　應對將來會被AI淘汰的工種及時裝備自己

# data[].attributes.attachment_link.description
圖片來源：Unsplash@mann_pantoja 在今天人工智能（AI）已經非常普及，我們隨時隨地也體會到，上網、電話都充斥著，之前Pew ...

# data[].attributes.body
所有人都應該帶着被時代拋棄的恐懼來趕緊學習AI，不要害怕AI會搶奪人類的工作，要勇於學習數字技能，利用AI造福人類。

# data[].attributes.attachment_link.readable_text
2019年1月28日，由百仁基金與一國兩制研究中心聯合主辦「AI時代：職場‧人才‧教育」論壇，於1月28日下午假香港會議展覽中心演講廳舉行，是次論壇得到香港特別行政區政務司司長張建宗蒞臨主禮；並邀得多位著名學者及商界代表出席。各位嘉賓從不同角度分享個人見解，引領大家從職場、人才、教育等3方面深入探討AI時代，務求讓大家作出改革，擁抱未來。

李家傑：讓AI真正有益於人類發展

主辦機構百仁基金主席李家傑博士致辭時表示﹕「相信大家都希望可以為下一代爭取更多的資源、創造更好的環境，但大家應該愈來愈感受到AI迅速發展帶來的挑戰。我們都肩負着一個共同的責任，就是面對AI新時代，我們可以為香港的未來做些什麼？」

李家傑博士表示﹕「今天非常榮幸邀請到來自美國、香港和內地的AI專家和我們分享他們的真知灼見，相信大家都有豐富的收穫，有所啟發。而今天的論壇只是一個開始，我們會繼續努力，在社會各個層面、不同的領域共同推動AI的發展，匯聚眾人的智慧，擁抱時代的革新。我過去8年利用個人慈善基金，幫助了超過2萬1000名患有先天性心臟病的貧困兒童，同時時常擔憂兒童的未來會如何。如果他們無法在社會上找尋自己的位置與價值，又如何活得開心精彩？」

他希望可以善用創新科技去重新設計和改善我們的工作及生活環境，真正提升我們生活的幸福指數。其中最重要的是，以新科技為驅動去改革我們現行教育的內容、模式，甚至體制。他表示，作為一個起點，他會推動編程技能普及化，讓每一個人學生，包括基層學童，都有平等機會學習數字技能，讓AI的發展真正有益於人類的發展。

張建宗：不需過分憂慮AI取代人類

香港特別行政區政務司司長張建宗主禮致辭時，對李家傑博士的普及數字技能計劃表示振奮。他指AI技術席捲全球，將顛覆傳統商業模式。但大家不需過分憂慮AI取代人類，AI普及反而能夠為人類帶來積極正面影響。張建宗司長勉勵年輕人了解社會發展需要和趨勢，早日為迎接新趨勢轉型。

朱松純：開創AI+教育的新時代

到了主題演講時，美國加州大學洛杉磯分校統計學系與計算機系教授朱松純教授以「如何用人工智能提升人類福祉」為主題，當中帶出AI是第四次工業革命，將帶領社會進入大統一時代。他提出強認知AI可以理解人的意圖，甚至達到因果推理，對中國教育及社會發展有重要意義。扶貧先扶智，善用AI扶智計劃於少兒教育，讓學生利用機器人的引導式陪伴學習，讓教育全面達至解惑、授業、傳道理念。AI扶智計劃已推廣至貧困藏區學生，促進教育資源均衡化，開創AI+教育的新時代。

美國上市公司「流利說」創始人、董事長及首席執行官王翌博士的主題為「AI+教育──開啟智慧學習新時代」，他特別指出人工智能的應用可以多方面造福人類，尤其在教育方面，可以幫助來自城鄉各階層的學生追求夢想和獲得智慧。此外更可利用大數據訂製個人化學習路徑，學生可以更高效地學習，不必規範於傳統學制年期。

而ExaLeap首席執行官兼首席技術官江朝暉女士則集中提出「在萬物互聯的世界找準定位」有關之方向，主要說到AI分開訊息採集，數據理解，及轉變應用三個階段。江朝暉女士強調未來5至10年的工作機會主要來自數據理解，需要大量科學家和編程人員分析及教育機械人及電腦理解數據，進而讓AI自動化作預測分析，轉變數據為應用。她亦對香港政府多個推動吸引AI人才的措施及計劃表示肯定及期待香港轉型，希望年輕人持續學習，迎接新機會。

黃錦輝：籲學生早日為未來事業裝備自己

香港中文大學工程學院副院長（外務）黃錦輝教授接着從「AI如何改變香港教育的未來」帶領大家探討教育的未來。黃錦輝教授解釋，人工智能就是雲計算、大數據及機器學習的合體，並指人工智能可作為平台支撐金融科技，智慧城市及生物技術。AI與金融結合誕生的未來金融工程有利造就強大可靠的系統以分析金融市場，而AI與服務結合的機器人將有效貢獻醫療保健。黃錦輝教授預視STEM教育（科學、技術、工程及數學）將提升人類在AI科技發展領域的競爭力，籲學生早日為未來事業裝備自己。

4位主講嘉賓分別作出分享後，由香港教育大學校長及公共政策講座教授張仁良教授作為論壇主持，帶領4位主講嘉賓及百仁基金第十屆主席、恒基兆業地產集團副主席、一國兩制研究中心主席李家傑博士在台上進行討論，並接受台下來賓發問，論壇討論非常熱烈，當中帶出了幾個重點﹕AI協助兒童開發學習興趣，不再過分着重考試成績，而忽略價值觀及理解的高層次教育。所有人都應該帶着被時代拋棄的恐懼來趕緊學習AI，不要害怕AI會搶奪人類的工作，要勇於學習數字技能，利用AI造福人類。教育需要根本的改變，鼓勵老師善用STEM教育及收集教育數據，分析數字後制定嶄新貼近現代的教學方案。最後更呼籲所有人要終身學習，利用既有AI模塊及平台學習AI，進而並積極開發新的AI技術，更重要是AI並非純粹將工作外包給電腦，要理解其背後對人類發展進步的重要啟發意義。

經過多位著名學者及商界代表積極分享及討論後，相信大家必定能為人工智能的大趨勢做好準備，並抓住人工智能的大趨勢，積極應對人工智能對就業市場的衝擊；同時幫助經濟和社會轉型，保持香港在全球競爭中的優勢地位。培養青年成為全人，駕馭、超越改變、不被機器人取代，達至全球教育改革的關鍵目標。

在此論壇開始前一小時，百仁基金邀請了17間中學280多名學生在隔壁會場，直接與李家傑博士及香港城市大學電子工程學系副教授張澤松博士一起討論AI時代，學生們都較為擔心未來就業前景，以及如何裝備自己迎接AI時代的來臨。

# data[].attributes.attachment_link.title
香港怎樣應對AI時代？|編輯精選

# data[].attributes.attachment_link.description
2019年1月28日，由百仁基金與一國兩制研究中心聯合主辦「AI時代：職場‧人才‧教育」論壇，於1月28日下午假香港會議展覽中心演講廳舉行，是次論壇得到香港特別行政區政務司司長張建宗蒞臨主禮；並邀得多位著名學者及商界代表出席。各位嘉賓從不同角度分享個人見解，引領大家從職場、人才、教育等3方面深入探討AI時代，務求讓大家作出改革，擁抱未來。 ...

# data[].attributes.body
Apple Siri 與 Google Now均有較高技術的語音系統

https://funiphone.pixnet.net/blog/post/37711402-apple-%E8%88%87-android-%E4%B9%8B%E9%96%93%E7%9A%84%E6%88%B0%E7%88%AD%EF%BC%8C%E8%AA%B0%E6%98%AF%E6%9C%80%E5%BC%B7%E7%9A%84%E8%AA%9E%E9%9F%B3%E5%8A%A9

# data[].attributes.attachment_link.readable_text
This story is part of a group of stories called

Finding the best ways to do good.

When I heard five or so years back that people in Silicon Valley were getting worried about artificial intelligence causing human extinction, my initial reaction was extreme skepticism.

A large reason for that was that the scenario just felt silly. What did these folks think would happen - was some company going to build Skynet and manufacture Terminator robots to slaughter anyone who stood in their way? It felt like a sci-fi fantasy, not a real problem.

This is a misperception that frustrates a lot of AI researchers. Nate Soares, who runs the Machine Intelligence Research Institute, which focuses on AI safety, has argued a better analogy than the Terminator is the "Sorcerer's Apprentice" scene in Fantasia. The problem isn't that AI will suddenly decide we all need to die; the problem is that we might give it instructions that are vague or incomplete and that lead to the AI following our orders in ways we didn't intend.

"The problem Mickey faces when he enchants a broom to help him fill a cauldron isn't that the broom rebels or acquires a will of its own, but that it fulfills the task it was given all too well," Soares stated in an interview last year. "He wants the cauldron full, and overflowing the workshop is a great way to be extra confident that the cauldron is full (and stays full). Mickey successfully 'aimed' his AI system but things still went poorly for Mickey."

This - not Soares's specific metaphor, but the observation that badly directed automated systems can lead to massive unintended consequences - is what made me start to agree with the argument that AI poses a major risk. I don't think AI has malevolent motives, or is really capable of having malevolent motives. I definitely think humans are bad at precisely stating what they want, and that combined with a powerful enough technology, human failure could lead to very bad outcomes.

So what does that look like at scale? Computer scientist Paul Christiano, who works on AI safety at OpenAI, has a nice recent post laying out two broad scenarios. And I do mean "broad": As a journalist, I've found it hard to get AI safety researchers to lay out a single concrete path through which they think AI could lead to doom, mostly because no one thinks any one particular scenario has a very high likelihood. The claim, rather, is that the odds of all possible catastrophically bad scenarios when added up represent a significant risk. But this makes it somewhat difficult to visualize the kind of outcome they fear.

Christiano's categories make it a little bit easier, though. He divides potential disasters into two broad categories:

1. Going out with a whimper

2. Going out with a bang

Going out with a whimper

Human institutions are, already, better at maximizing easy-to-measure outcomes than hard-to-measure outcomes. It's easier to increase standardized math test scores than it is to increase students' actual math knowledge. It's easier to cut reported robberies than it is to prevent actual robberies.

Machine-learning algorithms share this flaw, and exaggerate it in some ways. They are incredibly good at figuring out through trial and error how to achieve a human-specified quantitative goal. But humans aren't always good at specifying those goals, and AIs are not good at distinguishing between reasonable interpretations of human instructions and unreasonable interpretations.

Suppose you're the CEO of, say, Nike, and you have a new AI that can recommend approaches for maximizing your profits. If it recommends a new sneaker that will likely be incredibly popular, that's great! If it instead recommends large-scale money laundering and targeted assassinations of Reebok customers to strike fear in the consumer markets - that's less great!

That's a dramatic example (and it's mine, not Christiano's). As Christiano notes, we might be able to avoid these kinds of outcomes early on by just changing code to guard against obvious bad results. "For a while we will be able to overcome these problems by recognizing them ... and imposing ad-hoc restrictions that avoid manipulation or abuse," he writes. "But as the system becomes more complex, that job itself becomes too challenging for human reasoning to solve directly."

And by that point, there might not be agreement among humans that things are going awry, and without such agreement, it's doubtful regulations and controls could be imposed: "As this world goes off the rails, there may not be any discrete point where consensus recognizes that things have gone off the rails."

Humans may just accept ceding more and more decision-making authority to algorithms until much of human life appears to consist of humans implementing the recommendations of AI systems they believe to be smarter. That's particularly true if lobbyists for AI-development companies like Google and Facebook fight aggressively against any regulations that limit the role of AI.

Even this might sound sci-fi-ish. But ask yourself: Do you trust your ability to find the cheapest flight when you have to go from New York to San Francisco? Or do you trust algorithms at Kayak or Hipmunk? I, personally, tend to trust Hipmunk.

If you had the ability to pay for an AI stock adviser, like the one Betterment already sells, would you trust it over your own ability to pick stocks? Probably, right? What about when you're a college senior nervous about career choices, and a sophisticated AI recommends possible job paths? At what point do you jump off the train? Or do you just keep trusting the algorithms?

Going out with a bang

Even so, Christiano's first scenario doesn't precisely envision human extinction. It envisions human irrelevance, as we become agents of machines we created.

His second scenario is somewhat bloodier. Often, he notes, the best way to achieve a given goal is to obtain influence over other people who can help you achieve that goal. If you are trying to launch a startup, you need to influence investors to give you money and engineers to come work for you. If you're trying to pass a law, you need to influence advocacy groups and members of Congress.

That means that machine-learning algorithms will probably, over time, produce programs that are extremely good at influencing people. And it's dangerous to have machines that are extremely good at influencing people.

"Early in the trajectory, influence-seeking systems mostly acquire influence by making themselves useful and looking as innocuous as possible," Christiano writes. "They may provide useful services in the economy in order to make money for them and their owners, make apparently-reasonable policy recommendations in order to be more widely consulted for advice, try to help people feel happy, etc."

But eventually, the algorithms' incentives to expand influence might start to overtake their incentives to achieve the specified goal. That, in turn, makes the AI system worse at achieving its intended goal, which increases the odds of some terrible failure. And if a failure occurs, well:

An unrecoverable catastrophe would probably occur during some period of heightened vulnerability - a conflict between states, a natural disaster, a serious cyberattack, etc. -since that would be the first moment that recovery is impossible and would create local shocks that could precipitate catastrophe. The catastrophe might look like a rapidly cascading series of automation failures: A few automated systems go off the rails in response to some local shock. As those systems go off the rails, the local shock is compounded into a larger disturbance; more and more automated systems move further from their training distribution and start failing. Realistically this would probably be compounded by widespread human failures in response to fear and breakdown of existing incentive systems.

Human reliance on these systems, combined with the systems failing, leads to a massive societal breakdown. And in the wake of the breakdown, there are still machines that are great at persuading and influencing people to do what they want, machines that got everyone into this catastrophe and yet are still giving advice that some of us will listen to.

Naturally, both of Christiano's scenarios are fairly theoretical. But we can already see what increased reliance on AI looks like in our daily lives. It feels probable that the number of services like Hipmunk or Google Maps - commonly used algorithms that humans tend to trust more than they trust other humans - will multiply. And as Christiano suggests, that could have some major unintended consequences.
Sign up for the Future Perfect newsletter. Will you help keep Vox free for all? Twice a week, you'll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and - to put it simply - getting better at doing good.
Millions of people rely on Vox to understand how the policy decisions made in Washington, from health care to unemployment to housing, could impact their lives. Our work is well-sourced, research-driven, and in-depth. And that kind of work takes resources. Even after the economy recovers, advertising alone will never be enough to support it. If you have already made a contribution to Vox, thank you. If you haven't, help us keep our journalism free for everyone by making a financial contribution today, from as little as $3.

# data[].attributes.attachment_link.title
AI disaster won't look like the Terminator. It'll be creepier.

# data[].attributes.attachment_link.description
Finding the best ways to do good. When I heard five or so years back that people in Silicon Valley were getting worried about artificial intelligence causing human extinction, my initial reaction was extreme skepticism. A large reason for that was that the scenario just felt silly.

# data[].attributes.body
https://drive.google.com/file/d/1ZaHwmd7yb3qXr-SUeZhAjAfJRnuredwe/view?usp=sharing