# data[].attributes.headline
ChatGPT: Mayor starts legal bid over false bribery claim

# data[].attributes.subject
ChatGPT: Mayor starts legal bid over false bribery claim

# data[].attributes.attachment_link.readable_text
An Australian mayor said he may take legal action over false information shared by advanced chatbot ChatGPT.

Brian Hood, Mayor of Hepburn Shire Council, says the OpenAI-owned tool falsely claimed he was imprisoned for bribery while working for a subsidiary of Australia's national bank.

In fact, Mr Hood was a whistleblower and was never charged with a crime.

His lawyers have sent a concerns notice to OpenAI - the first formal step in defamation action in Australia.

OpenAI has 28 days to respond to the concerns notice, after which time Mr Hood would be able to take the company to court under Australian law.

If he pursues the legal claim, it would be the first time OpenAI has publicly faced a defamation suit over the content created by ChatGPT.

OpenAI has not responded to a BBC request for comment.

Millions of people have used ChatGPT since it launched in November 2022.

It can answer questions using natural, human-like language and it can also mimic other writing styles, using the internet as it was in 2021 as its database.

Microsoft has spent billions of dollars on it and it was added to Bing in February 2023.

'Plausible-sounding but incorrect'

When people use ChatGPT, they are shown a disclaimer warning that the content it generates may contain "inaccurate information about people, places, or facts".

And on its public blog about the tool, OpenAI says a limitation is that it "sometimes writes plausible-sounding but incorrect or nonsensical answers".

In 2005, Mr Hood was company secretary of Notes Printing Australia, a subsidiary of the Reserve Bank of Australia.

He told journalists and officials about bribery taking place at the organisation linked to a business called Securency, which was part-owned by the bank.

Securency was raided by police in 2010, ultimately leading to arrests and prison sentences worldwide.

Mr Hood was not one of those arrested, and said he was "horrified" to see what ChatGPT was telling people.

"I was stunned at first that it was so incorrect," he told Australian broadcaster ABC News.

"It's one thing to get something a little bit wrong, it's entirely something else to be accusing someone of being a criminal and having served jail time when the truth is the exact opposite.

"I think this is a pretty stark wake-up call. The system is portrayed as being credible and informative and authoritative, and it's obviously not."

Different chatbots, different answers

The BBC was able to confirm Mr Hood's claims by asking the publicly available version of ChatGPT on OpenAI's website about the role he had in the Securency scandal.

It responded with a description of the case, then inaccurately stated that he "pleaded guilty to one count of bribery in 2012 and was sentenced to four years in prison".

But the same result does not appear in the newer version of ChatGPT which is integrated into Microsoft's Bing search engine.

It correctly identifies him as a whistleblower, and specifically says he "was not involved in the payment of bribes... as claimed by an AI chatbot called ChatGPT".

# data[].attributes.attachment_link.title
ChatGPT: Mayor starts legal bid over false bribery claim

# data[].attributes.attachment_link.description
An Australian mayor said he may take legal action over false information shared by advanced chatbot ChatGPT. Brian Hood, Mayor of Hepburn Shire Council, says the OpenAI-owned tool falsely claimed he was imprisoned for bribery while working for a subsidiary of Australia's national bank.

# data[].attributes.headline
Surprising things happen when you put 25 AI agents together in an RPG townResearchers study emergent AI behaviors in a sandbox world inspired by The Sims.

# data[].attributes.subject
Surprising things happen when you put 25 AI agents together in an RPG townResearchers study emergent AI behaviors in a sandbox world inspired by The Sims.

# data[].attributes.attachment_link.readable_text
A group of researchers at Stanford University and Google have created a miniature RPG-style virtual world similar to The Sims, where 25 characters, controlled by ChatGPT and custom code, live out their lives independently with a high degree of realistic behavior. They wrote about their experiment in a preprint academic paper released on Friday.

"Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day," write the researchers in their paper, "Generative Agents: Interactive Simulacra of Human Behavior."

To pull this off, the researchers relied heavily on a large language model (LLM) for social interaction, specifically the ChatGPT API. In addition, they created an architecture that simulates minds with memories and experiences, then let the agents loose in the world to interact. And humans can interact with them, too.

"Users can observe and intervene as agents plan their days, share news, form relationships, and coordinate group activities," they write. It's the work of Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein.

Computer and video games have included computer-controlled characters since the 1970s, but never before have they been able to simulate a social environment with the complexity of natural language that might now be possible thanks to generative AI models like ChatGPT. While the group's research is not necessarily a "game," it could be a prototype of a future where dynamic RPG characters interact in complex and unexpected ways.

"Imagine killing an NPC and coming back to the city and seeing a funeral for them," joked a Twitter user named Dennis Hansen when replying to a thread about the paper's emergent implications. Judging by this research, that might not be a far-fetched scenario.

Life in Smallville

To study the group of AI agents, the researchers set up a virtual town called "Smallville," which includes houses, a cafe, a park, and a grocery store. For human interaction purposes, the world is represented on screen from an overhead view using retro-style pixel graphics reminiscent of a classic 16-bit Japanese RPG.

Smallville is home to a community of 25 distinct individuals, each represented by a basic sprite avatar. To capture each agent's identity and their connections with other members of the community, the researchers created a paragraph of natural language description as a seed memory. These descriptions include details about each agent's occupation and relationships with other agents. For example, here's an excerpt of one such seed memory provided in the paper:

John Lin is a pharmacy shopkeeper at the Willow Market and Pharmacy who loves to help people. He is always looking for ways to make the process of getting medication easier for his customers. John Lin is living with his wife, Mei Lin, who is a college professor, and son, Eddy Lin, who is a student studying music theory. John Lin loves his family very much.

As a virtual environment, Smallville is broken into both areas and objects. Human users can enter the world as an existing or new agent, and both users and agents can influence the state of objects through actions. Human users can also interact with AI agents through conversation or by issuing directives as an "inner voice." Users communicate in natural language, specifying a persona that the agent perceives them as, or can use the inner voice to influence the agent's actions.

# data[].attributes.attachment_link.title
Surprising things happen when you put 25 AI agents together in an RPG town

# data[].attributes.attachment_link.description
A group of researchers at Stanford University and Google have created a miniature RPG-style virtual world similar to The Sims, where 25 characters, controlled by ChatGPT and custom code, live out their lives independently with a high degree of realistic behavior. They wrote about their experiment in a preprint academic paper released on Friday.

# data[].attributes.attachment_link.readable_text
/

The 'My AI' bot will initially only be available to paying Snapchat Plus subscribers. CEO Evan Spiegel says it's just the beginning for the company's generative AI plans.

Snapchat is introducing a chatbot powered by the latest version of OpenAI's ChatGPT. According to Snap CEO Evan Spiegel, it's a bet that AI chatbots will increasingly become a part of everyday life for more people.

Named "My AI," Snapchat's bot will be pinned to the app's chat tab above conversations with friends. While initially only available for $3.99 a month Snapchat Plus subscribers, the goal is to eventually make the bot available to all of Snapchat's 750 million monthly users, Spiegel tells The Verge.

"The big idea is that in addition to talking to our friends and family every day, we're going to talk to AI every day," he says. "And this is something we're well positioned to do as a messaging service."

At launch, My AI is essentially just a fast mobile-friendly version of ChatGPT inside Snapchat. The main difference is that Snap's version is more restricted in what it can answer. Snap's employees have trained it to adhere to the company's trust and safety guidelines and not give responses that include swearing, violence, sexually explicit content, or opinions about dicey topics like politics.

It has also been stripped of functionality that has already gotten ChatGPT banned in some schools; I tried getting it to write academic essays about various topics, for example, and it politely declined. Snap plans to keep tuning My AI as more people use it and report inappropriate answers. (I wasn't able to conjure any in my own testing, though I'm sure others will.)

After trying My AI, it's clear that Snap doesn't feel the need to even explain the phenomenon that is ChatGPT, which is a testament to OpenAI building the fastest-growing consumer software product in history. Unlike OpenAI's own ChatGPT interface, I wasn't shown any tips or guardrails for interacting with Snap's My AI. It opens to a blank chat page, waiting for a conversation to start.

While ChatGPT has quickly become a productivity tool, Snap's implementation treats generative AI more like a persona. My AI's profile page looks like any other Snapchat user's profile, albeit with its own alien Bitmoji. The design suggests that My AI is meant to be another friend inside of Snapchat for you to hang out with, not a search engine.

"The big idea is that in addition to talking to our friends and family every day, we're going to talk to AI every day."

That distinction could save Snap some headaches. As Bing's implementation of OpenAI's tech has shown, the large language models (LLMs) underpinning these chatbots can confidently give wrong answers, or hallucinations, that are problematic in the context of search. If toyed with enough, they can even be emotionally manipulative and downright mean. It's a dynamic that has, at least so far, kept larger players in the space - namely Google and Meta - from releasing competing products to the public.

Snap is in a different place. It has a deceivingly large and young user base, but its business is struggling. My AI will likely be a boost to the company's paid subscriber numbers in the short term, and eventually, it could open up new ways for the company to make money, though Spiegel is cagey about his plans.

Snap is one of the first clients of OpenAI's new enterprise tier called Foundry, which lets companies run its latest GPT-3.5 model with dedicated compute designed for large workloads. Spiegel says Snap will likely incorporate LLMs from other vendors besides OpenAI over time and that it will use the data gathered from the chatbot to inform its broader AI efforts. While My AI is basic to start, it's the beginning of what Spiegel sees as a major investment area for Snap and, more importantly, a future in which we're all talking to AI like it's a person.

Sign up for Command Line, a paid weekly newsletter from Alex Heath about the tech industry's inside conversation.

$7/month

SUBSCRIBE

$70/year

SUBSCRIBE

$60/person/year

SUBSCRIBE

We accept credit card, Apple Pay and Google Pay.

# data[].attributes.attachment_link.title
Snapchat is releasing its own AI chatbot powered by ChatGPT

# data[].attributes.attachment_link.description
Snapchat is introducing a chatbot powered by the latest version of OpenAI's ChatGPT. According to Snap CEO Evan Spiegel, it's a bet that AI chatbots will increasingly become a part of everyday life for more people. Named "My AI," Snapchat's bot will be pinned to the app's chat tab above conversations with friends.

# data[].attributes.headline
Artificial intelligence model detects asymptomatic Covid-19 infections 

# data[].attributes.subject
Artificial intelligence model detects asymptomatic Covid-19 infections 

# data[].attributes.body
AI can detect COVID-19 infections through cellphone recorded coughs

# data[].attributes.attachment_link.provider_name
MIT News | Massachusetts Institute of Technology

# data[].attributes.attachment_link.readable_text
Asymptomatic people who are infected with Covid-19 exhibit, by definition, no discernible physical symptoms of the disease. They are thus less likely to seek out testing for the virus, and could unknowingly spread the infection to others.

But it seems those who are asymptomatic may not be entirely free of changes wrought by the virus. MIT researchers have now found that people who are asymptomatic may differ from healthy individuals in the way that they cough. These differences are not decipherable to the human ear. But it turns out that they can be picked up by artificial intelligence.

In a paper published recently in the IEEE Journal of Engineering in Medicine and Biology, the team reports on an AI model that distinguishes asymptomatic people from healthy individuals through forced-cough recordings, which people voluntarily submitted through web browsers and devices such as cellphones and laptops.

The researchers trained the model on tens of thousands of samples of coughs, as well as spoken words. When they fed the model new cough recordings, it accurately identified 98.5 percent of coughs from people who were confirmed to have Covid-19, including 100 percent of coughs from asymptomatics - who reported they did not have symptoms but had tested positive for the virus.

The team is working on incorporating the model into a user-friendly app, which if FDA-approved and adopted on a large scale could potentially be a free, convenient, noninvasive prescreening tool to identify people who are likely to be asymptomatic for Covid-19. A user could log in daily, cough into their phone, and instantly get information on whether they might be infected and therefore should confirm with a formal test.

"The effective implementation of this group diagnostic tool could diminish the spread of the pandemic if everyone uses it before going to a classroom, a factory, or a restaurant," says co-author Brian Subirana, a research scientist in MIT's Auto-ID Laboratory.

Subirana's co-authors are Jordi Laguarta and Ferran Hueto, of MIT's Auto-ID Laboratory.
Vocal sentiments
Prior to the pandemic's onset, research groups already had been training algorithms on cellphone recordings of coughs to accurately diagnose conditions such as pneumonia and asthma. In similar fashion, the MIT team was developing AI models to analyze forced-cough recordings to see if they could detect signs of Alzheimer's, a disease associated with not only memory decline but also neuromuscular degradation such as weakened vocal cords.

They first trained a general machine-learning algorithm, or neural network, known as ResNet50, to discriminate sounds associated with different degrees of vocal cord strength. Studies have shown that the quality of the sound "mmmm" can be an indication of how weak or strong a person's vocal cords are. Subirana trained the neural network on an audiobook dataset with more than 1,000 hours of speech, to pick out the word "them" from other words like "the" and "then."

The team trained a second neural network to distinguish emotional states evident in speech, because Alzheimer's patients - and people with neurological decline more generally - have been shown to display certain sentiments such as frustration, or having a flat affect, more frequently than they express happiness or calm. The researchers developed a sentiment speech classifier model by training it on a large dataset of actors intonating emotional states, such as neutral, calm, happy, and sad.

The researchers then trained a third neural network on a database of coughs in order to discern changes in lung and respiratory performance.

Finally, the team combined all three models, and overlaid an algorithm to detect muscular degradation. The algorithm does so by essentially simulating an audio mask, or layer of noise, and distinguishing strong coughs - those that can be heard over the noise - over weaker ones.

With their new AI framework, the team fed in audio recordings, including of Alzheimer's patients, and found it could identify the Alzheimer's samples better than existing models. The results showed that, together, vocal cord strength, sentiment, lung and respiratory performance, and muscular degradation were effective biomarkers for diagnosing the disease.

When the coronavirus pandemic began to unfold, Subirana wondered whether their AI framework for Alzheimer's might also work for diagnosing Covid-19, as there was growing evidence that infected patients experienced some similar neurological symptoms such as temporary neuromuscular impairment.

"The sounds of talking and coughing are both influenced by the vocal cords and surrounding organs. This means that when you talk, part of your talking is like coughing, and vice versa. It also means that things we easily derive from fluent speech, AI can pick up simply from coughs, including things like the person's gender, mother tongue, or even emotional state. There's in fact sentiment embedded in how you cough," Subirana says. "So we thought, why don't we try these Alzheimer's biomarkers [to see if they're relevant] for Covid."
"A striking similarity"
In April, the team set out to collect as many recordings of coughs as they could, including those from Covid-19 patients. They established a website where people can record a series of coughs, through a cellphone or other web-enabled device. Participants also fill out a survey of symptoms they are experiencing, whether or not they have Covid-19, and whether they were diagnosed through an official test, by a doctor's assessment of their symptoms, or if they self-diagnosed. They also can note their gender, geographical location, and native language.

To date, the researchers have collected more than 70,000 recordings, each containing several coughs, amounting to some 200,000 forced-cough audio samples, which Subirana says is "the largest research cough dataset that we know of." Around 2,500 recordings were submitted by people who were confirmed to have Covid-19, including those who were asymptomatic.

The team used the 2,500 Covid-associated recordings, along with 2,500 more recordings that they randomly selected from the collection to balance the dataset. They used 4,000 of these samples to train the AI model. The remaining 1,000 recordings were then fed into the model to see if it could accurately discern coughs from Covid patients versus healthy individuals.

Surprisingly, as the researchers write in their paper, their efforts have revealed "a striking similarity between Alzheimer's and Covid discrimination."

Without much tweaking within the AI framework originally meant for Alzheimer's, they found it was able to pick up patterns in the four biomarkers - vocal cord strength, sentiment, lung and respiratory performance, and muscular degradation - that are specific to Covid-19. The model identified 98.5 percent of coughs from people confirmed with Covid-19, and of those, it accurately detected all of the asymptomatic coughs.

"We think this shows that the way you produce sound, changes when you have Covid, even if you're asymptomatic," Subirana says.
Asymptomatic symptoms
The AI model, Subirana stresses, is not meant to diagnose symptomatic people, as far as whether their symptoms are due to Covid-19 or other conditions like flu or asthma. The tool's strength lies in its ability to discern asymptomatic coughs from healthy coughs.

The team is working with a company to develop a free pre-screening app based on their AI model. They are also partnerning with several hospitals around the world to collect a larger, more diverse set of cough recordings, which will help to train and strengthen the model's accuracy.

As they propose in their paper, "Pandemics could be a thing of the past if pre-screening tools are always on in the background and constantly improved."

Ultimately, they envision that audio AI models like the one they've developed may be incorporated into smart speakers and other listening devices so that people can conveniently get an initial assessment of their disease risk, perhaps on a daily basis.

# data[].attributes.attachment_link.title
Artificial intelligence model detects asymptomatic Covid-19 infections through cellphone-recorded coughs

# data[].attributes.attachment_link.description
Asymptomatic people who are infected with Covid-19 exhibit, by definition, no discernible physical symptoms of the disease. They are thus less likely to seek out testing for the virus, and could unknowingly spread the infection to others. But it seems those who are asymptomatic may not be entirely free of changes wrought by the virus.

# data[].attributes.body
Interesting explanation of K -Means Cluster Analysis , 3 most popular methods of clustering

# data[].attributes.attachment_link.readable_text
Clustering is a broad set of techniques for finding subgroups of observations within a data set. When we cluster observations, we want observations in the same group to be similar and observations in different groups to be dissimilar. Because there isn't a response variable, this is an unsupervised method, which implies that it seeks to find relationships between the observations without being trained by a response variable. Clustering allows us to identify which observations are alike, and potentially categorize them therein. K-means clustering is the simplest and the most commonly used clustering method for splitting a dataset into a set of k groups.

tl;dr

This tutorial serves as an introduction to the k-means clustering method.

1. Replication Requirements: What you'll need to reproduce the analysis in this tutorial

2. Data Preparation: Preparing our data for cluster analysis

3. Clustering Distance Measures: Understanding how to measure differences in observations

4. K-Means Clustering: Calculations and methods for creating K subgroups of the data

5. Determining Optimal Clusters: Identifying the right number of clusters to group your data

Replication Requirements

To replicate this tutorial's analysis you will need to load the following packages:

Data Preparation

To perform a cluster analysis in R, generally, the data should be prepared as follows:

1. Rows are observations (individuals) and columns are variables

2. Any missing value in the data must be removed or estimated.

3. The data must be standardized (i.e., scaled) to make variables comparable. Recall that, standardization consists of transforming the variables such that they have mean zero and standard deviation one.

Here, we'll use the built-in R data set USArrests, which contains statistics in arrests per 100,000 residents for assault, murder, and rape in each of the 50 US states in 1973. It includes also the percent of the population living in urban areas

To remove any missing value that might be present in the data, type this:

As we don't want the clustering algorithm to depend to an arbitrary variable unit, we start by scaling/standardizing the data using the R function scale:

Clustering Distance Measures

The classification of observations into groups requires some methods for computing the distance or the (dis)similarity between each pair of observations. The result of this computation is known as a dissimilarity or distance matrix. There are many methods to calculate this distance information; the choice of distance measures is a critical step in clustering. It defines how the similarity of two elements (x, y) is calculated and it will influence the shape of the clusters.

The choice of distance measures is a critical step in clustering. It defines how the similarity of two elements (x, y) is calculated and it will influence the shape of the clusters. The classical methods for distance measures are Euclidean and Manhattan distances, which are defined as follow:
Euclidean distance: Manhattan distance:
Where, x and y are two vectors of length n.

Other dissimilarity measures exist such as correlation-based distances, which is widely used for gene expression data analyses. Correlation-based distance is defined by subtracting the correlation coefficient from 1. Different types of correlation methods can be used such as:
Pearson correlation distance: Spearman correlation distance:
The spearman correlation method computes the correlation between the rank of x and the rank of y variables.

Where and .
Kendall correlation distance:
Kendall correlation method measures the correspondence between the ranking of x and y variables. The total number of possible pairings of x with y observations is n(n − 1)/2, where n is the size of x and y. Begin by ordering the pairs by the x values. If x and y are correlated, then they would have the same relative rank orders. Now, for each , count the number of (concordant pairs (c)) and the number of (discordant pairs (d)).

Kendall correlation distance is defined as follow:

The choice of distance measures is very important, as it has a strong influence on the clustering results. For most common clustering software, the default distance measure is the Euclidean distance. However, depending on the type of the data and the research questions, other dissimilarity measures might be preferred and you should be aware of the options.

Within R it is simple to compute and visualize the distance matrix using the functions get_dist and fviz_dist from the factoextra R package. This starts to illustrate which states have large dissimilarities (red) versus those that appear to be fairly similar (teal).

get_dist: for computing a distance matrix between the rows of a data matrix. The default distance computed is the Euclidean; however, get_dist also supports distanced described in equations 2-5 above plus others.

fviz_dist: for visualizing a distance matrix

K-Means Clustering

K-means clustering is the most commonly used unsupervised machine learning algorithm for partitioning a given data set into a set of k groups (i.e. k clusters), where k represents the number of groups pre-specified by the analyst. It classifies objects in multiple groups (i.e., clusters), such that objects within the same cluster are as similar as possible (i.e., high intra-class similarity), whereas objects from different clusters are as dissimilar as possible (i.e., low inter-class similarity). In k-means clustering, each cluster is represented by its center (i.e, centroid) which corresponds to the mean of points assigned to the cluster.

The Basic Idea

The basic idea behind k-means clustering consists of defining clusters so that the total intra-cluster variation (known as total within-cluster variation) is minimized. There are several k-means algorithms available. The standard algorithm is the Hartigan-Wong algorithm (1979), which defines the total within-cluster variation as the sum of squared distances Euclidean distances between items and the corresponding centroid:

where:

is a data point belonging to the cluster

is the mean value of the points assigned to the cluster

Each observation () is assigned to a given cluster such that the sum of squares (SS) distance of the observation to their assigned cluster centers () is minimized.

We define the total within-cluster variation as follows:

The total within-cluster sum of square measures the compactness (i.e goodness) of the clustering and we want it to be as small as possible.

K-means Algorithm

The first step when using k-means clustering is to indicate the number of clusters (k) that will be generated in the final solution. The algorithm starts by randomly selecting k objects from the data set to serve as the initial centers for the clusters. The selected objects are also known as cluster means or centroids. Next, each of the remaining objects is assigned to it's closest centroid, where closest is defined using the Euclidean distance (Eq. 1) between the object and the cluster mean. This step is called "cluster assignment step". After the assignment step, the algorithm computes the new mean value of each cluster. The term cluster "centroid update" is used to design this step. Now that the centers have been recalculated, every observation is checked again to see if it might be closer to a different cluster. All the objects are reassigned again using the updated cluster means. The cluster assignment and centroid update steps are iteratively repeated until the cluster assignments stop changing (i.e until convergence is achieved). That is, the clusters formed in the current iteration are the same as those obtained in the previous iteration.

K-means algorithm can be summarized as follows:

1. Specify the number of clusters (K) to be created (by the analyst)

2. Select randomly k objects from the data set as the initial cluster centers or means

3. Assigns each observation to their closest centroid, based on the Euclidean distance between the object and the centroid

4. For each of the k clusters update the cluster centroid by calculating the new mean values of all the data points in the cluster. The centroid of a Kth cluster is a vector of length p containing the means of all variables for the observations in the kth cluster;p is the number of variables.

5. Iteratively minimize the total within sum of square (Eq. 7). That is, iterate steps 3 and 4 until the cluster assignments stop changing or the maximum number of iterations is reached. By default, the R software uses 10 as the default value for the maximum number of iterations.

Computing k-means clustering in R

We can compute k-means in R with the kmeans function. Here will group the data into two clusters ( centers = 2). The kmeans function also has an nstart option that attempts multiple initial configurations and reports on the best one. For example, adding nstart = 25 will generate 25 initial configurations. This approach is often recommended.

The output of kmeans is a list with several bits of information. The most important being:

cluster: A vector of integers (from 1:k) indicating the cluster to which each point is allocated.

centers: A matrix of cluster centers.

totss: The total sum of squares.

withinss: Vector of within-cluster sum of squares, one component per cluster.

tot.withinss: Total within-cluster sum of squares, i.e. sum(withinss).

betweenss: The between-cluster sum of squares, i.e. $totss-tot.withinss$.

size: The number of points in each cluster.

If we print the results we'll see that our groupings resulted in 2 cluster sizes of 30 and 20. We see the cluster centers (means) for the two groups across the four variables ( Murder, Assault, UrbanPop, Rape). We also get the cluster assignment for each observation (i.e. Alabama was assigned to cluster 2, Arkansas was assigned to cluster 1, etc.).

We can also view our results by using fviz_cluster. This provides a nice illustration of the clusters. If there are more than two dimensions (variables) fviz_cluster will perform principal component analysis (PCA) and plot the data points according to the first two principal components that explain the majority of the variance.

Alternatively, you can use standard pairwise scatter plots to illustrate the clusters compared to the original variables.

Because the number of clusters (k) must be set before we start the algorithm, it is often advantageous to use several different values of k and examine the differences in the results. We can execute the same process for 3, 4, and 5 clusters, and the results are shown in the figure:

Although this visual assessment tells us where true dilineations occur (or do not occur such as clusters 2 & 4 in the k = 5 graph) between clusters, it does not tell us what the optimal number of clusters is.

Determining Optimal Clusters

As you may recall the analyst specifies the number of clusters to use; preferably the analyst would like to use the optimal number of clusters. To aid the analyst, the following explains the three most popular methods for determining the optimal clusters, which includes:

Elbow Method

Recall that, the basic idea behind cluster partitioning methods, such as k-means clustering, is to define clusters such that the total intra-cluster variation (known as total within-cluster variation or total within-cluster sum of square) is minimized:

where is the cluster and is the within-cluster variation. The total within-cluster sum of square (wss) measures the compactness of the clustering and we want it to be as small as possible. Thus, we can use the following algorithm to define the optimal clusters:

1. Compute clustering algorithm (e.g., k-means clustering) for different values of k. For instance, by varying k from 1 to 10 clusters

2. For each k, calculate the total within-cluster sum of square (wss)

3. Plot the curve of wss according to the number of clusters k.

4. The location of a bend (knee) in the plot is generally considered as an indicator of the appropriate number of clusters.

We can implement this in R with the following code. The results suggest that 4 is the optimal number of clusters as it appears to be the bend in the knee (or elbow).

Fortunately, this process to compute the "Elbow method" has been wrapped up in a single function ( fviz_nbclust):

Average Silhouette Method

In short, the average silhouette approach measures the quality of a clustering. That is, it determines how well each object lies within its cluster. A high average silhouette width indicates a good clustering. The average silhouette method computes the average silhouette of observations for different values of k. The optimal number of clusters k is the one that maximizes the average silhouette over a range of possible values for k.

We can use the silhouette function in the cluster package to compuate the average silhouette width. The following code computes this approach for 1-15 clusters. The results show that 2 clusters maximize the average silhouette values with 4 clusters coming in as second optimal number of clusters.

Similar to the elbow method, this process to compute the "average silhoutte method" has been wrapped up in a single function ( fviz_nbclust):

Gap Statistic Method

The gap statistic has been published by R. Tibshirani, G. Walther, and T. Hastie (Standford University, 2001). The approach can be applied to any clustering method (i.e. K-means clustering, hierarchical clustering). The gap statistic compares the total intracluster variation for different values of k with their expected values under null reference distribution of the data (i.e. a distribution with no obvious clustering). The reference dataset is generated using Monte Carlo simulations of the sampling process. That is, for each variable () in the data set we compute its range and generate values for the n points uniformly from the interval min to max.

For the observed data and the the reference data, the total intracluster variation is computed using different values of k. The gap statistic for a given k is defined as follow:

Where denotes the expectation under a sample size n from the reference distribution. is defined via bootstrapping (B) by generating B copies of the reference datasets and, by computing the average . The gap statistic measures the deviation of the observed value from its expected value under the null hypothesis. The estimate of the optimal clusters () will be the value that maximizes . This means that the clustering structure is far away from the uniform distribution of points.

In short, the algorithm involves the following steps:

1. Cluster the observed data, varying the number of clusters from , and compute the corresponding .

2. Generate B reference data sets and cluster each of them with varying number of clusters . Compute the estimated gap statistics presented in eq. 9.

3. Let , compute the standard deviation and define .

4. Choose the number of clusters as the smallest k such that .

To compute the gap statistic method we can use the clusGap function which provides the gap statistic and standard error for an output.

We can visualize the results with fviz_gap_stat which suggests four clusters as the optimal number of clusters.

In addition to these commonly used approaches, the NbClust package, published by Charrad et al., 2014, provides 30 indices for determining the relevant number of clusters and proposes to users the best clustering scheme from the different results obtained by varying all combinations of number of clusters, distance measures, and clustering methods.

Extracting Results

With most of these approaches suggesting 4 as the number of optimal clusters, we can perform the final analysis and extract the results using 4 clusters.

We can visualize the results using fviz_cluster:

And we can extract the clusters and add to our initial data to do some descriptive statistics at the cluster level:

K-means clustering is a very simple and fast algorithm. Furthermore, it can efficiently deal with very large data sets. However, there are some weaknesses of the k-means approach.

One potential disadvantage of K-means clustering is that it requires us to pre-specify the number of clusters. Hierarchical clustering is an alternative approach which does not require that we commit to a particular choice of clusters. Hierarchical clustering has an added advantage over K-means clustering in that it results in an attractive tree-based representation of the observations, called a dendrogram. A future tutorial will illustrate the hierarchical clustering approach.

An additional disadvantage of K-means is that it's sensitive to outliers and different results can occur if you change the ordering of your data. The Partitioning Around Medoids (PAM) clustering approach is less sensititive to outliers and provides a robust alternative to k-means to deal with these situations. A future tutorial will illustrate the PAM clustering approach.

For now, you can learn more about clustering methods with:

# data[].attributes.attachment_link.description
Clustering is a broad set of techniques for finding subgroups of observations within a data set. When we cluster observations, we want observations in the same group to be similar and observations in different groups to be dissimilar.

# data[].attributes.body
Bill Gates sits down with the U.K. Prime Minister Rishi Sunak to discuss U.K. innovation, technology and AI.

# data[].attributes.attachment_link.title
Rishi Sunak & Bill Gates answer questions from AI

# data[].attributes.body
Class based poster exercise on explaining k-means

# data[].attributes.body
Class based poster exercises on explaining K-means

# data[].attributes.headline
Colombian judge says he used ChatGPT in ruling

# data[].attributes.subject
Colombian judge says he used ChatGPT in ruling

# data[].attributes.attachment_link.readable_text
A judge in Colombia has caused a stir by admitting he used the artificial intelligence tool ChatGPT when deciding whether an autistic child's insurance should cover all of the costs of his medical treatment. He also used precedent from previous rulings to support his decision.

Juan Manuel Padilla, a judge in the Caribbean city of Cartagena, concluded that the entirety child's medical expenses and transport costs should be paid by his medical plan as his parents could not afford them.

While the judgment itself did not cause much fuss, the inclusion of Padilla's conversations with ChatGPT in the ruling has been more contentious.

Among Padilla's inquiries with the chatbot, the legal documents show Padilla asked ChatGPT the precise legal matter at hand: "Is an autistic minor exonerated from paying fees for their therapies?"

ChatGPT's response corresponded with the judge's final decision: "Yes, this is correct. According to the regulations in Colombia, minors diagnosed with autism are exempt from paying fees for their therapies."

The case has raised a discussion over the use of AI in law and has been criticised by some of Padilla's peers.

ChatGPT scours text across the internet to generate informed responses but has been shown to provide different answers to the same question. It also fabricates information on occasion to make inventive and compelling lies.

The nascent platform has caused alarm in recent weeks, including in schools, where teachers fear OpenAI's platform could be used by students for plagiarism.

Padilla defended his use of the technology, suggesting it could make Colombia's bloated legal system more efficient. The judge also used precedent from previous rulings to support his decision.

Padilla told Blu Radio on Tuesday that ChatGPT and other such programs could be useful to "facilitate the drafting of texts" but "not with the aim of replacing" judges.

Padilla also insisted that "by asking questions to the application, we do not stop being judges, thinking beings."

The judge argued that ChatGPT performs services previously provided by a secretary and did so "in an organised, simple and structured manner" that could "improve response times" in the justice system.

Prof Juan David Gutierrez of Rosario University was among those to express incredulity at the judge's admission.

He called for urgent "digital literacy" training for judges.

Colombia approved a law in 2022 that suggests that public lawyers should use technologies where possible to make their work more efficient.

Octavio Tejeiro, a judge in Colombia's supreme court, said AI caused moral panic in law as people feared robots would replace judges, but he predicted the tool would probably soon become accepted and commonplace.

"The justice system should make the most of technology as a tool but always while following ethics and taking into account that the administrator of justice is ultimately a human being," Tejeiro said. "It must be seen as an instrument that serves the judge to improve his judgment. We cannot allow the tool to become more important than the person."

Tejeiro told the Guardian he had not used ChatGPT but would consider using it in future.

The chatbot itself was more apprehensive about its new role in the justice system.

"Judges should not use ChatGPT when ruling on legal cases ... It is not a substitute for the knowledge, expertise and judgment of a human judge," it responded to a question from the Guardian.

"Journalists should exercise caution when using quotes generated by ChatGPT in their articles," the bot added.

# data[].attributes.attachment_link.title
Colombian judge says he used ChatGPT in ruling

# data[].attributes.attachment_link.description
A judge in Colombia has caused a stir by admitting he used the artificial intelligence tool ChatGPT when deciding whether an autistic child's insurance should cover all of the costs of his medical treatment. He also used precedent from previous rulings to support his decision.

# data[].attributes.headline
Genetic algorithms follow the natural process of evolution, and are described as follows:

# data[].attributes.subject
Genetic algorithms follow the natural process of evolution, and are described as follows:

# data[].attributes.body
3.1. Algorithm

The first step is to initialize the population. In the case of problem-solving, a set of solutions to the problem at hand is the initial population. Secondly, we evaluate the optimality of the population using a fitness function. The fitness function is an objective function  that evaluates the quality of the solutions.

The ultimate goal is to find the maxima or minima of this function . Take, for instance, a fitness function that assigns a score to the solutions in the population. Then, our goal here is to reach a global maximum for the function, .

If the solutions from the population are satisfactory, we usually stop the algorithm and crown these as the best individuals. Now, if the solutions are not satisfactory, we perform the process of selection and pick the fittest individuals to reproduce and generate new solutions. This is done iteratively.

Next, after selection, the fittest individuals reproduce through crossover, to generate offspring. The word offspring here refers to a new generation of solutions. During crossover, values are exchanged to generate new individuals.

Then, certain offspring produced will undergo mutation. This is a process of randomly changing the values or characteristics in the offspring to introduce diversity. Specifically, an iterative application of the mutation operation will allow the algorithm to get out of a local minimum during the search.

It is important to note that crossover and mutation are the two main methods used to generate offspring in genetic algorithms. Lastly, if the solutions or offsprings are satisfactory and there are no better offsprings to produce, the algorithm terminates and presents the best individuals as the optimal solutions.

4. Applications

Genetic algorithms are one of the most fundamental algorithms in computer science. Consequently, they have found many applications in the real world in different industries and for different tasks.

4.1. Robotics

Robotics encompasses the design, construction, and operation of autonomous robots. In robotics, genetic algorithms are used to provide insight into the decisions a robot has to make. For instance, given an environment, suppose a robot has to get to a specific position using the least amount of resources. Genetic algorithms are used to generate optimal routes the robot could use to get to the desired position.

4.2. Economics

Economics is the science of the use of resources in the production, distribution, and overall consumption of goods and services. In economics, genetic algorithms are used to create models of supply and demand over periods of time. Additionally, genetic models are also used to derive game theory and asset pricing, models.

4.3. Automated Design

Automated design constitutes the design and production of automobiles such as cars. In particular, a car manufacturing company may have specifications on how a car should operate. As an example, minimum fuel consumption is a desirable specification in the design of vehicles. As a result, genetic algorithms are used to derive designs of automobiles that satisfy constraints such as low fuel consumption.

4.4. Scheduling Tasks

Suppose, we have a task to schedule the timetable for a university. The goal of a scheduling task is to find an optimal solution that satisfies certain constraints. Genetic algorithms are used in this aspect to derive optimal schedules considering the courses, number of students, and lecture rooms that the university has.

4.5. Vehicle Routing

In most logistics companies, one main task is vehicle routing. Here, we have to figure out how to route and transport goods to different clients using a fleet of vehicles. Consequently, genetic algorithms are used to derive cost-effective routes to transport these goods to the rightful client at the right time.

4.6. Marketing

In marketing, the main idea is to promote goods and services in such a way that it will gain a lot of customers or clients. Genetic algorithms are used here to derive the best combinations of product specifications and attributes that will attract the most customers.

4.7. Medicine

In medicine and bioinformatics, genetic algorithms have been used to identify benign and malignant tumours in ultrasounds and mammograms.

5. Conclusions

In this article, we first defined evolution, then genetic algorithms. We’ve discussed the basic components and how they work. Lastly, we reviewed some real-life applications of these algorithms.

# data[].attributes.attachment_link.readable_text
A team of neuroscientists has managed to find a way to transfer memories from one individual to another via injection, but only in snails so far.

Transhumanists prophesy a future where our memories can be uploaded to the cloud and then transferred into a robotic body to live forever.

The reality is, we know so little about the human mind as it is that it is unlikely we'll find a way to do it any time soon.

But now, a team of researchers from the Society of Neuroscience has revealed in a paper in that it has successfully transferred memories between organisms simply by injecting RNA from one into the other.

The typical consensus is that memories are encoded in connections between our brain cells, but neurobiologist David Glanzman is of the opinion that RNA actually holds the key.

'Very specific kinds of memories'

The breakthrough was achieved with California sea hare snails whereby one was trained to perform a certain action, while another was untrained.

The RNA in the trained snail was used to create an engram - the elusive substrate of memory - by sensitising the snails with tail simulation, which triggers an involuntary defensive reflex.

Once injected into the untrained snails, they too showed a similar defensive response without having ever demonstrated it before.

The trained RNA also increased the excitability of cultured sensory neurons (obtained from untrained animals) that control this reflex. All of this raises the possibility that RNA could be used to modify memory in other organisms, including us.

Speaking of what this means, Glanzman said: "What we are talking about are very specific kinds of memories, not the sort that says what happened to me on my fifth birthday, or who is the president of the US."

'This work takes us down an interesting road'

However, speaking with The Guardian, Tomás Ryan, a researcher of memory at Trinity College Dublin, is not exactly convinced that Glanzman and his team have demonstrated an ability to transfer what we consider a personal memory.

"It's interesting, but I don't think they've transferred a memory," Ryan said.

"This work tells me that maybe the most basic behavioural responses involve some kind of switch in the animal and there is something in the soup that Glanzman extracts that is hitting that switch."

He did, however, say that the findings show optimism for a field where thinking outside the box is not actively encouraged.

"In a field like this, which is so full of dogma, where we are waiting for people to retire so we can move on, we need as many new ideas as possible," he said.

"This work takes us down an interesting road, but I have a huge amount of scepticism about it."

# data[].attributes.attachment_link.title
Amazing breakthrough shows transfer of memories is possible

# data[].attributes.attachment_link.description
A team of neuroscientists has managed to find a way to transfer memories from one individual to another via injection, but only in snails so far. Transhumanists prophesy a future where our memories can be uploaded to the cloud and then transferred into a robotic body to live forever.

# data[].attributes.attachment_link.readable_text
Credit: Unsplash/CC0 Public Domain

Engineers sometimes turn to nature for inspiration. Cold Spring Harbor Laboratory Associate Professor Saket Navlakha and research scientist Jonathan Suen have found that adjustment algorithms-the same feedback control process by which the Internet optimizes data traffic-are used by several natural systems to sense and stabilize behavior, including ant colonies, cells, and neurons.

Internet engineers route data around the world in small packets, which are analogous to ants. As Navlakha explains, "The goal of this work was to bring together ideas from machine learning and Internet design and relate them to the way ant colonies forage."

The same algorithm used by internet engineers is used by ants when they forage for food. At first, the colony may send out a single ant. When the ant returns, it provides information about how much food it got and how long it took to get it. The colony would then send out two ants. If they return with food, the colony may send out three, then four, five, and so on. But if ten ants are sent out and most do not return, then the colony does not decrease the number it sends to nine. Instead, it cuts the number by a large amount, a multiple (say half) of what it sent before: only five ants. In other words, the number of ants slowly adds up when the signals are positive, but is cut dramatically lower when the information is negative. Navlakha and Suen note that the system works even if individual ants get lost and parallels a particular type of "additive-increase/multiplicative-decrease algorithm" used on the internet.

Suen thinks ants might inspire new ways to protect computer systems against hackers or cyberattacks. Engineers could emulate how nature withstands a range of threats to health and viability. Suen explains, "Nature has been shown to be incredibly robust in a lot of aspects responding to changing environments. In cybersecurity [however] we find that a lot of our systems can be tampered with, can be easily broken, and are simply not robust. We wanted to look at nature, which survives across all sorts of natural disasters, evolutionary changes, human changes, and learn a lot from how nature changes its systems dynamically to survive."

While Suen plans to apply nature's algorithms to engineering programs, Navlakha would like to see if engineering solutions might offer alternative approaches to understanding gene regulation and immune feedback control. Navlakha hopes that "successful strategies in one realm could lead to improvements in the other."

More information: A feedback control principle common to several biological and engineered systems, Journal of the Royal Society Interface (2022). DOI: 10.1098/rsif.2021.0711. rsif.royalsocietypublishing.or ... .1098/rsif.2021.0711

# data[].attributes.attachment_link.title
Deciphering behavior algorithms used by ants and the internet

# data[].attributes.attachment_link.description
Engineers sometimes turn to nature for inspiration. Cold Spring Harbor Laboratory Associate Professor Saket Navlakha and research scientist Jonathan Suen have found that adjustment algorithms-the same feedback control process by which the Internet optimizes data traffic-are used by several natural systems to sense and stabilize behavior, including ant colonies, cells, and neurons.

# data[].attributes.headline
Researchers develop an AI-powered surveillance system for future pandemics

# data[].attributes.subject
Researchers develop an AI-powered surveillance system for future pandemics

# data[].attributes.attachment_link.readable_text
Credit: Pixabay/CC0 Public Domain

Nobody wants to think about the next pandemic. But we need to be prepared, and a critical step in prevention is early detection and intervention.

That's why GoodLabs Studio, a company with strong ties to the University of Waterloo, is advancing the power of machine learning and artificial intelligence (AI) to alert health-care authorities with real-time, data-driven insights for decision-making to prevent a future pandemic.

In early 2021, Canada's Department of National Defense (DND) invited proposals for innovations that strengthen the response to future pandemics. GoodLabs, co-founded by Thomas Lo jumped on the opportunity and have since won two successive grants from DND to develop the Syndrome Anomaly Detection System (SADS).

SADS performs widespread disease monitoring to detect patterns of atypical disease across communities so that healthcare and policy leaders can act quickly.

"We've learned from COVID-19 just how fast-moving pandemics are, and therefore how valuable reliable data in real-time is for understanding risk," says Dr. Jean-Paul Lam of the Department of Economics, special advisor and team lead for AI outbreak detection on the project.
How SADS works
It begins with a mobile app in a doctor's office or health care clinic. The SADS app uses natural language processing to anonymously capture symptoms described during the patient-doctor conversation. That data is then aggregated and categorized using deep language machine learning for the purpose of detecting increases in atypical symptoms within the population and evaluating risk of spread.

Of course, it's vital that the data collection and analysis do not compromise any patient's privacy. To maintain confidentiality, the team has deployed natural language processing (NLP) AI technology within the app rather than uploading the conversation data to the cloud. The patient's personal information is protected, and only the pertinent details-symptoms, age, gender, location-are collected and aggregated.

The SADS back-end platform uses machine learning analytics to code the symptoms according to the International Classification of Diseases (ICD-10) and rank how typical or atypical they are. By tracking the atypical symptoms over time, SADS builds a statistical visualization representing how a novel disease might be spreading in a community. The system generates an alert with the key information about a potential outbreak and shares it in real-time with health and government authorities.

When the team ran a simulation of the COVID-19 outbreak in 2020 in several Canadian cities, they found that Toronto, for example, already had a detectable outbreak a full week before the city declared a lockdown. The simulation suggests that if SADS had been available at the time, a more proactive response would have been possible.

With the potential to aggregate health data generated around the world, SADS could be used locally, nationally and globally-certainly, that is the vision.

"We fundamentally believe there is an unbounded opportunity for positive impact," says Lo. "We aim to deploy the Syndrome Anomaly Detection System in hospital triages, clinics, telehealth and eHealth forums-a system that can provide authorized government and health entities early warning of the next pandemic and its spread pattern."

# data[].attributes.attachment_link.title
Researchers develop an AI-powered surveillance system for future pandemics

# data[].attributes.attachment_link.description
Nobody wants to think about the next pandemic. But we need to be prepared, and a critical step in prevention is early detection and intervention. That's why GoodLabs Studio, a company with strong ties to the University of Waterloo, is advancing the power of machine learning and artificial intelligence (AI) to alert health-care authorities with real-time, data-driven insights for decision-making to prevent a future pandemic.

# data[].attributes.attachment_link.readable_text
MIT researchers have found that, if a certain type of machine learning model is trained using an unbalanced dataset, the bias that it learns is impossible to fix after the fact. They developed a technique that induces fairness directly into the model, no matter how unbalanced the training dataset was, which can boost the model's performance on downstream tasks. Credit: Image: Jose-Luis Olivares, MIT

If a machine-learning model is trained using an unbalanced dataset, such as one that contains far more images of people with lighter skin than people with darker skin, there is serious risk the model's predictions will be unfair when it is deployed in the real world.

But this is only one part of the problem. MIT researchers have found that machine-learning models that are popular for image recognition tasks actually encode bias when trained on unbalanced data. This bias within the model is impossible to fix later on, even with state-of-the-art fairness-boosting techniques, and even when retraining the model with a balanced dataset.

So, the researchers came up with a technique to introduce fairness directly into the model's internal representation itself. This enables the model to produce fair outputs even if it is trained on unfair data, which is especially important because there are very few well-balanced datasets for machine learning.

The solution they developed not only leads to models that make more balanced predictions, but also improves their performance on downstream tasks like facial recognition and animal species classification.

"In machine learning, it is common to blame the data for bias in models. But we don't always have balanced data. So, we need to come up with methods that actually fix the problem with imbalanced data," says lead author Natalie Dullerud, a graduate student in the Healthy ML Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT.

Dullerud's co-authors include Kimia Hamidieh, a graduate student in the Healthy ML Group; Karsten Roth, a former visiting researcher who is now a graduate student at the University of Tubingen; Nicolas Papernot, an assistant professor in the University of Toronto's Department of Electrical Engineering and Computer Science; and senior author Marzyeh Ghassemi, an assistant professor and head of the Healthy ML Group. The research will be presented at the International Conference on Learning Representations.
Defining fairness
The machine-learning technique the researchers studied is known as deep metric learning, which is a broad form of representation learning. In deep metric learning, a neural network learns the similarity between objects by mapping similar photos close together and dissimilar photos far apart. During training, this neural network maps images in an "embedding space" where a similarity metric between photos corresponds to the distance between them.

For example, if a deep metric learning model is being used to classify bird species, it will map photos of golden finches together in one part of the embedding space and cardinals together in another part of the embedding space. Once trained, the model can effectively measure the similarity of new images it hasn't seen before. It would learn to cluster images of an unseen bird species close together, but farther from cardinals or golden finches within the embedding space.

The similarity metrics the model learns are very robust, which is why deep metric learning is so often employed for facial recognition, Dullerud says. But she and her colleagues wondered how to determine if a similarity metric is biased.

"We know that data reflect the biases of processes in society. This means we have to shift our focus to designing methods that are better suited to reality," says Ghassemi.

This image shows two distinct PARADE embeddings for bird color. On the left, both example images are mapped to clusters with birds of the same plumage. On the right in the class label embedding, due to de-correlation, the images are separated from the region of space with other birds of the same plumage, but are still well-clustered, indicating that PARADE can find other attributes to distinguish these species clusters. Credit: Massachusetts Institute of Technology

The researchers defined two ways that a similarity metric can be unfair. Using the example of facial recognition, the metric will be unfair if it is more likely to embed individuals with darker-skinned faces closer to each other, even if they are not the same person, than it would if those images were people with lighter-skinned faces. Second, it will be unfair if the features it learns for measuring similarity are better for the majority group than for the minority group.

The researchers ran a number of experiments on models with unfair similarity metrics and were unable to overcome the bias the model had learned in its embedding space.

"This is quite scary because it is a very common practice for companies to release these embedding models and then people finetune them for some downstream classification task. But no matter what you do downstream, you simply can't fix the fairness problems that were induced in the embedding space," Dullerud says.

Even if a user retrains the model on a balanced dataset for the downstream task, which is the best-case scenario for fixing the fairness problem, there are still performance gaps of at least 20 percent, she says.

The only way to solve this problem is to ensure the embedding space is fair to begin with.
Learning separate metrics
The researchers' solution, called Partial Attribute Decorrelation (PARADE), involves training the model to learn a separate similarity metric for a sensitive attribute, like skin tone, and then decorrelating the skin tone similarity metric from the targeted similarity metric. If the model is learning the similarity metrics of different human faces, it will learn to map similar faces close together and dissimilar faces far apart using features other than skin tone.

Any number of sensitive attributes can be decorrelated from the targeted similarity metric in this way. And because the similarity metric for the sensitive attribute is learned in a separate embedding space, it is discarded after training so only the targeted similarity metric remains in the model.

Their method is applicable to many situations because the user can control the amount of decorrelation between similarity metrics. For instance, if the model will be diagnosing breast cancer from mammogram images, a clinician likely wants some information about biological sex to remain in the final embedding space because it is much more likely that women will have breast cancer than men, Dullerud explains.

They tested their method on two tasks, facial recognition and classifying bird species, and found that it reduced performance gaps caused by bias, both in the embedding space and in the downstream task, regardless of the dataset they used.

Moving forward, Dullerud is interested in studying how to force a deep metric learning model to learn good features in the first place.

"How do you properly audit fairness? That is an open question right now. How can you tell that a model is going to be fair, or that it is only going to be fair in certain situations, and what are those situations? Those are questions I am really interested in moving forward," she says.

More information: Is fairness only metric deep? Evaluating and addressing subgroup gaps in DML. openreview.net/pdf?id=js62_xuLDDv

This story is republished courtesy of MIT News (web.mit.edu/newsoffice/), a popular site that covers news about MIT research, innovation and teaching.

# data[].attributes.attachment_link.title
Injecting fairness into machine-learning models

# data[].attributes.attachment_link.description
If a machine-learning model is trained using an unbalanced dataset, such as one that contains far more images of people with lighter skin than people with darker skin, there is serious risk the model's predictions will be unfair when it is deployed in the real world.

# data[].attributes.attachment_link.title
Deep-learning diagnoses: AI detects COVID-19 from smartwatch sensors

# data[].attributes.attachment_link.description
Combining questions about a person’s health with data from smartwatch sensors, a new app developed using research at Princeton University can predict within minutes whether someone is infected with COVID-19.

# data[].attributes.headline
The role of artificial intelligence in tackling COVID-19

# data[].attributes.subject
The role of artificial intelligence in tackling COVID-19

# data[].attributes.attachment_link.readable_text
The past two decades were marked with the outbreaks of many viral diseases such as Chikungunya, Ebola, Zika, Nipah, H7N9 Bird flu, H1N1, SARS and MERS. The world woke up to this decade with a new disease outbreak. An outbreak of a novel Coronavirus emerged in Wuhan city in the Hubei province of China in December 2019. Most of the initially identified patients were traced back to the 'wet market' where live animals are slaughtered and sold. The market might have played a role as an amplification hotspot from where the virus spread to other parts of China and subsequently to 213 countries and territories in a very short time. The WHO named this disease 'COVID-19', which is an acronym of Coronavirus Disease 2019 on 11 February 2020. As of 17 August 2020, a total of 21.2 million confirmed cases and 761,000 deaths have been reported globally [ 1]. The worst outbreaks of COVID-19 are reported in the USA, India, Brazil and Russia where the number of cases has surpassed the confirmed cases in China. The WHO declared the current outbreak of COVID-19 a 'Public Health Emergency of International Concern' on 30 January 2020 and a 'pandemic' on 11 March 2020.

Although the fatality rate of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2; 2.9%) is much less compared with SARS-CoV (9.6%) and MERS-CoV (34.4%), the high infectivity rate of SARS-CoV-2 compared with other coronaviruses has become a global concern. Mortality and vulnerability to COVID-19 were found to be higher in males compared with females, which could be attributed to other gendered practices such as smoking [ 2]. The fatality rate of COVID-19 varied with an age gradient and it was also influenced by underlying co-morbidity, in other words, conditions such as diabetes, hypertension, cancer, cardiovascular diseases and chronic respiratory disease [ 3-5]. Vertical transmission of COVID-19 infection from mother to baby was not observed [ 6]. Children are vulnerable to COVID-19 but tend to show only mild symptoms [ 7].

SARS-CoV-2

The etiological agent was named as SARS-CoV-2 by the International Committee on Virus Taxonomy on 11 February 2020. SARS-CoV-2 is a beta coronavirus of zoonotic origin belonging to the subgenus Sarbecovirus in the Orthocoronavirinae subfamily of the family Coronaviridae transmitted to humans in a spillover event. Bats are thought to be the animal reservoir of SARS-CoV-2 but the other likely intermediate animal host is yet to be identified. The virus is a spherical particle of 70-90 nm [ 8], having spikes of glycoprotein projecting from its surface that bind to receptor angiotensin-converting enzyme 2 on the surface of the cell. These spikes give the virus a crown-like appearance.

The glycoprotein of SARS-CoV-2 has a furin polybasic cleavage site (PRRARS|V) located between the residues 682 and 685 at the boundary of two subunits S1/S2 that is catalyzed during biogenesis [ 9]. The presence of this cleavage site in SARS-CoV-2 that is observed in avian influenza viruses but not related viruses like SARS-CoV and SARSr-CoVs makes it distinct and has an impact on entry, tropism, spread and pathogenicity of the virus [ 9, 10]. Expression of furin proteases in the respiratory tract, brain, pancreas, liver, gastrointestinal tract and reproductive organs of the host enables the virus to infect different organs and also facilitates its release into the surrounding environment in many ways. At present, 249 protein structures and 255 whole-genome sequences belonging to SARS-CoV-2 are available in the public domain.

SARS-CoV-2 genome

A recent study suggested a single-source origin of SARS-CoV-2, as genomic sequences collected from different patients showed strikingly high identity and also indicated that SARS-CoV-2 is phylogenetically closer to bat-SL-CoVZC45 and bat-SL-CoVZXC21 [ 11]. Its genome size is approximately 30 kb [ 12]. A vast portion of the genome is occupied by two open-reading frames (ORF1a and ORF1b) that translate into pp1a and pp1ab polyproteins, which are then cleaved to 16 nonstructural proteins (nsp) like cysteine proteases, chymotrypsin-like, RNA-dependent RNA polymerase, helicase and so on. The rest of the genome encodes structural proteins like the spike(S), envelope (E), membrane (M) and nucleocapsid protein and 6-7 accessory proteins [ 13]. Genetic analysis revealed that SARS-CoV-2 has evolved in two lineages: ancestral S type and other more prevalent, aggressive and virulent L type derived from S type [ 14]. It is interesting to note that in the early stages of the epidemic, L type was more frequent, but its frequency decreased later and the frequency of S type increased, which can be attributed to differential selection pressure and epidemiological features [ 14].

Transmission

COVID-19 mainly spreads from human to human through direct contact by respiratory droplets during coughing or sneezing and through indirect contact route by fomites and regularly touched surfaces [ 15]. SARS-CoV-2 can remain viable on various surfaces for several hours to days [ 16]. Air-borne transmission is possible in a medical or hospital setting in processes that generate aerosols. Although fecal-oral transmission of COVID-19 has not been reported to date, it remains a potential route [ 17, 18].

Clinical symptoms

Most patients experience mild flu-like symptoms including fever, cough, malaise, fatigue, sputum production and respiratory problems. Less common symptoms such as headache, hemoptysis and gastrointestinal symptoms with diarrhea and serious symptoms like pneumonia and bronchitis were also observed. Complications like Acute Respiratory Distress Syndrome, RNAaemia, acute cardiac injury, acute kidney injury and secondary infections [ 19] were reported in some patients. Other lab parameters associated with COVID-19 were low white blood cells and lymphocyte count, an increase in erythrocyte sedimentation rate, C-reactive protein, infiltrates and bilateral ground-glass opacity in lung CT scans.

Prevention & control

It is imperative to adopt control measures such as case isolation, contact tracing, quarantine to limit human-to-human COVID-19 transmission. Personal hygiene measures such as frequent hand washing, respiratory hygiene, social distancing, use of face masks/shields and disinfection of surfaces can help in reducing the transmission.

Screening & diagnosis

Discriminant clinical features like hyposmia (loss of smell) and hypogeusia (loss of taste) can be explored for preliminary diagnosis in telemedicine and mass screening [ 20]. Specimen samples collected from oropharyngeal and nasopharyngeal swabs or blood samples are used for diagnosis. Although routinely used for COVID-19 diagnosis in outbreak settings, sole reliance on CT scans can be misleading due to indistinguishable images with other viral pneumonia. Molecular test reverse transcriptase-PCR (RT-PCR) is recommended by WHO as the method of choice for detecting the SARS-CoV-2 nucleic acid for diagnosis of COVID-19. As the false-negative rate of RT-PCR is high, it is imperative to use CT scan of the chest as a supplementary diagnostic measure to confirm the diagnosis. Point-of-care immunodiagnostic assays that detect proteins from the COVID-19 virus or human antibodies generated against the virus in blood samples are also being used routinely to complement molecular tests due to low cost and fast results, but these methods suffer from poor sensitivity and are only qualitative [ 1]. Utility of these serological methods in public health settings for contact tracing and evaluating the success of nonpharmaceutical interventions has been discussed elsewhere [ 21]. These serological methods have now received Emergency Use Authorization by the US FDA. CRISPR-Cas12-based assay that provides rapid results can be used in point-of-care testing in the future [ 22].

According to recent data from WHO, 13 candidate vaccines are being evaluated. An experimental vaccine developed by the University of Oxford/AstraZeneca has entered Phase III of clinical trials while vaccine candidates from CanSino Biological Inc./Beijing Institute of Biotechnology and Moderna/NIAID have reached Phase II trials and ten vaccine candidates have reached Phase I/II and Phase I stages. 129 other candidate vaccines are in the preclinical stage (WHO) and many are in pipeline [ 23, 24].

Therapeutic agents

Some of the potential drugs against COVID-19 beingconsidered and evaluated are remdesivir (GS-5734), baricitinib, a combination drug ritonavir/lopinavir, Ribavirin ®, umifenovir and IFN-β and other broad spectrum antiviral agents. Remdesivir was not found to be effective in treating COVID-19 patients in a placebo-controlled randomized trial of remdesivir [ 25]. In a recent development, the FDA has approved the use of remdesivir in confirmed and suspected cases of COVID-19. As of 25 June 2020, about 1235 clinical trials for various therapeutic agents against COVID-19 are being conducted across the globe [ 26].

Application of artificial intelligence in COVID-19 disease management

Unprecedented pace of efforts to address the COVID-19 pandemic situation is leveraged by big data and artificial intelligence (AI). Various offshoots of AI have been used in several disease outbreaks earlier. AI can play a vital role in the fight against COVID-19.

AI is being successfully used in the identification of disease clusters, monitoring of cases, prediction of the future outbreaks, mortality risk, diagnosis of COVID-19, disease management by resource allocation, facilitating training, record maintenance and pattern recognition for studying the disease trend. Several applications of AI that are garnering a lot of interest and raising hopes in the fight against COVID-19 are as follows:

AI in prediction & tracking

AI can be harnessed for forecasting the spread of virus and developing early warning systems by extracting information from social media platforms, calls and news sites and provide useful information about the vulnerable regions and for prediction of morbidity and mortality. Bluedot identified a cluster of pneumonia cases and predicted the outbreak and geographical location of the COVID-19 outbreak based on available data using machine learning. HealthMap collects the publicly available data on COVID-19 and makes it readily available to facilitate the effective tracking of its spread. Recently, the role of AI in identification and forecasting of COVID-19 outbreaks by employing multitudinal and multimodal data was emphasized [ 27].

AI in contact tracing

AI can augment mobile heath applications where smart devices like watches, mobile phones, cameras and range of wearable device can be employed for diagnosis, contact tracing and efficient monitoring in COVID-19 [ 28]. Applications like AI4COVID-19 that rely on audio recording samples of 2 s cough can be used in telemedicine [ 29].

AI in monitoring of COVID-19 cases

AI techniques are applied for monitoring patients in clinical settings and prediction of course of treatment. Based on the data derived from vital statistics and clinical parameters, AI may provide critical information for resource allocation and decision-making by prioritizing the need of ventilators and respiratory supports in the Intensive Care Unit [ 30]. AI can also be used for predicting the chances of recovery or mortality in COVID-19 and to provide daily updates, storage and trend analysis and charting the course of treatment.

AI in early diagnosis

AI was used for the detection and quantification of COVID-19 cases from chest x-ray and CT scan images [ 31-33]. Researchers have developed a deep learning model called COVID-19 detection neural network (COVNet), for differentiating between COVID-19 and community-acquired pneumonia based on visual 2D and 3D features extracted from volumetric chest CT scan [ 34]. Singh et al. developed a novel deep learning model using Multi-Objective Differential Evolution and convolutional neural networks for COVID-19 diagnosis using a chest CT scan [ 35]. COVID-ResNet developed using automatic and discriminative learning rate and progressive image resizing performed better than COVID-Net in diagnosing COVID-19 [ 36]. Alom et al. developed a system called COVID_MTNet by applying improved Inception Recurrent Residual Neural Network and NABLA-3 network models for detection and localization of regions of interests from both x-ray images and chest CT scan [ 37]. Another study used AI-based classifiers for predicting the outcome of RT-PCR results of COVID-19 cases using 16 simple parameters derived from complete blood profile [ 38]. This may find application in reducing the number of RT-PCR tests in resource-poor settings.

AI in reducing the burden from medical practitioners & healthcare staff

AI-based triage systems can help in reducing the work burden of medical staff and healthcare workers by automating several processes such as imparting training to practitioners, determination of the mode of treatment and care by analyzing clinical data using pattern recognition approaches, digitalization of patient's reports and also by offering solutions that minimize their contact with the patients [ 39-41]. AI can be used for classification of patients based on the severity of symptoms, genetic disposition and clinical reports in different categories like mild, moderate and severe, so that different approaches can be adopted for handling the patients in the most effective manner. AI in telemedicine can also be used to eliminate the need of frequent and unnecessary hospital visits by distant monitoring of cases and recording of patient's data in asymptomatic cases or patients with mild symptoms. AI-based medical chatbots can also be used for consultations, thereby reducing the physical crowding of hospitals as well as the spread of infection and thus prevent weighing down of efficient operation of critical care services [ 42, 43]. Chatbots like Clara from the Centre for Disease Control and Zini are providing much needed support to patients in remote settings [ 44]. A prognostic prediction algorithm predicted the mortality risk of patients by machine learning methods using extracted features derived from the data of other patients as training dataset [ 45]. A similar approach was used to predict the possibility of developing acute respiratory distress syndrome [ 46]. Service robots and anthropomorphic robots with AI core can be used for the delivery of essential services and routine tasks like cleaning, disinfecting and monitoring in hospital settings [ 47, 48].

AI in protein structure prediction

AI can help in predicting the structure of important proteins crucial for virus entry and replication and provide useful insight that can pave way for drug development in a very short time. AlphaFold algorithm of Google Deep mind employed deep residual networks (DRN) called ResNets for predicting protein structures of membrane protein, protein 3a, nsp2, nsp4, nsp6 and papain-like C-terminal domain of SARS-CoV-2, which will give huge impetus to drug discovery programs [ 49]. DeepTracer, a program based on customized deep convolutional neural network, was used to derive protein complex structure of SARS-CoV-2 from high-resolution cryoelectron microscopy density maps and amino acid sequence [ 50].

AI in development of therapeutics

AI techniques can boost and complement traditional technologies by reducing the time required in bringing a drug from bench to bed by speeding up lead discovery, virtual screening and validation processes by a huge margin. AI can also accelerate the pace by deriving useful data for drug repurposing or drug repositioning by screening properties of already approved and validated drugs based on molecular descriptors and properties, which may not be possible for a human expert. BenevolentAI used machine learning methods to accelerate its drug discovery program and identified baricitinib as a potential drug against COVID-19 [ 51, 52]. Insilico Medicine has identified several small molecules against COVID-19 using AI [ 53]. Another study combined virtual screening and supervised learning to identify potential drugs against COVID-19 [ 54]. Zhou et al. adopted an integrative network-based systems pharmacological methodology for finding potential drugs for SARS-CoV-2 from the already existing repertoire of drug molecules and drug combinations [ 53]. Several other AI-based endeavors including inclProject IDentif.AI (identifying infectious disease combination therapy with artificial intelligence) [ 55] and PolypharmDB [ 56] have been successful in identifying candidates against COVID-19. Many machine learning approaches and deep learning-based applications are also being used for expediting the drug discovery process [ 57-60].

AI in development of vaccines

Never before has mankind witnessed such a race for the development of a vaccine against a pathogen. The pace of the discovery can be accelerated manifold by harnessing the power of AI. Ong et al. predicted possible vaccine candidates for COVID-19 using the Vaxign reverse vaccinology-machine learning platform that relied on supervised classification models [ 61].

AI in curbing spread of misinformation

Due to the avalanche of information, this pandemic has turned into an infodemic. Understanding knowledge, awareness and practices toward COVID-19 by tapping information from social media platforms like Twitter, Facebook etc. can help in devising the strategy to assemble and disseminate timely and correct information for mitigating the impact of COVID-19 [ 62, 63]. Machine learning techniques can be used to identify trends and sentiment analysis and provide information regarding the origin of false information and help in curtailing the rumors and misinformation [ 64]. AI techniques can further be used for presenting a clear picture of recovery rates, accessibility and availability to healthcare and identification of the gaps. AI can provide the latest updates about the emerging evidence in diagnosis, treatment, spectrum of symptoms and therapeutic outcomes in this highly dynamic situation, which will help clinicians in real-world scenario and help public in overcoming fear and panic [ 65].

AI in genomics

Randhawa et al. devised a method for fast and accurate classification of available SARS-CoV-2 genomes by applying machine learning on identified genomic signatures [ 51]. Wang et al. used ontology-based side effect prediction framework and Artificial Neural Network to evaluate the side effects of Traditional Chinese Medicines for the treatment of SARS-CoV-2 [ 66].

Conclusion & future perspective

Adopting a three-pronged approach based on testing, isolation and contact tracing is warranted to combat COVID-19. It is necessary to exploit the available knowledge base to develop effective chemotherapeutic agents against COVID-19, taking cues from lessons learnt in the past during other such outbreaks.

As there is no silver bullet available to cure the disease, we need to hasten progress on all fronts ranging from surveillance and monitoring to prevention and treatment. As this is the third outbreak of a coronavirus in recent times and many coronaviruses are circulating in animal reservoirs, we must focus on deciphering the molecular mechanism of SARS-CoV-2 and other coronaviruses and increasing our preparedness by capacity building for preventing future outbreaks [ 67]. As the current scenario warrants the need for immediate delivery of solutions, response to this outbreak was hugely augmented by various digital technologies and AI [ 68]. AI was found to be on par with and even more accurate than human experts in COVID-19 diagnosis and drug discovery. We need bigger datasets for training AI models and a legal framework and ethical considerations for sharing data before AI takes the forefront in diagnosis and other areas. Several bottlenecks in harnessing AI to its full potential in the current scenario are availability and sharing of clinical and epidemiological data, computational resources, scalability, privacy and ethical concerns.

Financial & competing interests disclosure

N Arora thanks University Grants Commission for financial support. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

References

# data[].attributes.attachment_link.title
The role of artificial intelligence in tackling COVID-19

# data[].attributes.attachment_link.description
The past two decades were marked with the outbreaks of many viral diseases such as Chikungunya, Ebola, Zika, Nipah, H7N9 Bird flu, H1N1, SARS and MERS. The world woke up to this decade with a new disease outbreak. An outbreak of a novel Coronavirus emerged in Wuhan city in the Hubei province of China in December 2019.

# data[].attributes.headline
Can machine-learning models overcome biased datasets?

# data[].attributes.subject
Can machine-learning models overcome biased datasets?

# data[].attributes.attachment_link.readable_text
Credit: CC0 Public Domain

Artificial intelligence systems may be able to complete tasks quickly, but that doesn't mean they always do so fairly. If the datasets used to train machine-learning models contain biased data, it is likely the system could exhibit that same bias when it makes decisions in practice.

For instance, if a dataset contains mostly images of white men, then a facial-recognition model trained with this data may be less accurate for women or people with different skin tones.

A group of researchers at MIT, in collaboration with researchers at Harvard University and Fujitsu, Ltd., sought to understand when and how a machine-learning model is capable of overcoming this kind of dataset bias. They used an approach from neuroscience to study how training data affects whether an artificial neural network can learn to recognize objects it has not seen before. A neural network is a machine-learning model that mimics the human brain in the way it contains layers of interconnected nodes, or " neurons," that process data.

The new results show that diversity in training data has a major influence on whether a neural network is able to overcome bias, but at the same time dataset diversity can degrade the network's performance. They also show that how a neural network is trained, and the specific types of neurons that emerge during the training process, can play a major role in whether it is able to overcome a biased dataset.

"A neural network can overcome dataset bias, which is encouraging. But the main takeaway here is that we need to take into account data diversity. We need to stop thinking that if you just collect a ton of raw data, that is going to get you somewhere. We need to be very careful about how we design datasets in the first place," says Xavier Boix, a research scientist in the Department of Brain and Cognitive Sciences (BCS) and the Center for Brains, Minds, and Machines (CBMM), and senior author of the paper.

Co-authors include former graduate students Spandan Madan, a corresponding author who is currently pursuing a Ph.D. at Harvard, Timothy Henry, Jamell Dozier, Helen Ho, and Nishchal Bhandari; Tomotake Sasaki, a former visiting scientist now a researcher at Fujitsu; Frédo Durand, a professor of electrical engineering and computer science and a member of the Computer Science and Artificial Intelligence Laboratory; and Hanspeter Pfister, the An Wang Professor of Computer Science at the Harvard School of Enginering and Applied Sciences. The research appears today in Nature Machine Intelligence.
Thinking like a neuroscientist
Boix and his colleagues approached the problem of dataset bias by thinking like neuroscientists. In neuroscience, Boix explains, it is common to use controlled datasets in experiments, meaning a dataset in which the researchers know as much as possible about the information it contains.

The team built datasets that contained images of different objects in varied poses, and carefully controlled the combinations so some datasets had more diversity than others. In this case, a dataset had less diversity if it contains more images that show objects from only one viewpoint. A more diverse dataset had more images showing objects from multiple viewpoints. Each dataset contained the same number of images.

The researchers used these carefully constructed datasets to train a neural network for image classification, and then studied how well it was able to identify objects from viewpoints the network did not see during training (known as an out-of-distribution combination).

For example, if researchers are training a model to classify cars in images, they want the model to learn what different cars look like. But if every Ford Thunderbird in the training dataset is shown from the front, when the trained model is given an image of a Ford Thunderbird shot from the side, it may misclassify it, even if it was trained on millions of car photos.

The researchers found that if the dataset is more diverse-if more images show objects from different viewpoints-the network is better able to generalize to new images or viewpoints. Data diversity is key to overcoming bias, Boix says.

"But it is not like more data diversity is always better; there is a tension here. When the neural network gets better at recognizing new things it hasn't seen, then it will become harder for it to recognize things it has already seen," he says.
Testing training methods
The researchers also studied methods for training the neural network.

In machine learning, it is common to train a network to perform multiple tasks at the same time. The idea is that if a relationship exists between the tasks, the network will learn to perform each one better if it learns them together.

But the researchers found the opposite to be true-a model trained separately for each task was able to overcome bias far better than a model trained for both tasks together.

"The results were really striking. In fact, the first time we did this experiment, we thought it was a bug. It took us several weeks to realize it was a real result because it was so unexpected," he says.

They dove deeper inside the neural networks to understand why this occurs.

They found that neuron specialization seems to play a major role. When the neural network is trained to recognize objects in images, it appears that two types of neurons emerge-one that specializes in recognizing the object category and another that specializes in recognizing the viewpoint.

When the network is trained to perform tasks separately, those specialized neurons are more prominent, Boix explains. But if a network is trained to do both tasks simultaneously, some neurons become diluted and don't specialize for one task. These unspecialized neurons are more likely to get confused, he says.

"But the next question now is, how did these neurons get there? You train the neural network and they emerge from the learning process. No one told the network to include these types of neurons in its architecture. That is the fascinating thing," he says.

That is one area the researchers hope to explore with future work. They want to see if they can force a neural network to develop neurons with this specialization. They also want to apply their approach to more complex tasks, such as objects with complicated textures or varied illuminations.

Boix is encouraged that a neural network can learn to overcome bias, and he is hopeful their work can inspire others to be more thoughtful about the datasets they are using in AI applications.

More information: Spandan Madan, When and how convolutional neural networks generalize to out-of-distribution category-viewpoint combinations, Nature Machine Intelligence (2022). DOI: 10.1038/s42256-021-00437-5. www.nature.com/articles/s42256-021-00437-5

# data[].attributes.attachment_link.title
Can machine-learning models overcome biased datasets?

# data[].attributes.attachment_link.description
Artificial intelligence systems may be able to complete tasks quickly, but that doesn't mean they always do so fairly. If the datasets used to train machine-learning models contain biased data, it is likely the system could exhibit that same bias when it makes decisions in practice.

# data[].attributes.body
Some interesting bits in this and don't let the maths scare you.

# data[].attributes.attachment_link.readable_text
In this Neural Network tutorial we will take a step forward and will discuss about the network of Perceptrons called Multi-Layer Perceptron ( Artificial Neural Network).

We will be discussing the following topics in this Neural Network tutorial:

Limitations of Single-Layer Perceptron

What is Multi-Layer Perceptron (Artificial Neural Network)?

How Artificial Neural Networks Work?

Use-case

This blog on Neural Network tutorial will include a use-case in the end. For implementing that use-case, we will be using TensorFlow.

Now, I will start by discussing what are the limitations of Single-Layer Perceptron.

Limitations of Single-Layer Perceptron:

Well, there are two major problems:

Single-Layer Percpetrons cannot classify non-linearly separable data points.

Complex problems, that involve a lot of parameters cannot be solved by Single-Layer Perceptrons.

Single-Layer Percpetrons cannot classify non-linearly separable data points

Let us understand this by taking an example of XOR gate. Consider the diagram below:

Here, you cannot separate the high and low points with a single straight line. But, we can separate it by two straight lines. Consider the diagram below:

Complex problems, that involve a lot of parameters cannot be solved by Single-Layer Perceptrons:

Here also, I will explain with an example.

As an E-commerce firm, you have noticed a decline in your sales. Now, you try to form a marketing team who would market the products for increasing the sales.

The marketing team can market your product through various ways, such as:

Google Ads

Personal emails

Sale advertisement on relevant sites

Reference program

Blogs and so on . . .

Considering all the factors and options available, marketing team has to decide a strategy to do optimal and efficient marketing, but this task is too complex for a human to analyse, because number of parameters are quite high. This problem will have to be solved using Deep Learning. Consider the diagram below:

They can either use just one means to market their products or use a variety of them.

Each way would have different advantages and disadvantages as well, they will have to focus on a variety of factors and options such as:

Number of sales that would happen would be dependent on different categorical inputs, their sub categories and their parameters. However, computing and calculating from so many inputs and their sub parameters is not possible just through one neuron (Perceptron).

That is why more than one neuron would be used to solve this problem. Consider the diagram below:

Because of all these reasons, Single-Layer Perceptron cannot be used for complex non-linear problems.

Next up, in this Neural Network tutorial I will focus on Multi-Layer Perceptrons (MLP).

What is Multi-Layer Perceptron?

As you know our brain is made up of millions of neurons, so a Neural Network is really just a composition of Perceptrons, connected in different ways and operating on different activation functions.

Consider the diagram below:

Input Nodes - The Input nodes provide information from the outside world to the network and are together referred to as the "Input Layer". No computation is performed in any of the Input nodes - they just pass on the information to the hidden nodes.

Hidden Nodes - The Hidden nodes have no direct connection with the outside world (hence the name "hidden"). They perform computations and transfer information from the input nodes to the output nodes. A collection of hidden nodes forms a "Hidden Layer". While a network will only have a single input layer and a single output layer, it can have zero or multiple Hidden Layers. A Multi-Layer Perceptron has one or more hidden layers.

Output Nodes - The Output nodes are collectively referred to as the "Output Layer" and are responsible for computations and transferring information from the network to the outside world.

Yeah, you guessed it right, I will take an example to explain - how an Artificial Neural Network works.

Suppose we have data of a football team, Chelsea. The data contains three columns. The last column tells whether Chelsea won the match or they lost it. The other two columns are about, goal lead in the first half and possession in the second half. Possession is the amount of time for which the team has the ball in percentage. So, if I say that a team has 50% possession in one half (45 minutes), it means that, the team had ball for 22.5 minutes out of 45 minutes.

The Final Result column, can have two values 1 or 0 indicating whether Chelsea won the match or not. For example, we can see that if there is a 0 goal lead in the first half and in next half Chelsea has 80% possession, then Chelsea wins the match.

Now, suppose, we want to predict whether Chelsea will win the match or not, if the goal lead in the first half is 2 and the possession in the second half is 32%.

This is a binary classification problem where a multi layer Perceptron can learn from the given examples (training data) and make an informed prediction given a new data point. We will see below how a multi layer perceptron learns such relationships.

The process by which a Multi Layer Perceptron learns is called the Backpropagation algorithm, I would recommend you to go throughthe Backpropagation blog.

Consider the diagram below:

Forward Propagation:

Here, we will propagate forward, i.e. calculate the weighted sum of the inputs and add bias. In the output layer we will use the softmax function to get the probabilities of Chelsea winning or loosing.

If you notice the diagram, winning probability is 0.4 and loosing probability is 0.6. But, according to our data, we know that when goal lead in the first half is 1 and possession in the second half is 42% Chelsea will win. Our network has made wrong prediction.

If we see the error (Comparing the network output with target), it is 0.6 and -0.6.

Backward Propagation and Weight Updation:

I would recommend you to refer the Backpropagation blog.

We calculate the total error at the output nodes and propagate these errors back through the network using Backpropagation to calculate the gradients. Then we use an optimization method such as Gradient Descent to 'adjust' all weights in the network with an aim of reducing the error at the output layer.

Let me explain you how the gradient descent optimizer works:

Step - 1: First we calculate the error, consider the equation below:

Step - 2: Based on the error we got, it will calculate the rate of change of error w.r.t change in the weights.

Step - 3: Now, based on this change in weight, we will calculate the new weight value.

If we now input the same example to the network again, the network should perform better than before since the weights have now have been adjusted to minimize the error in prediction. Consider the example below, As shown in Figure, the errors at the output nodes now reduce to [0.2, -0.2] as compared to [0.6, -0.4] earlier. This means that our network has learnt to correctly classify our first training example.

We repeat this process with all other training examples in our dataset. Then, our network is said to have learnt those examples.

Now, I can feed in the input to our network. If I feed in goal lead in the first half as 2 and possession in the second half as 32%, our network will predict whether Chelsea will win that match or not.

Now in this Neural Network Tutorial, we will some have fun with hands-on. I will be using TensorFlow to model a Multi-Layer Neural Network.

Use-Case:

Let's look at our problem statement:

Now, let's look at the dataset, which we will be using to train our network.

The first four columns are features and the last column is label.

Data were extracted from images that were taken from genuine and forged banknote-like specimens. The final images have 400×400 pixels. Due to the object lens and distance to the investigated object gray-scale, pictures with a resolution of about 660 dpi were gained. Wavelet Transform tool were used to extract features from images.

To implement this use-case, we will be using the below flow diagram:

Let's execute it now:

import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split

# Reading the dataset
def read_dataset():
df = pd.read_csv("C:UsersSaurabhPycharmProjectsNeural Network Tutorialbanknote.csv")
# print(len(df.columns))
X = df[df.columns[0:4]].values
y = df[df.columns[4]]

# Encode the dependent variable
Y = one_hot_encode(y)
print(X.shape)
return (X, Y)


# Define the encoder function.
def one_hot_encode(labels):
n_labels = len(labels)
n_unique_labels = len(np.unique(labels))
one_hot_encode = np.zeros((n_labels, n_unique_labels))
one_hot_encode[np.arange(n_labels), labels] = 1
return one_hot_encode


# Read the dataset
X, Y = read_dataset()

# Shuffle the dataset to mix up the rows.
X, Y = shuffle(X, Y, random_state=1)

# Convert the dataset into train and test part
train_x, test_x, train_y, test_y = train_test_split(X, Y, test_size=0.20, random_state=415)

# Inpect the shape of the training and testing.
print(train_x.shape)
print(train_y.shape)
print(test_x.shape)

# Define the important parameters and variable to work with the tensors
learning_rate = 0.3
training_epochs = 100
cost_history = np.empty(shape=[1], dtype=float)
n_dim = X.shape[1]
print("n_dim", n_dim)
n_class = 2
model_path = "C:UsersSaurabhPycharmProjectsNeural Network TutorialBankNotes"

# Define the number of hidden layers and number of neurons for each layer
n_hidden_1 = 4
n_hidden_2 = 4
n_hidden_3 = 4
n_hidden_4 = 4

x = tf.placeholder(tf.float32, [None, n_dim])
W = tf.Variable(tf.zeros([n_dim, n_class]))
b = tf.Variable(tf.zeros([n_class]))
y_ = tf.placeholder(tf.float32, [None, n_class])


# Define the model
def multilayer_perceptron(x, weights, biases):

# Hidden layer with RELU activationsd
layer_1 = tf.add(tf.matmul(x, weights['h1']), biases['b1'])
layer_1 = tf.nn.relu(layer_1)

# Hidden layer with sigmoid activation
layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
layer_2 = tf.nn.relu(layer_2)

# Hidden layer with sigmoid activation
layer_3 = tf.add(tf.matmul(layer_2, weights['h3']), biases['b3'])
layer_3 = tf.nn.relu(layer_3)

# Hidden layer with RELU activation
layer_4 = tf.add(tf.matmul(layer_3, weights['h4']), biases['b4'])
layer_4 = tf.nn.sigmoid(layer_4)

# Output layer with linear activation
out_layer = tf.matmul(layer_4, weights['out']) + biases['out']
return out_layer


# Define the weights and the biases for each layer

weights = {
'h1': tf.Variable(tf.truncated_normal([n_dim, n_hidden_1])),
'h2': tf.Variable(tf.truncated_normal([n_hidden_1, n_hidden_2])),
'h3': tf.Variable(tf.truncated_normal([n_hidden_2, n_hidden_3])),
'h4': tf.Variable(tf.truncated_normal([n_hidden_3, n_hidden_4])),
'out': tf.Variable(tf.truncated_normal([n_hidden_4, n_class]))
}
biases = {
'b1': tf.Variable(tf.truncated_normal([n_hidden_1])),
'b2': tf.Variable(tf.truncated_normal([n_hidden_2])),
'b3': tf.Variable(tf.truncated_normal([n_hidden_3])),
'b4': tf.Variable(tf.truncated_normal([n_hidden_4])),
'out': tf.Variable(tf.truncated_normal([n_class]))
}

# Initialize all the variables

init = tf.global_variables_initializer()

saver = tf.train.Saver()

# Call your model defined
y = multilayer_perceptron(x, weights, biases)

# Define the cost function and optimizer
cost_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=y, labels=y_))
training_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost_function)

sess = tf.Session()
sess.run(init)

# Calculate the cost and the accuracy for each epoch

mse_history = []
accuracy_history = []

for epoch in range(training_epochs):
sess.run(training_step, feed_dict={x: train_x, y_: train_y})
cost = sess.run(cost_function, feed_dict={x: train_x, y_: train_y})
cost_history = np.append(cost_history, cost)
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
# print("Accuracy: ", (sess.run(accuracy, feed_dict={x: test_x, y_: test_y})))
pred_y = sess.run(y, feed_dict={x: test_x})
mse = tf.reduce_mean(tf.square(pred_y - test_y))
mse_ = sess.run(mse)
mse_history.append(mse_)
accuracy = (sess.run(accuracy, feed_dict={x: train_x, y_: train_y}))
accuracy_history.append(accuracy)

print('epoch : ', epoch, ' - ', 'cost: ', cost, " - MSE: ", mse_, "- Train Accuracy: ", accuracy)

save_path = saver.save(sess, model_path)
print("Model saved in file: %s" % save_path)

#Plot Accuracy Graph
plt.plot(accuracy_history)
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.show()

# Print the final accuracy

correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print("Test Accuracy: ", (sess.run(accuracy, feed_dict={x: test_x, y_: test_y})))

# Print the final mean square error

pred_y = sess.run(y, feed_dict={x: test_x})
mse = tf.reduce_mean(tf.square(pred_y - test_y))
print("MSE: %.4f" % sess.run(mse))

Once you run this code you will get the below output:

You can notice the final accuracy is 99.6364% and mean squared error is 2.2198. We can actually make it better by increasing the number of epochs.

Below is the graph of epoch vs accuracy:

You can see that, after every iteration, the accuracy is increasing.

I hope you have enjoyed reading this Neural Network Tutorial.
This article has been published from the source link without modifications to the text. Only the headline has been changed. Source link

# data[].attributes.attachment_link.title
Tutorial on Multi Layer Perceptron in Neural Network - BLOCKGENI

# data[].attributes.attachment_link.description
In this Neural Network tutorial we will take a step forward and will discuss about the network of Perceptrons called Multi-Layer Perceptron ( Artificial Neural Network). We will be discussing the following topics in this Neural Network tutorial: Limitations of Single-Layer Perceptron What is Multi-Layer Perceptron (Artificial Neural Network)?

# data[].attributes.attachment_link.readable_text
A multi-layer perceptron (one input, one output and multiple hidden layers), forward feed neural network with backward propagation and x number of neurons for each layer.

Characteristics

1 Input, [1-..] Hidden, 1 Output layer

Unlimited neurons per layer

Forward feeding, backward propagation

2 Methods, HyperTan / Sigmoid

Set number of hidden layer and neurons or allow automatic adjustment to find the optimal solution

Samples are divided in Train data, Verification data and Test data

Save / Load model from disk

Custom Modeler with instuctions (SimpleNeuralNetwork.ProblemModeler/Problems/Custom.cs)

One model with 3 input neurons and 2 output neurons (SimpleNeuralNetwork.ProblemModeler/Problems/AddSubtract.cs)

Output Neuron 1 substracts the three input values,

Output Neuron 2 adds them

One model with 49 input neurons and 49 output neurons for Lotto number predictions (SimpleNeuralNetwork.ProblemModeler/Problems/Lotto.cs)
Output Neurons 1-49, show probability for each number

Math Methods

Two available methods depending on the model.

Sigmoid as output method is in in the range of 0 to 1, so input/ouput data must me normalized from 0 to 1

HyperTan is in in the range of -1 to 1, so input/ouput data must me normalized from -1 to 1

Program.cs

Working example of how to train the Neural Network to add and substract three decimals

How to

Test NN efficiency by trying unknown numbers as variables with Run:

Use Custom Model in SimpleNeuralNetwork.ProblemModeler/Custom.cs to model your own problem.

Output after training

# data[].attributes.attachment_link.description
A multi-layer perceptron (one input, one output and multiple hidden layers), forward feed neural network with backward propagation and x number of neurons for each layer. Characteristics 1 Input, [1-..]

# data[].attributes.body
More information: Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans, Nature Machine Intelligence (2021). DOI: 10.1038/s42256-021-00307-0 , dx.doi.org/10.1038/s42256-021-00307-0

# data[].attributes.attachment_link.readable_text
Novel Coronavirus SARS-CoV-2 Transmission electron micrograph of SARS-CoV-2 virus particles, isolated from a patient. Image captured and color-enhanced at the NIAID Integrated Research Facility (IRF) in Fort Detrick, Maryland. Credit: National Institute of Allergy and Infectious Diseases, NIH

Researchers have found that out of the more than 300 COVID-19 machine learning models described in scientific papers in 2020, none of them is suitable for detecting or diagnosing COVID-19 from standard medical imaging, due to biases, methodological flaws, lack of reproducibility, and 'Frankenstein datasets.'

The team of researchers, led by the University of Cambridge, carried out a systematic review of scientific manuscripts-published between 1 January and 3 October 2020-describing machine learning models that claimed to be able to diagnose or prognosticate for COVID-19 from chest radiographs (CXR) and computed tomography (CT) images. Some of these papers had undergone the process of peer-review, while the majority had not.

Their search identified 2,212 studies, of which 415 were included after initial screening and, after quality screening, 62 studies were included in the systematic review. None of the 62 models was of potential clinical use, which is a major weakness, given the urgency with which validated COVID-19 models are needed. The results are reported in the journal Nature Machine Intelligence.

Machine learning is a promising and potentially powerful technique for detection and prognosis of disease. Machine learning methods, including where imaging and other data streams are combined with large electronic health databases, could enable a personalized approach to medicine through improved diagnosis and prediction of individual responses to therapies.

"However, any machine learning algorithm is only as good as the data it's trained on," said first author Dr. Michael Roberts from Cambridge's Department of Applied Mathematics and Theoretical Physics. "Especially for a brand-new disease like COVID-19, it's vital that the training data is as diverse as possible because, as we've seen throughout this pandemic, there are many different factors that affect what the disease looks like and how it behaves."

"The international machine learning community went to enormous efforts to tackle the COVID-19 pandemic using machine learning," said joint senior author Dr. James Rudd, from Cambridge's Department of Medicine. "These early studies show promise, but they suffer from a high prevalence of deficiencies in methodology and reporting, with none of the literature we reviewed reaching the threshold of robustness and reproducibility essential to support use in clinical practice."

Many of the studies were hampered by issues with poor quality data, poor application of machine learning methodology, poor reproducibility, and biases in study design. For example, several training datasets used images from children for their 'non-COVID-19' data and images from adults for their COVID-19 data. "However, since children are far less likely to get COVID-19 than adults, all the machine learning model could usefully do was to tell the difference between children and adults, since including images from children made the model highly biased," said Roberts.

Many of the machine learning models were trained on sample datasets that were too small to be effective. "In the early days of the pandemic, there was such a hunger for information, and some publications were no doubt rushed," said Rudd. "But if you're basing your model on data from a single hospital, it might not work on data from a hospital in the next town over: the data needs to be diverse and ideally international, or else you're setting your machine learning model up to fail when it's tested more widely."

In many cases, the studies did not specify where their data had come from, or the models were trained and tested on the same data, or they were based on publicly available 'Frankenstein datasets' that had evolved and merged over time, making it impossible to reproduce the initial results.

Another widespread flaw in many of the studies was a lack of involvement from radiologists and clinicians. "Whether you're using machine learning to predict the weather or how a disease might progress, it's so important to make sure that different specialists are working together and speaking the same language, so the right problems can be focused on," said Roberts.

Despite the flaws they found in the COVID-19 models, the researchers say that with some key modifications, machine learning can be a powerful tool in combatting the pandemic. For example, they caution against naive use of public datasets, which can lead to significant risks of bias. In addition, datasets should be diverse and of appropriate size to make the model useful for different demographic group and independent external datasets should be curated.

In addition to higher quality datasets, manuscripts with sufficient documentation to be reproducible and external validation are required to increase the likelihood of models being taken forward and integrated into future clinical trials to establish independent technical and clinical validation as well as cost-effectiveness.

More information: Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans, Nature Machine Intelligence (2021). DOI: 10.1038/s42256-021-00307-0 , dx.doi.org/10.1038/s42256-021-00307-0

# data[].attributes.attachment_link.title
Machine learning models for diagnosing COVID-19 are not yet suitable for clinical use: study

# data[].attributes.attachment_link.description
Researchers have found that out of the more than 300 COVID-19 machine learning models described in scientific papers in 2020, none of them is suitable for detecting or diagnosing COVID-19 from standard medical imaging, due to biases, methodological flaws, lack of reproducibility, and 'Frankenstein datasets.'

# data[].attributes.body
These things intrigue me.

# data[].attributes.attachment_link.description
Neuromorphic computing could unlock low-power machine learning that can run on edge devices. A new algorithm that implements an artificial neuron emitting a sparse number of spikes could help realize this goal.

# data[].attributes.body
A collection of books that I own and find useful.

# data[].attributes.body
Interesting article in the New Scientist on using quantum effects and machine learning.

# data[].attributes.attachment_link.readable_text
Quantum on the brain

Just_Super/Getty Images

Machine learning, a process used to train artificial intelligences, can take an extremely long time - but a quantum trick could massively speed things up for tasks involving particles of light called photons.

In reinforcement learning, an algorithm runs through the same problem over and over again and is given a numerical reward only when it reaches the correct answer. That process teaches it to find the correct answer more quickly when pitted against similar problems later on.

Advertisement

Now Valeria Saggio at the University of Vienna in Austria and her colleagues have added a quantum twist to accelerate this process. They set up an experiment involving a photon moving through a wave guide and ending up in one of four possible states. They tasked an AI with making sure the photon ended up in one particular state, and rewarded it for doing so.

In the classical version of this experiment, without any added quantum effects, the AI would only be able to move the photon to one specific state at a time, being rewarded when it made a correct guess. However, in the quantum version of the experiment, the AI could put the photon in a superposition of more than one state. This allowed it to narrow down the correct answer before making a final, classical guess at the goal state.

"Imagine you have a robot that is standing at a crossroads, and the robot has two options - it can go left or it can go right," says Saggio. "If the robot goes right, it does not receive a reward, but if it goes left it receives a reward. At the next round, the probability of it going left will increase."

That's the classical version of the experiment, but the quantum version would allow it to go left and right simultaneously at each guess, requiring far fewer guesses before it learns to always go left. This strategy sped up the learning time of the AI by 63 per cent, from 270 guesses to just 100.

Journal reference: Nature, DOI: 10.1038/s41586-021-03242-7

# data[].attributes.attachment_link.title
A quantum trick with photons gives machine learning a speed boost

# data[].attributes.attachment_link.description
Machine learning, a process used to train artificial intelligences, can take an extremely long time - but a quantum trick could massively speed things up for tasks involving particles of light called photons. In reinforcement learning, an algorithm runs through the same problem over and over again and is given a numerical reward only when it reaches the correct answer.

# data[].attributes.attachment_link.readable_text
UK spies will need to use artificial intelligence (AI) to counter a range of threats, an intelligence report says.

Adversaries are likely to use the technology for attacks in cyberspace and on the political system, and AI will be needed to detect and stop them.

But AI is unlikely to predict who might be about to be involved in serious crimes, such as terrorism - and will not replace human judgement, it says.

The report is based on unprecedented access to British intelligence.

The Royal United Services Institute (Rusi) think tank also argues that the use of AI could give rise to new privacy and human-rights considerations, which will require new guidance.

The UK's adversaries "will undoubtedly seek to use AI to attack the UK", Rusi says in the report - and this may include not just states, but also criminals.

Fire with fire

The future threats could include using AI to develop deep fakes - where a computer can learn to generate convincing faked video of a real person - in order to manipulate public opinion and elections.

It might also be used to mutate malware for cyber-attacks, making it harder for normal systems to detect - or even to repurpose and control drones to carry out attacks.

In these cases, AI will be needed to counter AI, the report argues.

"Adoption of AI is not just important to help intelligence agencies manage the technical challenge of information overload. It is highly likely that malicious actors will use AI to attack the UK in numerous ways, and the intelligence community will need to develop new AI-based defence measures," argues Alexander Babuta, one of the authors.

The independent report was commissioned by the UK's GCHQ security service, and had access to much of the country's intelligence community.

All three of the UK's intelligence agencies have made the use of technology and data a priority for the future - and the new head of MI5, Ken McCallum, who takes over this week, has said one of his priorities will be to make greater use of technology, including machine learning.

However, the authors believe that AI will be of only "limited value" in "predictive intelligence" in fields such as counter-terrorism.

The often-cited fictional reference is the film Minority Report where technology is used to predict those on the path to commit a crime before they have carried it out.

But the report argues this is less likely to be viable in real-life national security situations.

Acts such as terrorism are too infrequent to provide sufficiently large historical datasets to look for patterns - they happen far less often than other criminal acts, such as burglary.

Even within that data set, the background and ideologies of the perpetrators vary so much that it is hard to build a model of a terrorist profile. There are too many variables to make prediction straightforward, with new events potentially being radically different from previous ones, the report argues.

Any kind of profiling could also be discriminatory and lead to new human-rights concerns.

In practice, in fields like counter-terrorism, the report argues that "augmented" - rather than artificial - intelligence will be the norm - where technology helps human analysts sift through and prioritise increasingly large amounts of data, allowing humans to make their own judgements.

It will be essential to ensure human operators remain accountable for decisions and that AI does not act as a "black box", from which people do not understand the basis on which decisions are made, the report says.

Bit by bit

The authors are also wary of some of the hype around AI, and of talk that it will soon be transformative.

Instead, they believe we will see the incremental augmentation of existing processes rather than the arrival of novel futuristic capabilities.

They believe the UK is in a strong position globally to take a lead, with a concentration of capability in GCHQ - and more widely in the private sector, and in bodies like the Alan Turing Institute and the Centre for Data Ethics and Innovation.

This has the potential to allow the UK to position itself at the leading edge of AI use but within a clear framework of ethics, they say.

The deployment of AI by intelligence agencies may require new guidance to ensure safeguards are in place and that any intrusion into privacy is necessary and proportionate, the report says.

Read more from Gordon:

One of the thorny legal and ethical questions for spy agencies, especially since the Edward Snowden revelations, is how justifiable it is to collect large amounts of data from ordinary people in order to sift it and analyse it to look for those who might be involved in terrorism or other criminal activity.

And there's the related question of how far privacy is violated when data is collected and analysed by a machine versus when a human sees it.

Privacy advocates fear that artificial intelligence will require collecting and analysing far larger amounts of data from ordinary people, in order to understand and search for patterns, that create a new level of intrusion. The authors of the report believe new rules will be needed.

But overall, they say it will be important not to become over-occupied with the potential downsides of the use of technology.

"There is a risk of stifling innovation if we become overly-focused on hypothetical worst-case outcomes and speculations over some dystopian future AI-driven surveillance network," argues Mr Babuta.

"Legitimate ethical concerns will be overshadowed unless we focus on likely and realistic uses of AI in the short-to-medium term."

# data[].attributes.attachment_link.title
UK spies need artificial intelligence, report says

# data[].attributes.attachment_link.description
UK spies will need to use artificial intelligence (AI) to counter a range of threats, an intelligence report says. Adversaries are likely to use the technology for attacks in cyberspace and on the political system, and AI will be needed to detect and stop them.

# data[].attributes.attachment_link.readable_text
Comparison tables can be handy when it comes to getting a quick overview of a specific topic. Below are eight comparison tables from the areas of Artificial Intelligence, Data Science, IoT, and Cloud Computing.

1. Comparison of machine learning methods

Machine learning explores the study and construction of algorithms that can learn from and make predictions on data. Such algorithms operate by building a model from an example training set of input observations in order to make data-driven predictions or decisions expressed as outputs, rather than following strictly static program instructions. ( Wikipedia)

2. Comparison of open-source IoT platforms

The Internet of Things (IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that are provided with unique identifiers (UIDs) and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. ( Wikipedia)

3. The main differences between Data Science and Business Intelligence

Data Science

Data science is a multi-disciplinary field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from structured and unstructured data. Data science is the same concept as data mining and big data: "use the most powerful hardware, the most powerful programming systems, and the most efficient algorithms to solve problems" ( Wikipedia)

Business Intelligence

Business intelligence (BI) comprise the strategies and technologies used by enterprises for the data analysis of business information. BI technologies provide historical, current and predictive views of business operations. Common functions of business intelligence technologies include reporting, online analytical processing, analytics, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics. ( Wikipedia)

4. Comparison of Classification Algorithms

In machine learning and statistics, classification is the problem of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagnosis to a given patient based on observed characteristics of the patient (sex, blood pressure, presence or absence of certain symptoms, etc.). Classification is an example of pattern recognition. ( Wikipedia)

5. Comparison of Deep Learning Frameworks

Deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on artificial neural networks. Learning can be supervised, semi-supervised or unsupervised.

Deep learning architectures such as deep neural networks, deep belief networks, recurrent neural networks and convolutional neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design, medical image analysis, material inspection and board game programs, where they have produced results comparable to and in some cases superior to human experts. ( Wikipedia)

6. Comparison of pre-trained models

In computer vision, transfer learning is usually expressed through the use of pre-trained models. A pre-trained model is a model that was trained on a large benchmark dataset to solve a problem similar to the one that we want to solve. ( Towards Data Science)

7. AWS, Azure, GCP Comparison

Cloud computing is the on-demand availability of computer system resources, especially data storage and computing power, without direct active management by the user. The term is generally used to describe data centers available to many users over the Internet. Large clouds, predominant today, often have functions distributed over multiple locations from central servers. If the connection to the user is relatively close, it may be designated an edge server. ( Wikipedia)

8. RPA versus Conventional Technology

RPA Robotic process automation (or RPA) is an emerging form of business process automation technology based on the notion of metaphorical software robots or artificial intelligence workers.

In traditional workflow automation tools, a software developer produces a list of actions to automate a task and interface to the back-end system using internal application programming interfaces (APIs) or dedicated scripting language. In contrast, RPA systems develop the action list by watching the user perform that task in the application's graphical user interface (GUI) and then perform the automation by repeating those tasks directly in the GUI. This can lower the barrier to the use of automation in products that might not otherwise feature APIs for this purpose. ( Wikipedia)

Useful Comparison Tables for AI, Data Science, IoT & Cloud #AI #DataScience #IoT #CloudComputing Klick um zu Tweeten

Further reading:

# data[].attributes.attachment_link.title
Useful Comparison Tables for AI, Data Science, IoT & Cloud

# data[].attributes.attachment_link.description
Comparison tables can be handy when it comes to getting a quick overview of a specific topic. Below are eight comparison tables from the areas of Artificial Intelligence, Data Science, IoT, and Cloud Computing. Machine learning explores the study and construction of algorithms that can learn from and make predictions on data.

# data[].attributes.attachment_link.readable_text
Have you ever asked a question, why do we need to calculate the exact Posterior distribution ?

To understand the answer the above question, let us go to -

To understand the answer of the above question, I would like you to re-visit our basic Baye's rule.

Posterior = (Likelihood x Prior) / Evidence

Now to compute the Posterior we have following issues:

If prior is conjugate to likelihood, it is easy to compute the posterior, otherwise not

It is really very hard to compute the evidence of the points

So, what if we try and approximate our posterior!

Will it impact our results? .. let us check out with an example.

Let us suppose we have a probabilistic distribution as shown in the figure:

The computation of the exact posterior of the above distribution is very difficult.

Now suppose we try to approximate the above distribution with a gaussian distribution which looks as follows:

Now from the above figure we can say the error is very small as the gaussian distribution is conveniently satisfying the job. Also, in machine learning problems, we really do not want an exact value of the posterior as far as we can get relative values for decision making.

From the above example we saw that if we have good approximate distribution for the posterior, we are good to go !

The next question arises, how to compute this approximate posterior distribution.

Computing the Approximate Distribution

To compute the approximate distribution, we will have to consider a family of distribution and then we will try to minimize the KL Divergence parameter between the actual posterior and the approximate distribution.

Mathematically speaking,

KL Divergence :

Expanding the above equation, we get :

as q(z) is a probabilistic distribution and Log z is a constant for a given data, therefore It does not play a role in differentiation process of KL divergence to minimize the distance between the approximate posterior distribution and the actual distribution.

So, now we will understand a methodology to do this approximation.

The name suggests a word 'mean', we will see the importance of this later in the blog. Another word 'field' is basically coming from electromagnetism theory of physics, as this approximation methodology incorporates the impact of nearby neighbors in making the decision, thus incorporating the fields impacts of all neighbors.

So, let us start learning the concept behind this approximation.

Step 1 : Suppose the posterior distribution has n dimensions, then we will consider total n different distributions to approximate the posterior.

Step 2 : all the n different distributions will be multiplied together to approximate the posterior. The mathematical expression looks as :

Step 3 : now the expression of KL divergence is used to differentiate and minimize the distance between the posterior and approximation. We use coordinate descent method to differentiate the expression, which differentiates the expression w.r.t to one variable, update the expression and then differentiate w.r.t another variable and so on.

Mathematical Derivation

KL divergence expression :

Now, we only need to differentiate w.r.t k, therefore all the dimensions are constant as per coordinate descent.

By taking q(z k) as common and treating the remain multiplier as a constant and for the second object for P(z*), it becomes as Expected value for the random variable q(z ₁)*q(z ₂) ... q(z i) where i != k.

The final expression we get is :

Now, with a little conversion of the equation, it re occurs in form of KL divergence, and to differentiate and equate it to zero means equating the denominator and numerator. The final expression is as follows:

Now because of the mean impact of remaining dimensions results in th most optimal solution of the k th dimension, we call it mean fields.

Now, what you just saw may be very scary !!! right ..

Let us take a working example to understand the concept of mean field approximation in more practical manner.

Grey scale image de-noising is a very popular use case of MFVI. It uses 2 concepts - Ising model and approximating the ising model using Mean field approximation. Below are the details:

Ising Model : is a mathematical model of ferromagnetism in statistical mechanics.

Core Idea behind Merging two theories : ferromagnetic substances, at their molecular level can have 2 types of spins - clockwise or anticlockwise. If the spins of all the molecules are same and aligned, the object start behaving as a magnet. The spin of any molecule has a significant impact from its neighboring molecules. This impact varies from substance to substance. Thus it makes the occurrence of the spin for each molecule probabilistic. Ising Model defines this behavior mathematically.

Now, we can consider our image as 2-D substance sheet and can consider every pixel of the image as a molecule. In a binary-image, these pixels can take only 2 values i.e. black and white, analogously -1 and 1. This image can be de-noised by considering the same theory that each pixel will take value as per its neighbor. And so we can apply Ising model to re correct the spins/black-white/-1,+1 values of the image, if we control the neighbor impact property of the image.

Now, the joint probabilistic distribution of the Ising model is impossible to evaluate analytically, thus we use Mean Field Approximation theory to evaluate it.

I tried this approach on a grey image of a bird by adding a lot of noise to it and tried to retrieve as much as possible from it. Below are my results :

Noisy Image:

De-noised Image:

That is kind of potential I am talking about, i.e. in the noisy image you can see eventually nothing, and in the de-noised image we see a bird. !!!

Please read more about it, to try this cool stuff.

References:

shttps://github.com/vsmolyakov/experiments_with_python/blob/master/chp02/mean_field_mrf.ipynb
https://en.wikipedia.org/wiki/Ising_model https://en.wikipedia.org/wiki/Mean_field_theory https://www.cs.cmu.edu/~epxing/Class/10708-17/notes-17/10708-scribe...

# data[].attributes.attachment_link.description
Have you ever asked a question, why do we need to calculate the exact Posterior distribution ? To understand the answer the above question, let us go to - To understand the answer of the above question, I would like you to re-visit our basic Baye's rule.

# data[].attributes.body
Reading Resource/Reference in term 2.
By Francois Chollet.
Posted by Carl.H

# data[].attributes.attachment_link.title
Kismet: the Social Robot | PBS LearningMedia

# data[].attributes.attachment_link.description
Researchers in the Artificial Intelligence Laboratory working to engineer smarter robots are now building a machine that interacts socially with people in this video segment adapted from the Massachusetts Institute of Technology.

# data[].attributes.body
https://www.youtube.com/watch?v=kSLJriaOumA&feature=emb_logo

# data[].attributes.attachment_link.readable_text
Bayes Theorem is the extension of Conditional probability. Conditional probability helps us to determine the probability of A given B, denoted by P(A|B). So Bayes' theorem says if we know P(A|B) then we can determine P(B|A), given that P(A) and P(B) are known to us.

In this post I am concentrating on Bayes' theorem assuming you have good understanding of Conditional probability. In case you want to revise your concepts, you may refer my previous post on Conditional probability with examples.

Formula derivation:

From conditional probability we know that

Similarly

P(B|A) = P(B and A)/P(A) = P(A and B)/P(A) [In Joint Probability order does not matter]

P(A and B) = P(A) * P(B|A) -----[2]

From equation [1] and [2],
Which mean if we know P(A|B) then we can easily determine P(B|A) and vice versa. Assuming we know the total probabilities P(A) and P(B). More Generalized Definition:
Let A1, A2, A3... Ak be a collection of mutually exclusive and exhaustive events with probability P(Ai), I = 1, 2, 3...k. Then for any event B for which P(B) > 0

P(Ai|B) = P(Ai and B)/P(B)

= P(B|Ai)*P(Ai)/∑[P(B|Ai)*P(Ai)

Below is the tree representation of the Bayes' Theorem.

Bayes' Theorem using Probability Tree

Now let's solve some example to get a feeling of Bayes' theorem.

Example 1

Technicians regularly make repairs when breakdowns occur on an automated production line. Janak, who services 20% of the breakdowns, makes an incomplete repair 1 time in 20.Tarun, who services 60% of the breakdowns, makes an incomplete repair 1 time in 10. Gautham, who services 15% of the breakdowns, makes an incomplete repair 1 time in 10 and Prasad, who services 5% of the breakdowns, makes an incomplete repair 1 time in 20. For the next problem with the production line diagnosed as being due to an initial repair that was incomplete, what is the probability that this initial repair was made by Janak?

Solution:

First let's try to visualize using probability tree.

Bayes Theorem Example 1

Example 2 Clinical trials

Epidemiologists claim that probability of breast cancer among Caucasian women in their mid-50s is 0.005. An established test identified people who had breast cancer and those that were healthy. A new mammography test in clinical trials has a probability of 0.85 for detecting cancer correctly. In women without breast cancer, it has a chance of 0.925 for a negative result. If a 55-year-old Caucasian woman tests positive for breast cancer, what is the probability that she in fact has breast cancer?

Solution:

P(Cancer) = 0.005

P(Test Positive | Cancer) = 0.85

P(Test Neg|No cancer) = 0.925

P(Cancer|Test Positive)= P(Cancer) * P(Test Positive | Cancer) / P(Test Positive)

Probability Table and Tree Visualization for the given question

Bayes Theorem Example 2

Example 3

SpamAssassin works by having users train the system. It looks for patterns in the words in emails marked as spam by the user. For example, it may have learned that the word "free" appears in 20% of the mails marked as spam. Assuming 0.1% of non-spam mail includes the word "free" and 50% of all mails received by the user are spam, find the probability that a mail is spam if the word "free" appears in it.

Solution

Data Given:

P(Free | Spam) = 0.20

P(Free | Non Spam) = 0.001

P(Spam) = 0.50 => P(Non Spam) = 0.50

P(Spam | Free) = ?

Using Bayes' Theorem:

P(Spam | Free) = P(Spam) * P(Free | Spam) / P(Free)

P(Spam | Free) = 0.50 * 0.20 / (0.50 * 0.20 + 0.50 * 0.001)

P(Spam | Free) = 0.995

Now visualize using probability tree:

Bayes' Theorem Example 3

So that is all about Bayes' Theorem for data Science. In next posts I will be writing about Probability Distributions which will complete the Probability for Data Science Series.

Previous articles from Probability for Data Science Series are:

I hope you enjoyed the post. For any comments, use the below comment section to write back to me or even you can directly write to me using https://ashutoshtripathi.com/contact/. I will try to answer all your queries related to DS, ML and AI.

Thank You!

# data[].attributes.attachment_link.title
Bayes' Theorem with Example for Data Science Professionals

# data[].attributes.attachment_link.description
Bayes Theorem is the extension of Conditional probability. Conditional probability helps us to determine the probability of A given B, denoted by P(A|B). So Bayes' theorem says if we know P(A|B) then we can determine P(B|A), given that P(A) and P(B) are known to us.

# data[].attributes.attachment_link.readable_text
The semantic web focuses on data rather than on documents, making it a much more immersive and detailed way of accessing information compared to the World Wide Web invented by Tim Berners Lee in the late 1980s.

However, Berners-Lee still played a very important role in its inception, developing the idea alongside AI researcher James Hendler and computer scientist Ora Lassila. The idea was first revealed in 2001 in a Scientific America article, where the threesome discussed the idea of connecting information using a network that could be read by machines.

According to the World Wide Web Consortium ( W3C), the semantic web is "a common framework that allows data to be shared and reused across application, enterprise, and community boundaries".

The concept is to offer people the information they're looking for at the time they need it. One of its key philosophies is that although the information presented on the internet is useful, it's not always needed at every point.

Because the majority of data is created using forms and then converted into HTML, there's no way all data can be managed by everyone at all times. The semantic web makes this information more useful to everyone because it can be repurposed.

The semantic web essentially allows for the connection of information using a network that can be easily read by machines - whether computers, IoT devices, mobile phones or other devices commonly used to access information.

It's built on the premise that data within web pages is useful, but not in all circumstances. One of the biggest hurdles of the internet as it stands is that the majority of data is created using forms and there's no unified way of publishing data so anyone can manage it. The way data is presented using HTML can be difficult to handle and so the semantic web takes the idea that if this data can be re-purposed, it's more useful to everyone.

Schema.org has been formed by a number of organisations (notably Google, Bing and Yahoo) to boost the extent of semantic metadata. The goal of this is to answer questions from the best sources on the web, rather than serve up a search page full of document links.

The most important part of semantic web technologies is Resource Description Framework (RDF). This is a common framework for describing resources. It can represent metadata that can be parsed and processed by systems rather than just displayed to users.

Use cases

The semantic web is very useful for solving many of the problems raised with the World Wide Web.

For example, data siloes can be pretty much eradicated, with links between data and the outside world - or even more localised places such as within the organisation - operating seamlessly. The use of semantic metadata tags means the information can all reside in one place, searchable by the tags to make it infinitely easier to uncover.

If the linked data is in a publicly searchable place, such as the wider internet, users are also able to find intricate relationships between the data and the information they own, opening up its meaning to go far beyond the scope of the original data.

Another great use for the semantic web is for media management. For example, the BBC used the semantic web to power its database of player information during the 2010 World Cup and a lot of its website runs on semantic web technologies to make sure it can rapidly update and organise the vast amount information it holds.

In supply chains, the semantic web can be used to keep the rapidly changing data organised, whether that's information supplied by different parts of the chain, such as manufacturers, vendors, distributors, logistics firms, supply chain managers.

Benefits

One of the key benefits of the semantic web is having large amounts of data, knowledge and information made understandable and accessible to machines, especially artificially intelligent bots, virtual assistants and agents.

The simplicity of the RDF data structure and the schema's optional nature mean that it's easy to combine different sets of data. This is particularly useful for big data projects where the variety of data within an organisation can present a challenge.

# data[].attributes.attachment_link.description
The semantic web focuses on data rather than on documents, making it a much more immersive and detailed way of accessing information compared to the World Wide Web invented by Tim Berners Lee in the late 1980s. However, Berners-Lee still played a very important role in its inception, developing the idea alongside AI researcher James Hendler and computer scientist Ora Lassila.

# data[].attributes.attachment_link.readable_text
| Metalog

Many people don't have a clear idea of what the Semantic Web is. The best informal definition is maybe found in the May 2001 Scientific American article "The Semantic Web" (Berners-Lee et al.), that says " The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation." People who work on the Semantic Web have a more detailed idea of what this broad statement means, and base their work on the famous " semantic web tower", a product of Tim Berners-Lee's inspiring drawing on whiteboards. Even if you are not a geek, you might have read some article on the semantic web, or followed some presentation, and then you will surely have met the tower, which is a drawing like this

The article or presentation might have then continued by explaining what the "layers" of this tower are, starting from the basic bricks (Unicode and URIs), up to XML, and then up to more and more sophisticated layers like RDF, Ontology and so on.

Now, you might have understood how things work, and have been eagerly awaiting for the realization of this plan. Or, you might not have understood this quite, and have been waiting for someone to show you better what they meant. In fact, the plan as shown in the picture is rapidly taking place: many of the slots are filling up (RDF, RDF Schema, Ontology, Digital Signature). So, the picture as shown is getting completed. Still, most of the people are awaiting, and the Semantic Web has not yet taken off. What's the matter here?

The matter comes from the definition of Semantic Web that we have seen earlier: "The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation." Now, the sole fact that the tower is filling up with established standards corresponds to the first part of the definition, "an extension of the current web". The fact that these standards work corresponds to a piece of the second part of the definition: "enabling computers to work in cooperation". What is missing, is maybe the most crucial part of the definition: the people. Those people who are awaiting.

So, it's of little importance to those people that the slots work with computers: they want to be part of the game, they want the Semantic Web to fulfill its promise, to enable "computers and people to work in cooperation". Filling up the tower with standards reach the first part of the game, and makes computers happy, but not (yet) people. The tower represents the axis of expressive power, where more and more sophisticated semantics can be encoded. This technical axis doesn't much bother to explicitly take into account the other part of the Semantic Web world. Therefore, a different perspective would be to take that component into proper account, abstracting from the expressiveness axis: let's twist the tower, and build up, orthogonally, a new axis:

The "perception" axis is to signify the abstraction/adaptation of the technologies in the tower, towards the people. To enable their perception by the people. In fact, this axis could have been well named as the "People axis", but perception is maybe a more detailed term. In any case, as both perception and people start with P, this can simply be named asthe "P" axis.

The P axis is the one that points towards people, and as such, bases its foundation (talking to computers) in some part of the underlying tower. It is the connection axis between these two worlds, people and computers. This axis is not just made up of applications: just like the expressiveness axis (the "tower") is made by technologies, technologies can well (and should) be first-class citizens in this axis as well. Implementations alone are isolated objects, and can't tell how to communicate with other things. And so, the P axis cannot be left to implementations, leaving the "communication" part just to the underlying tower.

Technologies for the people, in this axis, might sacrifice part of the expressive power given by some underlying technology in the main tower, to build up a technology that is closer to the people. All this, in a standard and interoperable manner. In the overall picture, it's in the P axis that people matter.
[Next to: the PNL interface]

# data[].attributes.attachment_link.description
Metalog Many people don't have a clear idea of what the Semantic Web is. The best informal definition is maybe found in the May 2001 Scientific American article "The Semantic Web" (Berners-Lee et al.), that says " The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation."

# data[].attributes.headline
Basic video but gives foundations for understanding linked data 

# data[].attributes.subject
Basic video but gives foundations for understanding linked data 

# data[].attributes.attachment_link.readable_text
Following is the difference between the forward chaining and backward chaining:

Forward chaining as the name suggests, start from the known facts and move forward by applying inference rules to extract more data, and it continues until it reaches to the goal, whereas backward chaining starts from the goal, move backward by using inference rules to determine the facts that satisfy the goal.

Forward chaining is called a data-driven inference technique, whereas backward chaining is called a goal-driven inference technique.

Forward chaining is known as the down-up approach, whereas backward chaining is known as a top-down approach.

Forward chaining uses breadth-first search strategy, whereas backward chaining uses depth-first search strategy.

Forward and backward chaining both applies Modus ponens inference rule.

Forward chaining can be used for tasks such as planning, design process monitoring, diagnosis, and classification, whereas backward chaining can be used for classification and diagnosis tasks.

Forward chaining can be like an exhaustive search, whereas backward chaining tries to avoid the unnecessary path of reasoning.

In forward-chaining there can be various ASK questions from the knowledge base, whereas in backward chaining there can be fewer ASK questions.

Forward chaining is slow as it checks for all the rules, whereas backward chaining is fast as it checks few required rules only.

# data[].attributes.attachment_link.title
Difference Between Backward Chaining and Forward Chaining - Javatpoint

# data[].attributes.attachment_link.description
Following is the difference between the forward chaining and backward chaining: Forward chaining as the name suggests, start from the known facts and move forward by applying inference rules to extract more data, and it continues until it reaches to the goal, whereas backward chaining starts from the goal, move backward by using inference rules to determine the facts that satisfy the goal.

# data[].attributes.attachment_link.readable_text
A Self-Organising Map, additionally, uses competitive learning as opposed to error-correction learning, to adjust it weights. This means that only a single node is activated at each iteration in which the features of an instance of the input vector are presented to the neural network, as all nodes compete for the right to respond to the input.

The chosen node - the Best Matching Unit (BMU) - is selected according to the similarity, between the current input values and all the nodes in the grid.

The node with the smallest Euclidean difference between the input vector and all nodes is chosen, along with its neighbouring nodes within a certain radius, to have their position slightly adjusted to match the input vector.

By going through all the nodes present on the grid, the entire grid eventually matches the complete input dataset, with similar nodes grouped together towards one area, and dissimilar ones separated.

A Kohonen model with the BMU in yellow, the layers inside the neighbourhood radius in pink and purple, and the nodes outside in blue.

t is the current iteration

n is the iteration limit, i.e. the total number of iterations the network can undergo

λ is the time constant, used to decay the radius and learning rate

i is the row coordinate of the nodes grid

j is the column coordinate of the nodes grid

d is the distance between a node and the BMU

w is the weight vector

w_ij(t) is the weight of the connection between the nodes i,j in the grid, and the input vector's instance at the iteration t

x is the input vector

x(t) is the input vector's instance at iteration t

α(t) is the learning rate, decreasing with time in the interval [0,1], to ensure the network converges.

β_ij(t) is the neighbourhood function, monotonically decreasing and representing a node i, j's distance from the BMU, and the influence it has on the learning at step t.

σ(t) is the radius of the neighbourhood function, which determines how far neighbour nodes are examined in the 2D grid when updating vectors. It is gradually reduced over time.

1. Initialise each node's weight w_ij to a random value

2. Select a random input vector x_k

3. Repeat point 4. and 5. for all nodes in the map:

4. Compute Euclidean distance between the input vector x(t) and the weight vector w_ij associated with the first node, where t, i, j = 0.

5. Track the node that produces the smallest distance t.

6. Find the overall Best Matching Unit (BMU), i.e. the node with the smallest distance from all calculated ones.

7. Determine topological neighbourhood βij(t) its radius σ(t) of BMU in the Kohonen Map

8. Repeat for all nodes in the BMU neighbourhood: Update the weight vector w_ij of the first node in the neighbourhood of the BMU by adding a fraction of the difference between the input vector x(t) and the weight w(t) of the neuron.

9. Repeat this whole iteration until reaching the chosen iteration limit t=n

Step 1 is the initialisation phase, while step 2-9 represent the training phase.

The updates and changes to the variables are done according to the following formulas:

The weights within the neighbourhood are updated as:

The first equation tells us that the new updated weight w_ij (t + 1) for the node i, j is equal to the sum of old weight w_ij(t) and a fraction of the difference between the old weight and the input vector x(t). In other words, the weight vector is 'moved' closer towards the input vector. Another important element to note is that the updated weight will be proportional to the 2D distance between the nodes in the neighbourhood radius and the BMU.

Furthermore, the same equation 3.1 does not account for the influence of the learning being proportional to the distance a node is from the BMU. The updated weight should take into factor that the effect of the learning is close to none at the extremities of the neighbourhood, as the amount of learning should decrease with distance. Therefore, the second equation adds the extra neighbourhood function factor of βij(t), and is the more precise in-depth one.

The radius and learning rate are both similarly and exponentially decayed with time.

The neighbourhood function's influence β_i(t) is calculated by:

The Euclidean distance between each node's weight vector and the current input instance is calculated by the Pythagorean formula.

The BMU is selected from all the node's calculated distances as the one with the smallest.

# data[].attributes.attachment_link.description
A Self-Organising Map, additionally, uses competitive learning as opposed to error-correction learning, to adjust it weights. This means that only a single node is activated at each iteration in which the features of an instance of the input vector are presented to the neural network, as all nodes compete for the right to respond to the input.

# data[].attributes.attachment_link.readable_text
Introduction

In this post, we will take a look at a few cases, where KMC algorithm does not perform well or may produce unintuitive results. In particular, we will look at the following scenarios:

1. Our guess on the number of (real) clusters is off.

2. Feature space is highly dimensional.

3. The clusters come in strange or irregular shapes.

All of these conditions can lead to problems with K-Means, so let's have a look.

Wrong number of clusters

To make it easier, let's define a helper function compare, which will create and solve the clustering problem for us and then compare the results.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45



from sklearn import datasets
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs , make_circles , make_moons
from mpl_toolkits.mplot3d import Axes3D

import numpy as np
import pandas as pd
import itertools


def compare ( N_features , C_centers , K_clusters , dims = [ 0 , 1 ], * args ):
    data , targets = make_blobs (
        n_samples = n_samples if 'n_samples' in args else 400 ,
        n_features = N_features ,
        centers = C_centers ,
        cluster_std = cluster_std if 'cluster_std' in args else 0.5 ,
        shuffle = True ,
        random_state = random_state if 'random_state' in args else 0 )

    FEATS = [ 'x' + str ( x ) for x in range ( N_features )]
    X = pd . DataFrame ( data , columns = FEATS )
    X [ 'cluster' ] = \
		KMeans ( n_clusters = K_clusters , random_state = 0 ) . fit_predict ( X )

    fig , axs = plt . subplots ( 1 , 2 , figsize = ( 12 , 4 ))
    axs [ 0 ] . scatter ( data [:, dims [ 0 ]], data [:, dims [ 1 ]],
        c = 'white' , marker = 'o' , edgecolor = 'black' , s = 20 )
    axs [ 0 ] . set_xlabel ( 'x{} [a.u.]' . format ( dims [ 0 ]))
    axs [ 0 ] . set_ylabel ( 'x{} [a.u.]' . format ( dims [ 1 ]))
    axs [ 0 ] . set_title ( 'Original dataset' )
    axs [ 1 ] . set_xlabel ( 'x{} [a.u.]' . format ( dims [ 0 ]))
    axs [ 1 ] . set_ylabel ( 'x{} [a.u.]' . format ( dims [ 1 ]))
    axs [ 1 ] . set_title ( 'Applying clustering' )

    colors = itertools . cycle ([ 'r' , 'g' , 'b' , 'm' , 'c' , 'y' ])
    for k in range ( K_clusters ):
        x = X [ X [ 'cluster' ] == k ][ FEATS ] . to_numpy ()
        axs [ 1 ] . scatter (
		x [:, dims [ 0 ]],
		x [:, dims [ 1 ]],
		color = next ( colors ),
		edgecolor = 'k' ,
		alpha = 0.5
	)
    plt . show ()

Too few clusters

Figure 1a. Example of a 2-dimensional dataset with 4 centres, requesting 3 clusters (compare(2, 4, 3)).

Despite having distinct clusters in the data, we underestimated their number. As a consequence, some disjoint groups of data are forced to fit into one larger cluster.

Too many clusters

Figure 1b. Example of a 2-dimensional dataset with 2 centres, requesting 4 clusters (compare(2, 2, 4)).

In contrary to the last situation, trying to wrap the data into too many clusters creates artificial boundaries within real data clusters.

High(er) dimensional data

A dataset does not need to be that high in dimentionality before we begin to see problems. Although visualization and thus somewhat analysis of high dimensional data is already challenging (cursing now...), as KMC is often used to gain insight into the data, it does not help to be presented with ambiguities.

To explain the point, let's generate a three-dimensional dataset with clearly distinct clusters.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19



fig = plt . figure ( figsize = ( 14 , 8 ))
ax = fig . add_subplot ( 111 , projection = '3d' )

data , targets = make_blobs (
    n_samples = 400 ,
    n_features = 3 ,
    centers = 3 ,
    cluster_std = 0.5 ,
    shuffle = True ,
    random_state = 0 )

ax . scatter ( data [:, 0 ], data [:, 1 ],
    zs = data [:, 2 ], zdir = 'z' , s = 25 , c = 'black' , depthshade = True )
ax . set_xlabel ( 'x0 [a.u.]' )
ax . set_ylabel ( 'x1 [a.u.]' )
ax . set_zlabel ( 'x2 [a.u.]' )
ax . set_title ( 'Original distribution.' )
plt . grid ()
plt . show ()

Figure 2. Example of a 3-dimensional dataset with 3 centers.

Although there are infinitely many ways we can project this 3D dataset onto 2D, there are three primary orthogonal sub-spaces:

Looking at the x2 : x0 projection, the dataset looks like as if it only had two clusters. The lower-right "supercluster" is, in fact, two distinct groups and even if we guess K right (K = 3), it looks like an apparent error, despite the clusters are very localized.

Figure 3a. Projection on `x0 : x2` shows spurious result (compare(2, 2, 4, dims=[0, 2])).

To be sure, we have to look at the remaining projections to see the problem, literally, from different angles.

Figure 3b. Projection on `x1 : x2` resolves the ambiguity (compare(2, 2, 4, dims=[1, 2])).

Figure 3c. Projection on `x0 : x1` resolves the ambiguity (compare(2, 2, 4, dims=[0, 1])).

This makes more sense!

On the flip side, we had an incredible advantage. First, with three dimensions, we were able to plot the entire dataset. Secondly, the clusters that exist within the dataset were very distinct thus easy to spot. Finally, with three-dimensional dataset, we were facing only three standard 2D projections.

The case of N, N > 3 features, we would not be able to plot the whole dataset, and the number of 2D projections would scale quadratically with N:

not to mention that the dataset may have strangely shaped or non-localized clusters, which is our next challenge.

Irregular datasets

So far we mentioned problems that are on "our side". We looked at a very "well-behaved" dataset and discussed issues on the analytics side. However, what about if the dataset does not fit our solution, or, our solution does not fit the problem? This is exactly the case, where data distribution comes in strange or irregular shapes.

Being presented with just this graph, we may be tricked into believing that there are only two clusters in the data. However, when plotting the remaining projections, we quickly learn that this is not true.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38



fig , axs = plt . subplots ( 1 , 3 , figsize = ( 14 , 4 ))

# unequal variance
X , y = make_blobs ( n_samples = 1400 ,
    cluster_std = [ 1.0 , 2.5 , 0.2 ],
    random_state = 2 )
y_pred = KMeans ( n_clusters = 3 , random_state = 2 ) . fit_predict ( X )
colors = [[ 'r' , 'g' , 'b' ][ c ] for c in y_pred ]

axs [ 0 ] . scatter ( X [:, 0 ], X [:, 1 ],
	color = colors , edgecolor = 'k' , alpha = 0.5 )
axs [ 0 ] . set_title ( "Unequal Variance" )

# anisotropically distributed data
X , y = make_blobs ( n_samples = 1400 , random_state = 156 )
transformation = [
	[ 0.60834549 , - 0.63667341 ],
	[ - 0.40887718 , 0.85253229 ]
]
X = np . dot ( X , transformation )
y_pred = KMeans ( n_clusters = 3 , random_state = 0 ) . fit_predict ( X )
colors = [[ 'r' , 'g' , 'b' ][ c ] for c in y_pred ]

axs [ 1 ] . scatter ( X [:, 0 ], X [:, 1 ],
	color = colors , edgecolor = 'k' , alpha = 0.5 )
axs [ 1 ] . set_title ( "Anisotropicly Distributed Blobs" )

# irregular shaped data
X , y = make_moons ( n_samples = 1400 , shuffle = True ,
	noise = 0.1 , random_state = 120 )
y_pred = KMeans ( n_clusters = 2 , random_state = 0 ) . fit_predict ( X )
colors = [[ 'r' , 'g' , 'b' ][ c ] for c in y_pred ]

axs [ 2 ] . scatter ( X [:, 0 ], X [:, 1 ],
	color = colors , edgecolor = 'k' , alpha = 0.5 )
axs [ 2 ] . set_title ( "Irregular Shaped Data" )

plt . show ()

Figure 4. Misleading clustering results are shown on irregular datasets.

The left graph shows data whose distribution although, Gaussian, does not have equal standard deviation. The middle graph presents anisotropic data, meaning data that is elongated along a specific axis. Finally, the right graph shows data that is completely non-Gaussian, despite organized in clear clusters.

In either case, the irregularity makes KMC algorithm underperform. Since the algorithm treats every data point equally and completely independently from other points, the algorithm fails to spot any possible continuity or local variations within a cluster. What it does is simply taking the same metrics and applying it to every point. As a result, the KMC algorithm may produce strange or counter-intuitive clustering within the data even if we guess K correctly and the features N are not that many.

Conclusions

In this post, we have discussed three main reasons for the K-Means Clustering algorithm to give us wrong answers.

First, as the number of clusters K needs to be decided a priori, there is a high chance that we will guess it wrongly.

Secondly, clustering in higher dimensional space becomes cumbersome from the analytics point of view, in which case KMC will provide us with insights that may be misleading.

Finally, for any irregularly shaped data, KMC is likely to artificial clusters that do not conform to common sense.

Knowing these three fallacies, KMC is still a useful tool, especially when inspecting of the data or constructing labels.

# data[].attributes.attachment_link.title
Top three mistakes with K-Means Clustering during data analysis

# data[].attributes.attachment_link.description
In this post, we will take a look at a few cases, where KMC algorithm does not perform well or may produce unintuitive results. In particular, we will look at the following scenarios: Our guess on the number of (real) clusters is off. Feature space is highly dimensional.

# data[].attributes.attachment_link.readable_text
All around the world, from small-town Illinois in the US to Rochdale in England, from Perth, Australia, to Dumka in northern India, a revolution is under way in how governments treat the poor.

You can't see it happening, and may have heard nothing about it. It's being planned by engineers and coders behind closed doors, in secure government locations far from public view.

Only mathematicians and computer scientists fully understand the sea change, powered as it is by artificial intelligence (AI), predictive algorithms, risk modeling and biometrics. But if you are one of the millions of vulnerable people at the receiving end of the radical reshaping of welfare benefits, you know it is real and that its consequences can be serious - even deadly.

The Guardian has spent the past three months investigating how billions are being poured into AI innovations that are explosively recasting how low-income people interact with the state. Together, our reporters in the US, Britain, India and Australia have explored what amounts to the birth of the digital welfare state.

Their dispatches reveal how unemployment benefits, child support, housing and food subsidies and much more are being scrambled online. Vast sums are being spent by governments across the industrialized and developing worlds on automating poverty and in the process, turning the needs of vulnerable citizens into numbers, replacing the judgment of human caseworkers with the cold, bloodless decision-making of machines.

At its most forbidding, Guardian reporters paint a picture of a 21st-century Dickensian dystopia that is taking shape with breakneck speed. The American political scientist Virginia Eubanks has a phrase for it: "The digital poorhouse."

The Australian government's Department of Human Services website. Photograph: Dave Hunt/AAP

Listen to governments, and you will hear big promises about how new technologies will transform poverty as a noble and benign enterprise. They will speed up benefits payments, increase efficiency and transparency, reduce waste, save money for taxpayers, eradicate human fallibility and prejudice, and ensure that limited resources reach those most in need. But so often, those pledges have fallen flat.

At a time when austerity dominates the political landscape, millions have had their benefits slashed or stopped by computer programs that operate in ways that few seem able to control or even comprehend. Mistakes have become endemic, with no obvious route for the victims of the errors to seek redress.

This week, the automation of poverty will be brought on to the world stage. Philip Alston, a human rights lawyer who acts as the UN's watchdog on extreme poverty, will present to the UN general assembly in New York a groundbreaking report that sounds the alarm about the human rights implications of the rush to digitalize social protection.

Alston's analysis is based partly on his official UN studies of poverty in the UK and US, and partly on submissions from governments, human rights organisations and experts from more than 34 countries. It is likely to provide the definitive snapshot of where the world lies now, and where it is going, addressing the harassment, targeting and punishment of those living in the rapidly expanding digital poorhouse.

Mistakes have become endemic, with no obvious route for the victims to seek redress

In Illinois, the Guardian has found that state and federal governments have joined forces to demand that welfare recipients repay "overpayments" stretching back in some cases 30 years. This system of "zombie debt", weaponized through technology, is invoking fear and hardship among society's most vulnerable.

As one recipient described it: "You owe what you have eaten."

In the UK, we investigate the secure government site outside Newcastle where millions are being spent developing a new generation of welfare robots to replace humans. Private companies including a New York outfit led by the world's first bot billionaire, are supercharging a process which has spawned a whole new jargon: "virtual workforce", "augmented decision-making", "robot process automation".

The government is rushing forward with its digital mission despite the pain already being inflicted on millions of low-income Britons by the country's "digital by default" agenda. Claimants spoke of the hunger, filth, fear and panic that they are enduring.

Activists protesting against the Indian government's decision to link free lunch meals for children with the national Aadhaar biometric cards. Photograph: -/AFP/Getty Images

In Australia, where the Guardian has reported extensively on robodebt, the scheme that has been accused of wrongly clawing back historic debts through a flawed algorithm, we now disclose that the government has opened a new digital front: using automation to suspend millions of welfare payments. Recipients are finding their money cut off without notice.

The most disturbing story comes from Dumka in India. Here, we learn of the horrifying human impact that has befallen families as a result of Aadhaar, a 12-digit unique identification number that the Indian government has issued to all residents in the world's largest biometric experiment.

New high-tech approaches sweep through social services, work, disability and health, often with minimal public debate

Motka Manjhi paid the ultimate price when the computer glitched and his thumbprint - his key into Aadhaar - went unrecognised. His subsistence rations were stopped, he was forced to skip meals and he grew thin. On 22 May, he collapsed outside his home and died. His family is convinced it was starvation.

The Guardian investigations illuminate the shared features of these new systems, whether in developing or developed countries, east or west. The most glaring similarity is that all this is happening at lightning speed, with hi-tech approaches sweeping through social services, work and pensions, disability and health, often with minimal public debate or accountability.

Within that revolution, the human element of the welfare state is being diluted. Instead of talking to a caseworker who personally assesses your needs, you now are channeled online where predictive analytics will assign you a future risk score and an algorithm decide your fate.

In the new world, inequality and discrimination can be entrenched. What happens if you are one of the five million adults in the UK without regular access to the internet and with little or no computer literacy? What if the algorithm merely bakes in existing distortions of race and class, making the gulf between rich and poor, white and black, college-educated and manual worker, even more pronounced?

There is also a chilling Kafkaesque quality that spans the globe. As Manjhi so tragically discovered, mistakes are made. Machines glitch. If there is no one within reach who sees you as a person and not as a 12-digit number to be processed, the results can be fatal.

The computer says "No payments". Now what do you do?
Automating Poverty will run all week. If you have a story to tell about being on the receiving end of the new digital dystopia, email ed.pilkington@theguardian.com

# data[].attributes.attachment_link.title
Digital dystopia: how algorithms punish the poor

# data[].attributes.attachment_link.description
In an exclusive global series, the Guardian lays bare the tech revolution transforming the welfare system worldwide - while penalising the most vulnerable

# data[].attributes.attachment_link.title
K Means Clustering: Pros and Cons of K Means Clustering

# data[].attributes.body
First 25 mins very informative, may want to skip some sections but others really help explain how it works.

# data[].attributes.attachment_link.title
K Means Clustering Algorithm | K Means Clustering Example | Machine Learning Algorithms |Simplilearn

# data[].attributes.body
Nice

# data[].attributes.attachment_link.title
A* Pathfinding (E01: algorithm explanation)

# data[].attributes.attachment_link.readable_text
(Image credit: Getty Images)

Brain.js is a fantastic way to build a neural network. Simply put, a neural network is a method of machine learning that works in a similar way to a human brain. Given the correct answer to a question (like 'which option will this user choose?'), it slowly learns the pattern and relationship between the inputs and answers. One example of a neural network is Facebook's facial recognition system, DeepFace.

But due to the complex domain language of neural networks and seemingly steep learning curve, it can be hard to get started.

In this tutorial, we will distil the theory down to need-to-knows and, importantly, get stuck in with actually using brain.js to create a neural network. By the end, you will have a web app that asks multiple-choice questions about a user's optimism. When they submit, these answers will train a neural network to the probability of our user selecting each option for a brand new question.
Download the files you'll need for this tutorial.
01. Set up the project

Firstly, download and install the necessary dependencies. This tutorial presumes you have a working knowledge of React, or the equivalent mapping to a preferred alternative.

Create a React app using your desired method. You can try Facebook's create-react-app tool, installed using the following:

npm install create-react-app -g

02. Start your app

Now we can build, install brain.js, and start our app:

npx create-react-app optimism-nn cd optimism-nn npm install brainjs npm start

We are going to perform the neural network computation on the browser. Neural networks are resource intensive and should be offloaded to a server. However, this way is quick to set up and fine for our basic needs. Now let's add brain.js to our entry point (in my case, App.js).

import brain from 'brain.js';

03. Define your training questions

We need to define our training questions next. In a separate questions.js file, we'll need a trainingQuestions and validationQuestions array. You can find my list on the Git repo or create your own. The more training questions you have, the more accurate your results. Remember to import these into your entry point.

export const trainingQuestions = [ { id: 'q1', question: 'Do you often see the best in things?', options: [ { id: 'q1a', label: 'Not really', value: 0.2, }, { id: 'q1b', label: 'Always', value: 1.0, }, { id: 'q1c', label: 'Usually, yeah', value: 0.7, }, { id: 'q1d', label: 'Never!', value: 0.0, }, ], }, ];

For both arrays, we need a question, an array of four options that contain a label and an optimism value. This value will be the input for our neural network.

Ensure you vary the order and balance of values, or the neural network may focus too much on the index of the options in the array! Our neural network takes four inputs and gives four outputs. Our training data needs to match this, so in our constructor we need some state for the quiz and the user's options:

this.state = { trainingAnswers: trainingQuestions.map(() => Array(4).fill(0)), training: false, predictions: undefined, };

04. Initialise the neural network

The initialisation of trainingAnswers creates an array for each question containing [0, 0, 0, 0] - our default state with no selection. We're also going to need to initialise our neural network - just a single line with brain.js:

this.net = new brain.NeuralNetwork({ hiddenLayers: [4] });

05. Build the quiz framework

To build the framework for our quiz, we need to loop over our training questions and options. This is quite verbose and not very interesting, so I'll give an example output for you to aim for instead:

render() { return ( <main> <form onSubmit={this.onSubmit}> [. . .] // Iterate over questions & options <div className="question"> <h4>{question}</h4> <div className="options"> <label htmlFor={optionId}> <span>{label}</span> <input type="radio" required name={questionId} id={optionId} checked={() => this.isOptionChecked(questionIndex, optionIndex)} onChange={() => this.onOptionChange(questionIndex, optionIndex)} /> </label> [. . .] </div> </div> [. . .] <button type="submit">Submit</button> </form> </main> ); }

If you're new to React, see the documentation for building forms.

We can write our isOptionChecked and onOptionChange functions next:

isOptionChecked = (questionIndex, optionIndex) => ( this.state.trainingAnswers[questionIndex][optionIndex] !== 0 ); onOptionChange = (questionIndex, optionIndex) => { this.setState(prevState => { const { trainingAnswers } = Object.assign(prevState, {}); trainingAnswers[questionIndex] = Array(4).fill(0); trainingAnswers[questionIndex][optionIndex] = 1; return { trainingAnswers }; }); };

06. Train the neural network

Now, when our user clicks an option, we update the relevant trainingAnswers array to feature a 1 in the selected index and change the state of the radio button to show it as checked.

Time to add our onSubmit function, where we build the training data and train the neural network:

 onSubmit = e => { e.preventDefault(); const { trainingAnswers } = this.state; const trainingData = trainingQuestions.map((q, i) => ({ input: q.options.map(o => o.value), output: trainingAnswers[i], })); this.setState({ training: true, }); this.net.trainAsync(trainingData) .then(res => { console.log(res); // Log the error rate and # iterations this.getPredictions() }); }

Looping over trainingQuestions, we create the input and output arrays we need. We get the input data by taking the optimism value of each option and we get the output data from looking in the trainingAnswers array at the same index as the question.

After that, we update the state with training: true to inform the user that the neural network is learning. Depending on the processing power of the client device and how many questions you have, the process can take seconds, minutes or longer!

Finally, we pass the training data over to our neural network and tell it to train asynchronously. This returns a promise that is fulfilled when the network has found the pattern or given up.

Keep an eye on the error rate we log in trainAsync. Ideally it should be between 0 - 0.05. If it's higher, check your training data.

From there, we can get our predictions:

getPredictions = () => { const predictions = validationQuestions.map(q => ( this.net.run(q.options.map(o => o.value)) )); this.setState({ training: false, predictions, }); }

Using net.run, we ask our newly trained neural network to give us its predictions for each of the validation questions we defined earlier.

For the grand finale, we add our conditional loading logic and present a finding to the user.

render() { const { training, predictions } = this.state; const validationQuestion = validationQuestions[0]; return ( <main> {training && ( <h2>Loading...</h2> )} {!predictions && !training && ( [. . .] // Training questions form )} {predictions && !training && ( <div> <h2>We asked the neural network:</h2> <div className="question"> <h4>{validationQuestion.question}</h4> <div className="options"> {validationQuestion.options.map((o, i) => ( <label key={o.id}> {/* display the label and probability as a round percentage */} <span>{${o.label}: ${Math.round(predictions[0][i] * 100)}%}</span> </label> ))} </div> </div> </div> )} </main> ); } }

07. Extend the network

Now you have the basic framework for the quiz, try extending it with the following:

Find the real error rate of your neural network by letting your user answer your validation questions. See how many times they chose your best guess.

Train the neural network with these additional answers and see if you can improve the accuracy.

Move the neural network calculations over onto a Node server with the brain.js toFunction() and toJSON() methods.
This article originally appeared in issue 321 in Read more: net magazine, the world's leading web design magazine. Buy issue 321 or subscribe to net.

# data[].attributes.attachment_link.description
(Image credit: Getty Images) Brain.js is a fantastic way to build a neural network. Simply put, a neural network is a method of machine learning that works in a similar way to a human brain. Given the correct answer to a question (like 'which option will this user choose?'), it slowly learns the pattern and relationship between the inputs and answers.

# data[].attributes.attachment_link.readable_text
If you're a game developer, you might have always wanted to implement A* as character (or enemy) pathfinding. I know a couple of years ago I did, but with my level of programming at the time I had issues with the current articles out there. I wanted to write this as an easy introduction with a clear source code example to A* pathfinding so anyone can easily pick it up and use it in their game.

One important aspect of A* is f = g + h. The f, g, and h variables are in our Node class and get calculated every time we create a new node. Quickly I'll go over what these variables mean.

F is the total cost of the node.

G is the distance between the current node and the start node.

H is the heuristic - estimated distance from the current node to the end node.

Let's take a look at a quick graphic to help illustrate this.

wow such numbers, very color

Awesome! Let's say node(0) is our starting position and node(19) is our end position. Let's also say that our current node is at the the red square node(4).

G

G is the distance between the current node and the start node.

If we count back we can see that node(4) is 4 spaces away from our start node.

We can also say that G is 1 more than our parent node ( node(3)). So in this case for node(4), currentNode.g = 4.

H

H is the heuristic - estimated distance from the current node to the end node.

So let's calculate the distance. If we take a look we'll see that if we go over 7 spaces and up 3 spaces, we've reached our end node ( node(19)).

Let's apply the Pythagorean Theorem! a² + b² = c². After we've applied this, we'll see that currentNode.h = 7² + 3². Or currentNode.h = 58.
But don't we have to apply the square root to 58? Nope! We can skip that calculation on every node and still get the same output. Clever!
With a heuristic, we need to make sure that we can actually calculate it. It's also very important that the heuristic is always an underestimation of the total path, as an overestimation will lead to A* searching for through nodes that may not be the 'best' in terms of f value.

F

F is the total cost of the node.

So let's add up h and g to get the total cost of our node. currentNode.f = currentNode.g + currentNode.h. Or currentNode.f = 4+ 58. Or currentNode.f = 62.

Alright, so that was a lot of work. Now with all that work, what am I going to use this f value for?

With this new f value, we can look at all our nodes and say, "Hey, is this the best node I can pick to move forward with right now?". Rather than running through every node, we can pick the ones that have the best chance of getting us to our goal.

Here's a graphic to illustrate. On top, we have Dijkstra's Algorithm, which searches without this f value, and below we have A* which does use this f value.

Dijkstra's Algorithm (Wikipedia)
Dijkstra's Algorithm
A* Algorithm (Wikipedia)
A* Algorithm
So taking a look at Dijkstra's algorithm, we see that it just keeps searching. It has no idea which node is ' best', so it calculates paths for them all.

With A*,we see that once we get past the obstacle, the algorithm prioritizes the node with the lowest f and the 'best' chance of reaching the end.

# data[].attributes.attachment_link.description
If you're a game developer, you might have always wanted to implement A* as character (or enemy) pathfinding. I know a couple of years ago I did, but with my level of programming at the time I had issues with the current articles out there.

# data[].attributes.headline
Speech recognition using artificial neural networks and artificial bee colony optimization

# data[].attributes.subject
Speech recognition using artificial neural networks and artificial bee colony optimization

# data[].attributes.attachment_link.readable_text
October 7, 2019 by Ingrid Fadelli, Phys.org

Block diagram of proposed model. Credit: Shukla & Jain.

Over the past decade or so, advances in machine learning have paved the way for the development of increasingly advanced speech recognition tools. By analyzing audio files of human speech, these tools can learn to identify words and phrases in different languages, converting them into a machine-readable format.

While several machine learning-based models have achieved promising results on speech recognition tasks, they do not always perform well in all languages. For instance, when a language has a vocabulary with many similar-sounding words, the performance of speech recognition systems can decline considerably.

Researchers at Mahatma Gandhi Mission's College of Engineering & Technology and Jaypee Institute of Information Technology, in India, have developed a speech recognition system to tackle this problem. This new system, presented in a paper published in Springer Link's International Journal of Speech Technology, combines an artificial neural network (ANN) with an optimization technique known as opposition artificial bee colony (OABC).

"In this work, the default structure of ANNs is redesigned using the Levenberg-Marquardt algorithm to retrieve an optimal prediction rate with accuracy," the researchers wrote in their paper. "The hidden layers and neurons of the hidden layers are further optimized using the opposition artificial bee colony optimization technique."

A unique characteristic of the system developed by the researchers is that it uses an OABC optimization algorithm to optimize the ANN's layers and artificial neurons. As the name would suggest, artificial bee colony (ABC) algorithms are designed to simulate the behavior of honey bees to tackle a variety of optimization problems.

"Generally, optimization algorithms randomly initialize the solutions in the matching domain," the researchers explained in their paper. "But this solution could lie in the opposite direction of the best solution, thereby increasing the computational overhead significantly. Hence this opposition-based initialization is termed as OABC."

The system developed by the researchers considers individual words spoken by different people as an input speech signal. Subsequently, it extracts so-called amplitude modulation (AM) spectrogram features, which are essentially sound-specific characteristics.

The features extracted by the model are then used to train the ANN to recognize human speech. After it is trained on a large database of audio files, the ANN learns to predict isolated words in new samples of human speech.

The researchers tested their system on a series of human speech audio clips and compared it with more conventional speech recognition techniques. Their technique outperformed all the other methods, attaining remarkable accuracy scores.

"The sensitivity, specificity, and accuracy of the proposed method are 90.41 percent, 99.66 percent and 99.36 percent, respectively, which is better than all the existing methods," the researchers wrote in their paper.

In the future, the speech recognition system could be used to achieve more effective human-machine communication in a variety of settings. In addition, the approach they used to develop the system could inspire other teams to design similar models, which combine ANNs and OABC optimization techniques.

More information: Shilpi Shukla et al. A novel system for effective speech recognition based on artificial neural network and opposition artificial bee colony algorithm, International Journal of Speech Technology (2019). DOI: 10.1007/s10772-019-09639-0

© 2019 Science X Network

# data[].attributes.attachment_link.keywords[]
electronics and technology advances news

# data[].attributes.attachment_link.title
Speech recognition using artificial neural networks and artificial bee colony optimization

# data[].attributes.attachment_link.description
October 7, 2019 by Ingrid Fadelli, Phys.org Over the past decade or so, advances in machine learning have paved the way for the development of increasingly advanced speech recognition tools. By analyzing audio files of human speech, these tools can learn to identify words and phrases in different languages, converting them into a machine-readable format.

# data[].attributes.headline
Video: Building a Simple Neural Network in Excel

# data[].attributes.subject
Video: Building a Simple Neural Network in Excel

# data[].attributes.attachment_link.title
Building a simple neural network in EXCEL

# data[].attributes.headline
Video: Training a single Neurone in Excel

# data[].attributes.subject
Video: Training a single Neurone in Excel

# data[].attributes.attachment_link.title
Artificial Neural Network - Training a single Neuron using Excel

# data[].attributes.body
Online Editor for writing Prolog code

# data[].attributes.attachment_link.readable_text
This is second in a two-post series on building a neural network using microbits with micropython. In the first post python was used to produce a neural network without the microbits. In this post the network is as shown in figure 1 is developed.
The figure below shows the arrangement of the connections to be built; pin 2 is the output of each neuron. The two micro:bits/neurons on the left of the picture taking in the two same inputs ; the output from these neurons are the two inputs to the output neuron on the right.

All opinions in this blog are the Author's and should not in any way be seen as reflecting the views of any organisation the Author has any association with. Twitter @scottturneruon

# data[].attributes.attachment_link.title
Produce a Microbit python neural network 2: Building a Physical Microbit Neural Network

# data[].attributes.attachment_link.description
This is second in a two-post series on building a neural network using microbits with micropython. In the first post python was used to produce a neural network without the microbits. In this post the network is as shown in figure 1 is developed.