# data[].attributes.body
It can process and learn from much more data than previous approaches.

Why is AI so relevant now?
1. Compute power
2. Data availability
3. Lower cost

Extra sources: Andrew Ng's Deep Learning course chart:

# data[].attributes.body
"The opportunities are endless, as massive changes in methods can be brought to life by any unique mind, debates on AI and data ethics will continue, and businesses will rely more and more on these methods as their most valuable resource. Taking the time to understand where we came from and where we are going can allow everyone to develop their own vision of the future. The global matrix of these unique human visions is what will lead us into a bright future with AI at our side." Source: https://towardsdatascience.com/the-state-of-ai-in-2020-1f95df336eb0

# data[].attributes.attachment_link.description
Our 2020 AI survey shows artificial intelligence has become a revenue driver and companies earning the most from AI plan to invest more in response to COVID-19.

# data[].attributes.body
Artificial neural networks (ANNs) are comprised of node layers, containing an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network.

# data[].attributes.attachment_link.readable_text
Neural networks reflect the behavior of the human brain, allowing computer programs to recognize patterns and solve common problems in the fields of AI, machine learning, and deep learning.

What are neural networks?

Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks (SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. Their name and structure are inspired by the human brain, mimicking the way that biological neurons signal to one another.

Artificial neural networks (ANNs) are comprised of a node layers, containing an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network.

Neural networks rely on training data to learn and improve their accuracy over time. However, once these learning algorithms are fine-tuned for accuracy, they are powerful tools in computer science and artificial intelligence, allowing us to classify and cluster data at a high velocity. Tasks in speech recognition or image recognition can take minutes versus hours when compared to the manual identification by human experts. One of the most well-known neural networks is Google's search algorithm.

How do neural networks work?

Think of each individual node as its own linear regression model, composed of input data, weights, a bias (or threshold), and an output. The formula would look something like this:

∑wixi + bias = w1x1 + w2x2 + w3x3 + bias

output = f(x) = 1 if ∑w1x1 + b>= 0; 0 if ∑w1x1 + b < 0

Once an input layer is determined, weights are assigned. These weights help determine the importance of any given variable, with larger ones contributing more significantly to the output compared to other inputs. All inputs are then multiplied by their respective weights and then summed. Afterward, the output is passed through an activation function, which determines the output. If that output exceeds a given threshold, it "fires" (or activates) the node, passing data to the next layer in the network. This results in the output of one node becoming in the input of the next node. This process of passing data from one layer to the next layer defines this neural network as a feedforward network.

Let's break down what one single node might look like using binary values. We can apply this concept to a more tangible example, like whether you should go surfing (Yes: 1, No: 0). The decision to go or not to go is our predicted outcome, or y-hat. Let's assume that there are three factors influencing your decision-making:

1. Are the waves good? (Yes: 1, No: 0)

2. Is the line-up empty? (Yes: 1, No: 0)

3. Has there been a recent shark attack? (Yes: 0, No: 1)

Then, let's assume the following, giving us the following inputs:

X1 = 1, since the waves are pumping

X2 = 0, since the crowds are out

X3 = 1, since there hasn't been a recent shark attack

Now, we need to assign some weights to determine importance. Larger weights signify that particular variables are of greater importance to the decision or outcome.

W1 = 5, since large swells don't come around often

W2 = 2, since you're used to the crowds

W3 = 4, since you have a fear of sharks

Finally, we'll also assume a threshold value of 3, which would translate to a bias value of -3. With all the various inputs, we can start to plug in values into the formula to get the desired output.

Y-hat = (1*5) + (0*2) + (1*4) - 3 = 6

If we use the activation function from the beginning of this section, we can determine that the output of this node would be 1, since 6 is greater than 0. In this instance, you would go surfing; but if we adjust the weights or the threshold, we can achieve different outcomes from the model. When we observe one decision, like in the above example, we can see how a neural network could make increasingly complex decisions depending on the output of previous decisions or layers.

In the example above, we used perceptrons to illustrate some of the mathematics at play here, but neural networks leverage sigmoid neurons, which are distinguished by having values between 0 and 1. Since neural networks behave similarly to decision trees, cascading data from one node to another, having x values between 0 and 1 will reduce the impact of any given change of a single variable on the output of any given node, and subsequently, the output of the neural network.

As we start to think about more practical use cases for neural networks, like image recognition or classification, we'll leverage supervised learning, or labeled datasets, to train the algorithm. As we train the model, we'll want to evaluate its accuracy using a cost (or loss) function. This is also commonly referred to as the mean squared error (MSE). In the equation below,

i represents the index of the sample,

y-hat is the predicted outcome,

y is the actual value, and

m is the number of samples.

𝐶𝑜𝑠𝑡 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛= 𝑀𝑆𝐸=1/2𝑚 ∑129_(𝑖=1)^𝑚▒(𝑦 ̂^((𝑖) )−𝑦^((𝑖) ) )^2

Ultimately, the goal is to minimize our cost function to ensure correctness of fit for any given observation. As the model adjusts its weights and bias, it uses the cost function and reinforcement learning to reach the point of convergence, or the local minimum. The process in which the algorithm adjusts its weights is through gradient descent, allowing the model to determine the direction to take to reduce errors (or minimize the cost function). With each training example, the parameters of the model adjust to gradually converge at the minimum.

See this IBM Developer article for a deeper explanation of the quantitative concepts involved in neural networks.

Most deep neural networks are feedforward, meaning they flow in one direction only, from input to output. However, you can also train your model through backpropagation; that is, move in the opposite direction from output to input. Backpropagation allows us to calculate and attribute the error associated with each neuron, allowing us to adjust and fit the parameters of the model(s) appropriately.

Types of neural networks

Neural networks can be classified into different types, which are used for different purposes. While this isn't a comprehensive list of types, the below would be representative of the most common types of neural networks that you'll come across for its common use cases:

The perceptron is the oldest neural network, created by Frank Rosenblatt in 1958. It has a single neuron and is the simplest form of a neural network:

Feedforward neural networks, or multi-layer perceptrons (MLPs), are what we've primarily been focusing on within this article. They are comprised of an input layer, a hidden layer or layers, and an output layer. While these neural networks are also commonly referred to as MLPs, it's important to note that they are actually comprised of sigmoid neurons, not perceptrons, as most real-world problems are nonlinear. Data usually is fed into these models to train them, and they are the foundation for computer vision, natural language processing, and other neural networks.

Convolutional neural networks (CNNs) are similar to feedforward networks, but they're usually utilized for image recognition, pattern recognition, and/or computer vision. These networks harness principles from linear algebra, particularly matrix multiplication, to identify patterns within an image.

Recurrent neural networks (RNNs) are identified by their feedback loops. These learning algorithms are primarily leveraged when using time-series data to make predictions about future outcomes, such as stock market predictions or sales forecasting.

Neural networks vs. deep learning

Deep Learning and neural networks tend to be used interchangeably in conversation, which can be confusing. As a result, it's worth noting that the "deep" in deep learning is just referring to the depth of layers in a neural network. A neural network that consists of more than three layers-which would be inclusive of the inputs and the output-can be considered a deep learning algorithm. A neural network that only has two or three layers is just a basic neural network.

To learn more about the differences between neural networks and other forms of artificial intelligence, like machine learning, please read the blog post " AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What's the Difference? "

History of neural networks

The history of neural networks is longer than most people think. While the idea of "a machine that thinks" can be traced to the Ancient Greeks, we'll focus on the key events that led to the evolution of thinking around neural networks, which has ebbed and flowed in popularity over the years:

1943: Warren S. McCulloch and Walter Pitts published " A logical calculus of the ideas immanent in nervous activity (PDF, 1 MB) (link resides outside IBM)" This research sought to understand how the human brain could produce complex patterns through connected brain cells, or neurons. One of the main ideas that came out of this work was the comparison of neurons with a binary threshold to Boolean logic (i.e., 0/1 or true/false statements).

1958: Frank Rosenblatt is credited with the development of the perceptron, documented in his research, " The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain " (PDF, 1.6 MB) (link resides outside IBM). He takes McCulloch and Pitt's work a step further by introducing weights to the equation. Leveraging an IBM 704, Rosenblatt was able to get a computer to learn how to distinguish cards marked on the left vs. cards marked on the right.

1974: While numerous researchers contributed to the idea of backpropagation, Paul Werbos was the first person in the US to note its application within neural networks within his PhD thesis (PDF, 8.1 MB) (link resides outside IBM).

1989: Yann LeCun published a paper (PDF, 5.7 MB) (link resides outside IBM) illustrating how the use of constraints in backpropagation and its integration into the neural network architecture can be used to train algorithms. This research successfully leveraged a neural network to recognize hand-written zip code digits provided by the U.S. Postal Service.

Neural networks and IBM Cloud

For decades now, IBM has been a pioneer in the development of AI technologies and neural networks, highlighted by the development and evolution of IBM Watson. Watson is now a trusted solution for enterprises looking to apply advanced natural language processing and deep learning techniques to their systems using a proven tiered approach to AI adoption and implementation.

Watson uses the Apache Unstructured Information Management Architecture (UIMA) framework and IBM's DeepQA software to make powerful deep learning capabilities available to applications. Utilizing tools like IBM Watson Studio, your enterprise can seamlessly bring open source AI projects into production while deploying and running models on any cloud.

For more information on how to get started with deep learning technology, explore IBM Watson Studio and the Deep Learning service.

Sign up for an IBMid and create your IBM Cloud account.

# data[].attributes.attachment_link.description
Learn about neural networks that allow programs to recognize patterns and solve common problems in artificial intelligence, machine learning and deep learning.

# data[].attributes.body
What is ML?

"Computer algorithms that improve automatically trough experience." - Tom Mitchel

ML Techniques:
* Supervised Learning:
Maps an input to an output. It uses a training set to teach models to yield the desired output. This training dataset includes inputs and correct outputs.
* Unsupervised Learning:
The training set is unlabeled.  These algorithms discover hidden patterns or data groupings without the need for human intervention
* Reinforcement Learning
The learning system (agent) observes the context, selects and performs actions based on the possibility of gaining rewards (or penalties, i.e., negatives rewards). It learns by itself what is the best strategy, called policy, to get the most reward over time.

Extra sources:
Check this Mindmap:

# data[].attributes.attachment_link.title
Chapter 1: The Machine Learning Landscape

# data[].attributes.headline
4. Industry applications and Affecting industry

# data[].attributes.subject
4. Industry applications and Affecting industry

# data[].attributes.body
Sectors with great potential:

* Retail: pricing, promotion, and customer service management
*Consumer goods: supply-chain management, demand forecasting
*Finance: marketing and sales, assessing and managing risks.

Example: Blue River technology - using computer vision, their tractors can only spread herbicide on weeds, and not the crops.

Extra sources:

# data[].attributes.attachment_link.title
28 Examples of Artificial Intelligence Shaking Up Business as Usual

# data[].attributes.attachment_link.description
These artificial intelligence examples are transforming a number of industries, including robotics, healthcare, transportation, finance and more.

# data[].attributes.body
AI integrates perception, prediction, and decision-making

# data[].attributes.body
AI: "The science and engineering of making computers behave in ways that, until recently, we thought required human intelligence." Andrew Moore.

Applications: Face recognition, text recognition, speech recognition, computer vision, recognition of mechanical failures, NPL, etc.

Extra sources:

# data[].attributes.attachment_link.title
Artificial Intelligence (AI): What it is and why it matters

# data[].attributes.attachment_link.description
With artificial intelligence (AI), machines learn from experience and perform human-like tasks. AI works by combining vast amounts of data with fast, iterative processing and intelligent algorithms. Learn more in our primer.