# data[].attributes.body
couldn't connect to anything :(

# data[].attributes.body
I got my server working and Fernando connected to my laptop

# data[].attributes.body
I am trying to implement a fraud detection

# data[].attributes.body
I am successfully reading transactions from your host! I need to read it to a csv, but I am leaving class with no errors which is a huge win for me

# data[].attributes.body
I was able to connect to Soren's server

# data[].attributes.body
Did client & server locally, connected to Artzi, and connected to Artzi's to test fraud_collector.py by using DataStreamVisualClient.

# data[].attributes.headline
Carter Slaton - Data Stream Progress 9-18-2025

# data[].attributes.subject
Carter Slaton - Data Stream Progress 9-18-2025

# data[].attributes.body
Running Stream visual, with console on the side showing connection to the professor's server

# data[].attributes.body
I am sorry I am late to posting. I thought I was close to getting it to work but little did I know I would spend a couple hours debugging and forget to post it. However, it is now working!

# data[].attributes.attachment_link.description
Contribute to isac-artzi/AIT-204-ANN-Data-Stream development by creating an account on GitHub.

# data[].attributes.body
https://thegrio.com/2025/09/15/black-student-found-hanging-from-tree-on-at-delta-state-university/

# data[].attributes.body
article: https://www.nbcnews.com/news/us-news/man-scuba-gear-suspected-robbing-disney-springs-restaurant-rcna231686

# data[].attributes.body
I am having an issue with training the model on Streamlit. Will fix next class

source: https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html

# data[].attributes.body
I made my own article to test the file.

# data[].attributes.body
I used a prompt about a graphics card.

# data[].attributes.body
The 20 Newsgroups classifier is a feedforward neural network with two hidden layers of 512 and 256 neurons, using ReLU activations and 0.3 dropout to reduce overfitting. It takes 5000-dimensional TF-IDF features as input and outputs 20 classes, trained with CrossEntropyLoss and the Adam optimizer. On test data, it achieves around 72% accuracy, performing well overall but confusing some similar topics.

# data[].attributes.body
Unable to upload to Streamlit at the current time.

Article used: Computer | Definition, History, Operating Systems, & Facts | Britannica

# data[].attributes.body
Article used: https://time.com/7317021/trump-sanctions-russia-ukraine-oil/

# data[].attributes.body
Article used: https://apnews.com/article/israel-gaza-genocide-palestinians-c9d40ab3714b46957c5716132f9eb2a6

# data[].attributes.body
Not shown, but the PyTorch+Streamlit repository was cloned, currently trying to set up my laptop to utilize the GPU for training

# data[].attributes.body
Article used: After Sun article, Industry minister promises to delist Samidoun as not-for-profit

Ran the application on an app, have to figure out errors: 20 Newsgroups Classifier · Streamlit

# data[].attributes.attachment_link.readable_text
# text_classification_20newsgroups_pytorch_skeleton.py
# Purpose: TF-IDF + PyTorch MLP for 20 Newsgroups (STRUCTURE-ONLY SKELETON)

import os
import random
import numpy as np

# ---- Reproducibility (optional but recommended) ----
SEED = 42
os.environ["PYTHONHASHSEED"] = str(SEED)
random.seed(SEED)
np.random.seed(SEED)

import torch
torch.manual_seed(SEED)
torch.cuda.manual_seed_all(SEED)

from sklearn.datasets import fetch_20newsgroups
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer

# =========================
# Load Dataset
# =========================
data = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes'))
X_raw, y = data.data, data.target
num_classes = len(data.target_names)

# =========================
# TODO: Preprocess text data
#  - Convert to lowercase, remove punctuation if desired
#  - Tokenization is handled by TfidfVectorizer, but you can add custom steps if needed
# =========================

# =========================
# Convert Text Data to Numerical Format
# =========================
vectorizer = TfidfVectorizer(
    max_features=5000,          # Limit to 5000 most frequent words
    lowercase=True,
    stop_words='english',
    strip_accents='unicode',
    token_pattern=r"(?u)\b[a-zA-Z][a-zA-Z]+\b"
)
X_vec = vectorizer.fit_transform(X_raw)
X_vec = X_vec.toarray()         # NOTE: densifies; OK at 5k features, but watch RAM on small machines

# =========================
# Split data
# =========================
X_train, X_test, y_train, y_test = train_test_split(
    X_vec, y, test_size=0.2, stratify=y, random_state=SEED
)

# =========================
# Torch Tensors & Dataloaders
# =========================
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

X_train_t = torch.tensor(X_train, dtype=torch.float32)
y_train_t = torch.tensor(y_train, dtype=torch.long)
X_test_t  = torch.tensor(X_test,  dtype=torch.float32)
y_test_t  = torch.tensor(y_test,  dtype=torch.long)

from torch.utils.data import TensorDataset, DataLoader

train_ds = TensorDataset(X_train_t, y_train_t)
test_ds  = TensorDataset(X_test_t,  y_test_t)

# TODO: Tune batch_size
train_loader = DataLoader(train_ds, batch_size=64, shuffle=True,  drop_last=False)
test_loader  = DataLoader(test_ds,  batch_size=256, shuffle=False, drop_last=False)

# =========================
# Design Neural Network Architecture
# =========================
import torch.nn as nn

class NewsMLP(nn.Module):
    def __init__(self, input_dim, num_classes):
        super().__init__()
        # TODO: Define layers
        # Example (commented): 
        # self.net = nn.Sequential(
        #     nn.Linear(input_dim, 512),
        #     nn.ReLU(),
        #     nn.Dropout(0.3),
        #     nn.Linear(512, 256),
        #     nn.ReLU(),
        #     nn.Dropout(0.3),
        #     nn.Linear(256, num_classes)
        # )
        self.net = nn.Identity()  # <-- replace with real layers

    def forward(self, x):
        # TODO: Forward pass using your defined layers
        return self.net(x)

input_dim = X_train_t.shape[1]
model = NewsMLP(input_dim=input_dim, num_classes=num_classes).to(device)

# =========================
# Compile the Model (PyTorch-style setup)
# =========================
# TODO: Choose optimizer, loss, (optional) scheduler
# Example (commented):
# optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
# criterion = nn.CrossEntropyLoss()
optimizer = None     # <-- replace
criterion = None     # <-- replace
scheduler = None     # <-- optional

# =========================
# Train the Model
# =========================
# TODO: Implement the training loop
# - model.train()
# - for each batch: zero_grad ��� forward ��� compute loss ��� backward ��� optimizer.step()
# - (optional) track running loss/accuracy, use scheduler.step(validation_metric) if you add validation
def train(num_epochs=10):
    assert optimizer is not None and criterion is not None, "Set optimizer/criterion before training."
    for epoch in range(num_epochs):
        model.train()
        # running_loss, running_correct, total = 0.0, 0, 0
        for xb, yb in train_loader:
            xb, yb = xb.to(device), yb.to(device)

            # optimizer.zero_grad()
            # logits = model(xb)
            # loss = criterion(logits, yb)
            # loss.backward()
            # optimizer.step()

            # # (optional) metrics
            # preds = logits.argmax(dim=1)
            # running_loss += loss.item() * xb.size(0)
            # running_correct += (preds == yb).sum().item()
            # total += xb.size(0)

        # epoch_loss = running_loss / max(1, total)
        # epoch_acc  = running_correct / max(1, total)
        # print(f"Epoch {epoch+1}/{num_epochs} | loss={epoch_loss:.4f} acc={epoch_acc:.4f}")

        # if scheduler is not None:
            # scheduler.step(epoch_loss)  # or any metric you prefer

# =========================
# Evaluate the Model
# =========================
# TODO: Implement evaluation on test data
# - model.eval()
# - with torch.no_grad(): forward on test_loader, compute accuracy & classification report if desired
def evaluate():
    model.eval()
    # all_preds, all_targets = [], []
    with torch.no_grad():
        for xb, yb in test_loader:
            xb, yb = xb.to(device), yb.to(device)
            # logits = model(xb)
            # preds = logits.argmax(dim=1)
            # all_preds.append(preds.cpu().numpy())
            # all_targets.append(yb.cpu().numpy())

    # import numpy as np
    # y_pred = np.concatenate(all_preds)
    # y_true = np.concatenate(all_targets)

    # from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
    # print("Test Accuracy:", accuracy_score(y_true, y_pred))
    # print(classification_report(y_true, y_pred, target_names=data.target_names))
    # print("Confusion Matrix:\n", confusion_matrix(y_true, y_pred))

if __name__ == "__main__":
    # ===== Fill these before running =====
    # optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
    # criterion = nn.CrossEntropyLoss()
    # scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode="min", factor=0.5, patience=2)

    # train(num_epochs=10)
    # evaluate()
    pass

# data[].attributes.body
I finally got my github to work properly.

# data[].attributes.body
My app for now is having errors due to missing imports

# data[].attributes.body
Post a screenshot of your ANN app deployed on Streamlit Cloud

# data[].attributes.attachment_link.description
Contribute to isac-artzi/AIT-204-ANN development by creating an account on GitHub.

# data[].attributes.body
Load Model

It starts by loading a previously trained neural network model from the file mnist_model.h5. This model is designed to classify handwritten digits.

Preprocess Image

It defines a function to prepare the input image. This function reads the image in grayscale, inverts its colors (since MNIST models are typically trained on white digits on a black background), resizes it to the required 28x28 pixels, and normalizes the pixel values.

Predict

The script processes your specified image file (e.g., path_to_your_image.jpg) using the function from the previous step. It then feeds this processed image into the loaded model to get a prediction.

Display Result

Finally, it uses Matplotlib to display the original image along with the digit the model predicted as the title.

# data[].attributes.body
Loads and preprocesses the MNIST dataset: normalizes image pixel values to [0,1] and converts digit labels into one-hot encoded vectors.

Builds a simple neural network:

Input layer flattens 28x28 images into a 1D vector.

Hidden dense layer with 128 neurons using ReLU activation.

Output layer with 10 neurons using softmax for digit classification (0–9).

Compiles and trains the model: uses the Adam optimizer, categorical cross-entropy loss, and tracks accuracy over 10 epochs.

Evaluates model performance on the test dataset and prints the test accuracy.

Saves the trained model to mnist_model.h5 for future use.

Generates predictions for the test images and compares them to true labels.

Visualizes predictions: plots a sample of test images with predicted labels (blue if correct, red if incorrect).

# data[].attributes.body
The program relies on two Python files to compile and run the model.

ANN.py:

Imports all required libraries.

Loads the MNIST dataset into usable training and testing images and labels.

Normalizes the images as part of preprocessing and converts the labels into one-hot encoded format.

Builds a neural network model using the Sequential API. The images are flattened into 1-D arrays and passed through two activation functions: ReLU and Softmax. ReLU corresponds to a fully connected layer with 128 neurons, while Softmax serves as the output layer for the 10 digit classes (0–9).

Compiles and trains the model with a batch size of 32 for 10 epochs.

Evaluates the model for validation accuracy and then saves it.

Finally, sets up a plot displaying 15 sample test images along with the model’s predictions.

Running the Model / Making Predictions:

Similar to ANN.py, but skips the building and compiling steps.

Imports the necessary libraries.

Loads the pre-trained model.

Takes a user-provided image (specified by a path), preprocesses it with computer vision techniques, and formats it to match the training data requirements.

Uses the model to predict the digit and displays the result using Matplotlib.

# data[].attributes.body
Data Prep

Load MNIST dataset (28×28 grayscale images of digits 0–9).

Normalize pixel values (0–255 → 0–1).

One-hot encode labels (e.g., 7 → [0,0,0,0,0,0,0,1,0,0]).

Model

Input: Flatten 28×28 → 784 values.

Hidden Layer: 128 neurons, ReLU activation.

Output Layer: 10 neurons, Softmax activation (digit probabilities).

Training

Optimizer: Adam.

Loss: Categorical cross-entropy.

Runs for 10 epochs, batch size 32.

Evaluation & Results

Tests accuracy on the test set.

Saves model (mnist_model.h5).

Predicts on test images.

Shows results: blue = correct, red = wrong.

Concepts Demonstrated:

Feed-forward neural network.

Supervised learning (classification).

Batch training.

Activation functions (ReLU, Softmax).

# data[].attributes.body
The model is trained to recognize single handwritten digits by learning patterns in the training images.

Input images are prepared first: resized to 28x28 pixels, turned into grayscale, and normalized so the numbers are easier for the model to process.

After training, the model can take in a new handwritten digit and predict the number it is.

To handle multi-digit numbers the image is split in to seperate digit sections.

The model is applied to each section to predict each digit one by one.

# data[].attributes.body
Imports libraries

Provide citations for MNIST, KMMIST, EMNIST, etc

Each dataset is represented as a class

Data are splitted into train & test sets

Read images and label files, use functions like extract_mnist_images to decode them into NumPy arrays

Helper functions read image files, skip the header, and reshape

# data[].attributes.body
• Takes a photo of 4 handwritten digits and automatically recognizes what numbers you wrote

• Splits the image into separate digits, then uses the MNIST model to identify each one

• Combines the results to give you the complete 4-digit number

• Shows confidence scores so you know how sure it is about each digit

• Works best with clear handwriting and good spacing between digits

• Added visualization tools to see how it found and processed each digit

# data[].attributes.body
The code creates a 4-digit handwritten number recognition system that can take a photo of handwritten numbers and convert them into digital text.

How It Works

Takes Input: provide an image file containing a handwritten 4-digit number (like 1234, 5678, etc.)

Cleans the Image:

Converts to black and white

Removes noise and blur

Makes the image clear for processing

Finds Individual Digits:

Scans the image to locate each digit

Separates them into 4 individual digit images

Sorts them left-to-right (so 1234 stays in order, not 3142)

Recognizes Each Digit:

Uses a trained MNIST neural network model

Processes each digit through the AI model

Gets a prediction (0-9) and confidence score for each

Combines Results:

Puts the 4 individual predictions together

Creates the final 4-digit number

Calculates overall confidence

Shows Results:

Displays the recognized number

Shows visual proof with bounding boxes around detected digits

Provides confidence scores for reliability

# data[].attributes.body
Load an image containing a 4-digit number.

Preprocess the image to make the digits easy to find (e.g., convert to black and white).

Use contour detection to find the outline of each individual digit.

Sort these detected digits from left to right to maintain the correct numerical order.

For each isolated digit, apply the preprocessing logic from predict.py (resize to 28x28, normalize, etc.).

Feed each preprocessed digit into the loaded MNIST model to get a prediction.

Combine the four individual predictions to form the final 4-digit number.

# data[].attributes.body
Model Architecture: A simple feedforward neural network (MLP)

The current model recognizes a single digit (0-9) from a 28x28 grayscale image using a pre-trained MNIST model.

To recognize 4-digit numbers, the solution must:

Segment a single input image containing four adjacent digits into four separate 28x28 digit regions.

Process each segmented region through the existing digit recognition pipeline.

Combine individual digit predictions into a single 4-digit number.

The solution requires adding a new segmentation module (not present in current code) before prediction.

Output must be a string of four digits (e.g., "1234"), not individual predictions.

# data[].attributes.body
Summary

Trained MNIST digit classifier and saved as mnist_model.h5

For 4-digit recognition, I generated a strip of digits: digits_3219.png. Then I split the image into exact 28x28 tiles so that it would match the training

The model predicted the sequence: 3219 with high confidence for each of the digits

Errors came from segmentation/preprocessing but using exact tiling fixed the issue

# data[].attributes.body
Preliminary understanding of solution

Training an MNIST digit classifier and save the model

Load an image that has four handwritten digits in a row

Convert it to grayscale

Extract the four digits

For each digit center 28x28 to classify 0-9 with model

Order the predictions from left to right and join them together

# data[].attributes.body
Implements a preprocessing function to converts the images to match the MNIST training data format

Displays the original image the provide feedback on the model's classification performance

Classifies single digits by adjusting the weights of 0-9

Flips image colors since MNIST digits are on white/light on dark backgrounds

# data[].attributes.body
The program is dependent on the two python files to be able to compile and run the model.

The ANN.py:

Imports all necessary libraries.

Loads MNIST dataset into usable training and testing images and labels.

The images are normalized as this serves as the preprocessing section, as well as converting labels to one-hot ended formats.

Then a neural network model is created, as using a Sequential format. As the images are computed into 1-D array, then passed through 2 activation functions of ReLU and Softmax. ReLU is the fully connected layer of 128 neurons, and following Softmax being the output layer for the 10 classes (0-9).

Then the model is complied and then trained with a batch size of 32 and 10 epochs.

Then the model is evaluated for validation accuracy and saved.

The rest of code sets up a plot with 15 sample test images, running the model's predictions.

Running the model / Making Predictions:

This code summarized follows the ANN.py similarly without the building and compiling it.

Imports nessessary libraries

The model is loaded.

A personal image needs to intilaized as a apth. That image is preproceesed using computer vision, the formatting is the same as it was in the training to fit the model.

The model predict the image also providing the predicted output, on a matplotlib.

# data[].attributes.body
The computer learns to recognize handwritten numbers by practicing with lots of examples of single digits (0-9)

It adjusts its internal settings until it gets good at identifying what each digit looks like, then when you show it a new handwritten digit, it can guess what number it is

For longer numbers like "1234," it splits the image into separate pieces and identifies each digit one by one

It then combines all the individual digit guesses to read the complete number

# data[].attributes.body
The code begins with loading the MNIST dataset with the 60,000 training data (images + label) and the 10,000 testing data (images + label) as two separate sets of pairs

Since RGB values scale from 0-255, dividing by 255 simplifies the RGB values of each image to 0-1

For the model:

Flatten breaks down the 28x28 images into each pixel’s data, such that it forms a 1D array

The initial Dense specifies the hidden layers with an activation function ReLu, which outputs 0 or the max/input value

The second Dense specifies the output layer such that there are 10 classifications (digits 0-9) with the softmax activation function (which keeps the values between 0 to 1)

The use of the softmax activation function for each output classification helps determine if the classification cannot be determined (in the middle: 0, 0.5), is likely not (closer to -1), and likely is (closer to 1)

# data[].attributes.body
This program:

imports the data

cleans and encodes the data

builds the neural network model (flattens images to 1d array)

makes predictions using the model and given images on what number appears to be drawn.

I imagine what happens behind the scenes is there are mathematical functions to simulate numbers, for example the number one would be x = 2, and then the algorithm takes points on the drawing, plots them, and then calculates which number the data best fits.

# data[].attributes.body
The trained model learns to classify single digits by adjusting weights during training.

Input images are preprocessed (resized, normalized, and reshaped) to match the MNIST input format.

Once trained, the model can predict digits from new handwritten images.

The task extends beyond recognizing a single digit to recognizing a sequence of digits (e.g., 4-digit numbers) by segmenting the image into multiple digit regions and applying the digit recognizer on each segment.

The final output is the predicted multi-digit number, constructed by combining predictions from each digit in sequence.

# data[].attributes.attachment_link.description
Contribute to isac-artzi/AIT-204-ANN-Data-Stream development by creating an account on GitHub.

# data[].attributes.body
Real-World Mimicry: Streaming data more closely mimics real-world scenarios where data often arrives in a sequential manner, such as financial transactions, sensor readings, or social media posts. This allows the model to be tested and refined in an environment similar to its intended use.

Adaptability: With streaming data, an ANN can continually learn and adapt to new patterns or changes in the data distribution, making the model more robust and applicable to dynamically changing environments.

Resource Efficiency: Processing data as it arrives (online learning) can be more memory-efficient than loading a large static dataset into memory, especially critical for systems with limited resources.

Timely Decision-Making: Streaming data allows the ANN to make real-time predictions and decisions. This is crucial in scenarios like fraud detection or real-time analytics, where delayed decision-making can have significant consequences.

Continuous Improvement: As the model is exposed to more data over time, it has the potential to improve its performance, unlike a model trained on a static dataset which doesn't evolve after its initial training.

Data Variability: Streaming data often encompasses a broader range of data variability and different scenarios, providing a more comprehensive training environment for the ANN.

Immediate Feedback: With streaming data, the model can potentially receive immediate feedback on its predictions (depending on the application), allowing for faster and more targeted adjustments to the model.

# data[].attributes.body
from sklearn.metrics import confusion_matrix, accuracy_score, precision_score, recall_score, f1_score

# Example data

y_true = [1, 0, 1, 1, 0, 1]
y_pred = [1, 0, 1, 0, 0, 1]

# Generate confusion matrix
cm = confusion_matrix(y_true, y_pred)
print("Confusion Matrix:")
print(cm)

# Calculate metrics
accuracy = accuracy_score(y_true, y_pred)
precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)

print("\nMetrics:")
print(f"Accuracy: {accuracy}")
print(f"Precision: {precision}")
print(f"Recall: {recall}")
print(f"F1-Score: {f1}")

# data[].attributes.body
A confusion matrix is a table that is used to evaluate the performance of a classification model. It compares the actual vs. predicted classifications.

True Positive (TP): Correctly predicted positive cases.

True Negative (TN): Correctly predicted negative cases.

False Positive (FP): Incorrectly predicted positive cases.

False Negative (FN): Incorrectly predicted negative cases.

# data[].attributes.body
Data Preprocessing:

Load the Dataset: Use pandas or a similar library to load the dataset.

Text Cleaning: For each review in "Review_Text", convert the text to lowercase, remove punctuation, and tokenize. This normalization helps in treating words uniformly.

Reason: Neural networks work best with normalized and numerical input. Tokenization helps in converting text into numbers.

Label Encoding: Convert category labels in the "Category" column to integers (e.g., "Electronics" to 0, "Clothing" to 1, etc.).

Reason: Neural networks cannot operate on string labels. They require numerical values as input and target output.

Feature Extraction:

TF-IDF Vectorization: Convert tokenized reviews into vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) technique.

Reason: Neural networks require fixed-size input vectors. TF-IDF provides a way to convert variable-length reviews into fixed-size vectors while also giving importance to meaningful words.

Building the ANN:

Input Layer: Number of neurons equals the length of the TF-IDF vectors.

Hidden Layer 1: Choose a number (e.g., 128) of neurons. Activation function can be 'ReLU'.

Hidden Layer 2: Another set of neurons (e.g., 64). Activation function can be 'ReLU'.

Output Layer: Number of neurons equals the number of categories (in this case, 4). Activation function should be 'softmax'.

Reason for Softmax: Softmax ensures the output values are in the range (0,1) and sum to 1, which can be interpreted as probabilities for each category.

Training the ANN:

Use categorical cross-entropy as the loss function.

Reason: This is a multi-class classification problem, and categorical cross-entropy is suitable for such tasks.

Use an optimizer like 'adam'.

Split the dataset into training and testing subsets.

Train the model on the training subset and validate on the testing subset.

Evaluation:

After training, evaluate the model's performance on the test data.

Use metrics like accuracy, confusion matrix, etc., to understand the model's strengths and weaknesses per category.

Deliverable:

Submit the Python code with the implemented ANN.

A short report detailing the architecture decisions, observed model performance, and any insights or observations from the reviews' classifications.

# data[].attributes.body
From the Dataset: In a supervised learning setting, the categories (or classes) are often provided in the training dataset itself. In our example, the "Category" column in the dataset provides these classes.

Domain Knowledge: Sometimes, the categories might be predefined based on business or domain knowledge. In an e-commerce setting, the platform would already know the types of products they sell and thus the potential categories for reviews.

Exploratory Data Analysis (EDA): If the dataset is large and varied, exploratory data analysis can help in understanding the potential categories. For instance, by examining the unique values in the 'Category' column of our dataset, we can identify the distinct categories.

# data[].attributes.body
The input_shape argument in the first Dense layer specifies the shape of the input data (in this case, the length of the TF-IDF vector).

The first hidden layer has 128 units and uses the ReLU activation function.

The second hidden layer has 64 units, again with the ReLU activation function.

The output layer has 2 units (since we're classifying texts into two categories) and uses the softmax activation function to output class probabilities.

The model is compiled with the Adam optimizer, a commonly used categorical loss function (sparse_categorical_crossentropy), and is set to track accuracy as a metric.

We then train the model on the training data with validation data provided to track performance on unseen data.

# data[].attributes.body
Define the architecture of the network in terms of layers, units, and activation functions.

Specify the input shape that corresponds to the feature dimensionality.

Determine the number of units in the hidden layers based on the complexity of the problem and the amount of data available.

Use appropriate activation functions for the hidden layers (often ReLU is a good default choice).

This architecture is a simple feed-forward neural network (often referred to as a multi-layer perceptron). The specific architecture—meaning the number of layers, the number of units in each layer, and the choice of activation functions—is something that you'd typically experiment with and adjust based on performance on validation dat

import numpy as np
import tensorflow as tf
from sklearn.datasets import fetch_20newsgroups
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer

# 1. Load Data
categories = ['alt.atheism', 'soc.religion.christian']
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)
newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)
X_train_raw, X_val_raw, y_train, y_val = train_test_split(newsgroups_train.data, newsgroups_train.target, test_size=0.2, random_state=42)

# 2. Preprocessing
vectorizer = TfidfVectorizer(max_features=5000, stop_words='english', lowercase=True, strip_accents='unicode', decode_error='ignore')
X_train = vectorizer.fit_transform(X_train_raw).toarray()
X_val = vectorizer.transform(X_val_raw).toarray()

# 3. Neural Network Model
model = tf.keras.models.Sequential([
    # Input Layer (implicitly present by specifying input_shape)

    # Hidden Layer 1
    tf.keras.layers.Dense(128, activation='relu', input_shape=(X_train.shape[1],)),

    # Hidden Layer 2
    tf.keras.layers.Dense(64, activation='relu'),

    # Output Layer
    tf.keras.layers.Dense(2, activation='softmax')  # Two categories
])

# 4. Compile the Model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# 5. Train the Model
model.fit(X_train, y_train, epochs=10, batch_size=32, validation_data=(X_val, y_val))

# data[].attributes.headline
What is TF-IDF and why is it needed here?

# data[].attributes.subject
What is TF-IDF and why is it needed here?

# data[].attributes.body
TF-IDF stands for Term Frequency-Inverse Document Frequency. It's a statistical measure used to evaluate the importance of a word in a document relative to a collection of documents, often called a corpus. TF-IDF is commonly used in text processing for feature extraction and text vectorization.

In the context of this class activity (text classification using an ANN), the TF-IDF vectorization offers a way to transform raw text data into a numerical format that the neural network can process. Each document (or text snippet) becomes a vector, where each dimension corresponds to a term's TF-IDF score. These vectors serve as input features for the ANN.

Here's a breakdown:

Term Frequency (TF): It represents how frequently a term (word) appears in a document. Given a term t and a document d, the term frequency TF(t,d) is often the raw count of term t in document d. However, it can be normalized (e.g., by the total number of words in d).|

​

Inverse Document Frequency (IDF): It gauges how important a term is within the entire corpus. For a term t and a corpus D:

The main idea is that terms that appear frequently in many documents are less informative than terms that appear in fewer documents.

TF-IDF Vector: It's the product of TF and IDF:

TF-IDF(t,d, D)=TF(t, d)×IDF(t, D)

This gives a weight to every term for each document, which can be used as features for training machine learning models.

Why is it needed?

Dimensionality Reduction: Instead of using all terms in the corpus as features, one might focus on the terms with the highest TF-IDF scores, which are more informative, thus reducing the feature space.

Relevance: It helps in highlighting words that are more interesting, i.e., frequent in a document but not across documents.

Noise Reduction: Common words (e.g., "and", "the") might appear in many documents. While they have high TF, their IDF is low, giving them lower TF-IDF weights. This de-emphasizes frequent but unimportant words.

Sparse Representation: For text classification and clustering tasks, the high-dimensional data can be challenging. The sparsity of TF-IDF representations can be leveraged by certain algorithms for efficiency.

Improves Model Performance: Often, models trained on TF-IDF representations of text outperform those trained on simple raw counts or binary occurrences.

# data[].attributes.headline
Why lowercase, remove punctuation, and tokenize?

# data[].attributes.subject
Why lowercase, remove punctuation, and tokenize?

# data[].attributes.body
Convert to Lowercase:

Consistency: Words like "Hello", "HELLO", and "hello" are treated as different tokens if not standardized to a common case. Converting everything to lowercase ensures that the algorithm recognizes them as the same word, leading to a consistent word count and representation.

Reduces Dimensionality: By converting all the words to lowercase, you reduce the size of the vocabulary, thereby reducing computational cost and the risk of overfitting.

Remove Punctuation:

Noise Reduction: Punctuation marks might not add significant meaning in some text analysis tasks. For instance, while "What is this?" and "What is this!" convey slightly different emotions to humans, they might be treated similarly in a basic sentiment analysis model. Removing punctuation ensures the model isn't swayed by them unnecessarily.

Standardization: Some texts might have excessive or inconsistent punctuation use. Removing them helps standardize the dataset.

Computational Efficiency: Simplifying text by removing punctuation can lead to faster processing.

Tokenize:

Text to Tokens: Most algorithms and feature representation techniques don't work directly on raw strings. They require input to be in the form of tokens or sequences. Tokenization breaks text into individual words or subwords, making it suitable for such representations.

Vocabulary Building: Tokenization aids in building a vocabulary of the dataset, which is essential for methods like bag-of-words or word embeddings.

Facilitates Further Processing: Once a text is tokenized, it's easier to perform further preprocessing steps, such as stemming or lemmatization, which reduce words to their base or root form.

# data[].attributes.body
A working Streamlit app of the Neural Network model.

A brief report explaining the architecture of their neural network, choices made (like number of neurons, activation functions, etc.), and performance metrics on test data.

# data[].attributes.body
# Required Libraries
import numpy as np
from sklearn.datasets import fetch_20newsgroups
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from keras.models import Sequential
from keras.layers import Dense

# Load Dataset
data = fetch_20newsgroups(subset='all')
X, y = data.data, data.target

# TODO: Preprocess text data: Convert to lowercase, remove punctuation, tokenize

# Convert Text Data to Numerical Format
vectorizer = TfidfVectorizer(max_features=5000) # Limit to 5000 most frequent words
X_vectorized = vectorizer.fit_transform(X)
X_vectorized = X_vectorized.toarray()

# Split data
X_train, X_test, y_train, y_test = train_test_split(X_vectorized, y, test_size=0.2)

# Design Neural Network Architecture
model = Sequential()
# TODO: Add Input layer
# TODO: Add First Hidden Layer
# TODO: Add Second Hidden Layer
# TODO: Add Output Layer

# Compile the Model
# TODO: Compile model specifying optimizer, loss and metrics

# Train the Model
# TODO: Fit the model using training data

# Evaluate the Model
# TODO: Predict on test data and evaluate the performance

# data[].attributes.attachment_link.title
5.6.2. The 20 newsgroups text dataset — scikit-learn 0.19.2 documentation

# data[].attributes.attachment_link.description
This is documentation for an old release of Scikit-learn (version 0.19). Try the latest stable release (version 1.3) or development (unstable) versions.

# data[].attributes.body
https://github.com/isac-artzi/AIT-204-ANN-PyTorch Implementing an ANN for Text Classification

Objective:

To build and train an artificial neural network (ANN) with at least two hidden layers to classify text documents into predefined categories.

Motivation: Imagine you are dealing with customer reviews for a multi-category e-commerce platform. Each review is associated with different categories such as "Electronics", "Clothing", "Home & Kitchen", and "Books".

The aim is to build an Artificial Neural Network (ANN) that, given a new customer review, can automatically predict and classify the review into one of these predefined categories based on the content of the review.

Why this is useful: Automatically classifying reviews can help in streamlining product feedback to specific departments without manual intervention. It can also assist in analytics, reporting, and tailoring user experiences.

Dataset:

For simplicity, use the 20 Newsgroups dataset, a collection of approximately 20,000 newsgroup documents, partitioned across 20 different newsgroups. This dataset is available in the scikit-learn library.

Steps:

Data Preparation:

Load the dataset.

Preprocess the text data: Convert to lowercase, remove punctuation, and tokenize.

Convert the text data into numerical format using techniques like TF-IDF (Term Frequency-Inverse Document Frequency).

Split data into training and testing sets.

Design the Neural Network Architecture:

Input layer size corresponds to the number of features (length of the TF-IDF vectors).

Two hidden layers. Students can experiment with the number of neurons in each.

Output layer size corresponds to the number of classes (newsgroups). Use softmax activation for multiclass classification.

Train the Neural Network:

Define the loss function, e.g., categorical crossentropy for multiclass problems.

Use backpropagation to update the weights.

Train the model for a set number of epochs or until the loss converges.

Evaluate the Model:

Use the test data to evaluate the model's performance.

Calculate accuracy, precision, recall, and F1-score.

# data[].attributes.attachment_link.description
Contribute to isac-artzi/AIT-204-ANN-PyTorch development by creating an account on GitHub.

# data[].attributes.body
From a mathematical standpoint, using a single hidden layer with enough neurons could theoretically approximate any function. This is the Universal Approximation Theorem. However, the challenge lies in the practicality and efficiency.

In a one-layer system, the transformation from input to hidden activations would be:
A=f(X×W+b)

To capture the same complexity as our two-layer example, the width (number of neurons) of this layer would potentially need to be very large, resulting in a high-dimensional weight matrix
W, which may be computationally infeasible and challenging to train.

By adding depth (more layers), neural networks can represent hierarchical features with fewer overall parameters, leading to a more efficient learning process and often better performance.

# data[].attributes.body
Using the activations A 2​ from the second hidden layer, we can calculate the final output
O=f(A 2​×W 3​+b 3​)

This transformation, A 2​×W 3​, represents the interpretation of mid-level features to make a final prediction about the facial expression.

# data[].attributes.body
The activations A 1​ from the first hidden layer serve as inputs for the second hidden layer. The activations A 2​ for the second hidden layer are:

A 2​=f(A 1​×W 2​+b 2​)

Here, A 1​×W 2​ represents the transformation of low-level features from the first layer to more complex structures in the second layer.

# data[].attributes.body
Given input X, the activations A 1​ for the first hidden layer are:

A 1​=f(X×W 1​+b 1​)

Here, X×W 1​ calculates a weighted sum of inputs for each neuron in the first hidden layer. This represents the raw strength of activation for those neurons. Once biases are added and the activation function is applied, we get A 1​, the final activations for Hidden Layer 1.

# data[].attributes.body
Notations:

Let X be the input matrix where each row represents an image in its flattened form.

W 1​, W 2​, and W 3​ represent the weight matrices for the connections between the input and the first hidden layer, the first and second hidden layer, and the second hidden layer and the output layer, respectively.

b 1​, b 2​, and b 3​ are bias vectors for each layer.

Activation function f() is applied element-wise.

# data[].attributes.body
While it's theoretically possible for a single hidden layer to approximate any function (given enough neurons), in practice, it's often computationally infeasible and inefficient. The hierarchical nature of data like images often means that deeper networks, which learn a hierarchy of features, can do so more efficiently (with fewer parameters) than a shallow, wide network.

In our facial recognition example, a single hidden layer would have to recognize both low-level features (like edges) and mid-level features (like the shape of a smiling face) simultaneously. This intertwined, non-hierarchical learning can be more challenging and inefficient.

The use of multiple layers allows for a gradual build-up of complexity, mirroring the hierarchical structure of the data. Each layer operates at a different level of abstraction, and this layered abstraction makes learning more efficient and often leads to better performance.

# data[].attributes.body
Neurons in this layer might integrate patterns from the first hidden layer. For instance, if certain neurons in the first layer activated in response to the boundary of eyes and others for the curve of a smile, neurons in the second layer might activate when these patterns appear together, signifying a "smiling face" or "raised eyebrows."

# data[].attributes.body
Neurons in this layer might integrate patterns from the first hidden layer. For instance, if certain neurons in the first layer activated in response to the boundary of eyes and others for the curve of a smile, neurons in the second layer might activate when these patterns appear together, signifying a "smiling face" or "raised eyebrows."