• 5G trends are driven by apps and services needed globally

1G --> Analog voice calls

2G --> Digital voice and text SMS (short message service)

3G --> SMS, MMS (multimedia service), web & emails, increased transfer rate, bandwidth (transfer capacity), features & coverage

4G --> IP enabled, mobile broadband (MBB high-speed internet access), fast mobile data transport smartphones with Social Media

5G --> eMBB (enhanced MBB increase bandwidth decreased latency), IOT, AI, AR, VR, Cloud, more reliable and capacity-rich

-- 5G Areas --

• 3 key areas 4 faster & better mobile connectivity

  1. eMBB --> Enhanced Mobile Broadband (first phase)

  2. mMTC --> Massive Machine Type Communication (second phase)

  3. URLLC --> Ultra-Reliable and Low Latency Communication (second phase)

• 4 usage scenarios

  1. mobile broadband mobility & high data rate (eMBB)

  2. Dense user crowds high capacity & high data rate (eMBB)

  3. IOT sensor network high volume of devices & battery life, moderate bandwidth (mMTC 4 smart city)

  4. IOT control network for low latency &high reliability (URLLC 4 self-driving vehicles)

• 3GPP --> 3 Gen Partnership proj refering to standards organisation 4 global mobile tech

• ITU-R (International Union Radio communication sector) regulates international radio freq & satellite resources and green lit IMT-2020 (5G research international mobile telecommunications)

-- AIOT --

• interconnection, via the internet, of computing devices embedded in everyday objects enabling them to send and receive data. We follow the 6 C’s of IoT in order to conceive the IoT system. The 6 C’s are:

• Collect - collection of data

• Communicate - wireless (or wired) communication technologies

• Connect - network connection, usually through the telco’s WAN infrastructure, with the use of switches and routers

• Cloud - computing resources such as storage and server applications residing in AWS (Amazon Web Services), (Microsoft’s)Azure, Google Cloud etc.

• Comprehend - data analytics to make sense of the data

• Create - value creation through mobile apps

  
  

-- 5G Characteristics --

Syntax

• // floor division resulting in int of x/y no rounding

• % modulo returns int remainder of division 

• indentation - shows relationship

• string.title() --> first letter in each word upper case

Data types (py cannot mix data types)

• int

• string

• float

• None - empty

• Structured data --> tabled

• Unstructured data --> like video etc

• boolean

• check using type()

• implicit conversion --> auto convert 1 data type into another (into float)

• explicit conversion --> manual convert from one data type to another str(), int()

Functions (combines calls to function making code reuseable)

• print(), len(), 

• def functionName(parameters):

           body

           return value, x

• value, x = functionName(parameters)

• function exits once it reaches a return statement in branch

Style

• good style like human language easier to understand and maintain

• readable, less errors during updating more straight forward 

• self-documenting code --> written in a way that's readable and doesn't conceal it's intent 

• refactoring a code to be self documenting to reflect code's purpose with good naming and comments

• #after this is comment

Comparison (returns a Boolean)

• cannot compare amt < >int str types error

• == and != Can compare types

• true and true = true

• true or false = false

• not flip Boolean result

Branching --> the ability of a program to alter its executing sequence (different decisions based on guidelines)

• if condition: body (carried out if condition met / is true)

• if, elif (checks only if false elif true), else (all else false)

ARM Cortex A72: many resources, short development cycle, increases in processing power

-- Embedded systems --

• Input > Process > Output

1. Fixed purpose, specialised computers

2. Smaller size physically, part of a larger system

3. Limited memory (little Ram program stored in ROM)

4. Failure tolerance (no hang has control of wonky environment)

5. Real time control (operation processing dependent on speed with complex cal or human interactions)

6. Guaranteed response time (requests to device fast when multitasking)

7. Reliability

8. Simple I/O

9. low power consumption

10. low cost

-- Organisation --

1. CPU

   • processes data/instruction

   • responds request

2. Memory

   • RAM: Random Access Memory, changing running program data, volatile (no power gone)

   • ROM: Read Only Memory, fixed data, non-volatile (no power not gone)

3. I/O: external input (start processing) and output (display result)

4. Buses

   • Address: memory location being accessed (unidirectional, memory capacity = 2 ^ [no. of lines] )

   • Data: transfer data (bidirectional, 1 line = 1 bit = 1 memory word size)

   • Control: synchronize transfer data (bidirectional, no. of lines affects control, interrupt and transfer rate)

-- Hardware considerations --

• Interrupts --> only operate when triggered (ISR Interrupt Service Routine)

• Hardware Timers

  1. Software delay loops (pro: simple, cons: processor occupied, no multitasking, inaccurate execution varies)

  2. Hardware timer chips external to CPU (set value to timer, count up/down to interrupt, accurate)

• Direct Memory Access --> DMA

  1. memory and I/O data transfer without CPU, frees CPU to process other things)

• Power Management

  1. Programming --> code executes fast & low power mode (main loop others ISR)

  2. Power supply --> low drop out voltage regulators (soft), SMPS Switch Mode Power Supplies (loud), alt power

  3. Processor --> different operating frequencies / voltages for sleep mode

-- Benefits of embedded sys --

1. Computing power increase --> connectivity of devices

   • media processing

   • floating point data

   • sophisticated algorithms (e.g fuzzy logic, neural networks)

   • real-time processing (speed)

2. IOT --> users data and analytics of future trends

3. self powered --> conserve energy

-- Increase Performance --

• Architecture design: pipeline, parallel, hardware optimization

• increase clk freq increase processing speed

• increase address or data bus size (in increments of 8)

1. it needs to be socially beneficial

2. avoid creating unfair bias with sensitive characteristics such as gender, race, income, religion etc

3. build and test for safety -> avoid results that create risks

4. incorporate privacy design principles such as consent and with privacy safeguards & transparency

5. accountable to people -> provide appropriate opportunities for feedback, explanations & appeal

6. hold high standards for scientific excellence

7. be available for uses that accord with principles -> limit possible harmful or abusive applications and uses

google cloud infrastructure consists of 3 layers with Network & Security (support) being the base, Compute and Storage (separated to scale independently based on need), and Data & AI products (perform task to ingest, store, process & deliver business insights, data pipelines & ML models)

middle layer:

1. Compute

   More data needs require more compute power.

   Processing power: TCU (Google's custom-developed for specific uses, it is domain specific HW)

   Services:

   • Compute Engine (IaaS)

     Compute, storage, NW, max flexibility

   • K8s engine

     Containerized app in the cloud envir. (not on VM like CE)

   • App Engine (PaaS)

     bind code to libraries that access to infra

   • Cloud Functions

     execute code in response to events, severless

   • Cloud Run

     fully managed platform that allows event-driven stateless workloads, autoscaling, and pay for what you use

2. Storage

   for proper scaling capabilities, compute & storage are decoupled

   Services:

   • Cloud storage (unstructed data)

   • Cloud bigtable (NoSQL DB)

   • Cloud SQL (relational DB)

   • Cloud Spanner (relation DB)

   • Firestore (NoSQL DB)

   • bigQuery (unstructured data)

   Analytical workloads: involve processing large amounts of data to find patterns, trends, or insights

   Transactional workloads: everyday tasks and actions, like making purchases, updating information, or handling user interactions

top layer:

Data-to-AI workflow:

ingestion & process -> storage -> analytics (BigQuery provides embedded AI features allowing direct calls to SQL commands to train model)-> AI/ML

• Source file ABC.Java

-- Scanner input --

String name;

int age;

double gpa;

Scanner sc = new Scanner(System.in);

name = sc.nextLine();

age = sc.nextInt();

gpa = sc.nextDouble();

method overloading --> diff signature call diff based on parameters durring runtime

• indentation, variables

• casting --> int(), str(), float()

• customizing parameters: %s --> holder for string, %d --> holder for number

Collection / Arrays

• list --> ordered & changeable --> my_list = ["apple", "banana", "cherry"]

• Tuple --> ordered & unchangeable --> my_tuple = ("apple", "banana", "cherry")

• Set --> unordered (cannot be called by index) & unchangeable --> my_set = {"apple", "banana", "cherry"}

• Dictionary --> collection unordered, changeable, indexed key: value --> my_dict = {"brand": "Ford", "model": "Mustang", "year": 1964}

Loops

• if --> true, elif --> if false elif true, else --> all else false

• range(start, end when less than, increment value)

• for x in range array: #loop through each element in array

• for x in range(n): #loop through i to n-1 or n times

• while, break --> exits loop if true, continue --> go back to start of loop skips rest of code below

Functions

• def yanged(name):

  print(str(name)+" Yang")

  return name.upper()

  x = yanged("Celine")

  print(x)

  x = yanged("Rachel")

  print(x)

Class

• Class --> grp tgt related data and functions which act on that data

• Object --> unique instance of a data structure defined by its class

• class HiClass: #class name declare

  me = "celine" #varible

  def say_hi(self):

  print('Hello %s, how are you?' % (self))

  def name(self):

  print('Hello %s, how are you?'% (self.me))

  p = HiClass() #object p in HiClass

  p.say_hi()

  p.name()

• call object use self.obj or p.attribute

• class Dog():

  """A simple attempt to simulate a dog"""

  def init (self,name,age):

  """Initialize the name and age attributes."""

  self.name = name #attribute

  self.age = age

  def sit(self):

  """Simulate a dog sitting when ordered."""

  print(self.name.title()+" is now sitting")

  def roll_over(self):

  """Simulate a dog rolling over when ordered."""

  print(self.name.title()+" rolled over!")

  def info(self):

  print(self.name + "is %d years old!" % (self.age))

  #def testing():

  #print()

  # Instantiate a class.

  dog = Dog ("Husky",2)

  dog.sit()

  dog.roll_over()

  dog.info()

  # dog.testing() --> Type error 0 arguments but 1 self was given

• Inheritance --> type subtype relations between classes for code reusability

• class Parent: # Define the parent class.

  parentAttr = 100

  def init(self):

  print("Invoke the parent class to construct a function.")

  def parentMethod(self):

  print('Invoke a parent class method.')

  def setAttr(self, attr):

  Parent.parentAttr = attr

  def getAttr(self):

  print("Parent attribute:", Parent.parentAttr)

  class Child(Parent): # Define a child class.

  def init(self):

  print("Invoke a child class to construct a method.")

  def childMethod(self):

  print('Invoke a child method.')

  c = Child() # Instantiate a child class.

  c.childMethod() # Invoke the method of a child class.

  c.parentMethod() # Invoke the method of a parent class.

  c.setAttr(200) # Invoke the method of the parent class again to set the attribute value.

  c.getAttr() # Invoke the method of the parent class again to obtain the attribute value

• _privateVar --> only accessible in class

• publicVar

Anaconda

• integrated, install different framework --> different environment

• py many integrated libraries & framework to save development time

• Numpy --> numeric computing handles vectors and matrices

• Pandas --> data manipulation data structures and tools of analysis

  1. grouping data

  2. combining data

  3. filtering

  4. time-series

  5. re-indexing

  6. iteration

  7. Sorting

  8. Aggregation

  9. Concatenation

  10. Visualization

• PIP --> py libraries: tool for installing PyPI python packages

• Conda --> package manager: open source package management system and environment management system quickly installs, runs and updates packages and their dependencies and switches between environments on your local computer.

Anaconda PowerShell

• conda search packageName --> see specific package

• conda list --> see installed packages

• conda install numpy --> installs numpy or speciic version numpy=1.16.5

• conda remove numpy --> remove package in current environment

• conda create -n my_env numpy --> create environment with numpy installed

• conda create -n my_env python=3.3 --> create environment with py 3.3

• conda activate env_name --> active environment before start of use

• conda deactive --> deactivate current environment

• conda env list --> list all virtual envirnments

• jupyter notebook --> start new notebook open web pg

-- Numpy --

• import numpy as np

• arr = np.array([]) --> type(arr) is numpy.ndarray --> N-directional array

• one_d_array.ndim --> shows number of dimensions of an array

• one_d_array.size --> shows length of array

• np.arange(10) --> array with sequence 0-9 with len 10, step 1

  np.arange(1,10) --> np.arrange(start, end) array with start to end -1, step 1

  np.arange(1,10,2) --> np.arrange(start,end,step)

• np.arange(10).reshape(2,5) --> one dimentional array to 2 dimensional

  np.arange(10).reshape(2,5).ravel() --> flatten

• a.shape) --> returns tuple of array length per axis 1D:(no. of elements,) & 2D:(no. of row, no. )

• Indexing:

  rank 1 --> arrayName[index]

  rank 2 --> arrayName[row, col]

• : --> colon is a from index to one item before operator

  print(data[:]) --> prints start to end

  print(data[1:5]) --> prints element 1 to 4

• Slicing array using colon is a from index to one item before operator

  data[:,:-1] --> selects all rows (:) and all columns except the last one (:-1). It extracts the features from each row

  data[:,-1] --> selects all rows (:) and only the last column (-1). It extracts the target variable from each row

  data[:,1] --> all rows first col

  data[:,:1] --> all rows all col before col 1\

  data[:2,:] --> rows before row 2, all col

• Array stacking --> joining arrays along existing axis

  np.vstack([a,b]) --> vertical stack no. of col same

  np.hstack([a,b]) --> horizontal stack no.of rows same

• Matrixes made of list of lists e.g.

  m = np.array([[1,2,3],[4,5,6],[7,8,9]])

  n = m[1][2] --> indexing row 1's index 2

• np.dot(a,b) --> matrix Multiplication --> first matrix a row and next matrix b col must be same

• 1 base tower at a time for users, cell transfer users between radio towers when they move to the next area, lose cell when in areas with weak signal / transfer too slow

• bandwidth is capacity of people using it

• MIMO increases capacity by increasing antennas on receiver and transmitter antennas

• frequency = periods / second

-- Network architecture & components --

• 5G Stand Alone Network / SA 5G --> cellular infrastructure built to service 5G through protocols in radio network & controller core

• low latency connectivity great progress for wireless

• supports mission-critical communication, high-speed MBB

• system main parts

  1. 5G UE (user equipment) --> smartphones, computers, etc

     • 
       

  2. RAN (radio access network) --> handles radio related tasks

     • GNB = CU (central unit can be placed in the cloud) + DU

     • Control data transmission and radio access

     • DU: lower-level RF processing and data plane processing are the purview of the

     • CU: higher-level operations and control plane signalling are handled by the CU

  3. CN (core network) -->session management & mobility

     • AMF (Access and Mobility Function): managing the access and mobility aspects of UE certificates

     • UPF (User Plane Function): supports features and capabilities to facilitate user plane operation. Examples include: packet forwarding and routing, interconnection to the Data Network (DN), policy enforcement and data buffering

  4. DN (data network) --> establishes connections with other networks e.g. internet

     • 
       

• Architecture parts

  1. gNodeB (GNB / next gen node B) -->

  2. Access & Mobility Function (AMF) -->

  3. User Plane Function (UPF) -->

  4. Session Management Function (SMF) -->

  5. Authentication Server Function (AUSF) -->

  6. Unified Data Management (UDM) -->

  7. Policy Control Function (PCF) --> handovers and smooth transition

  8. Application Function (AF) --> interface

  9. 
     

  10. CU (central unit) + DU (distribute unit)

• 
  

• 
  

-- Spectrum & frequency bands --

• sub 6 means below fz 6

-- Radio Access Technology (RAT) --

• OFDM (Orthogonal Frequency Division Multiplexing): sub carriers simultaneously transmit data through OFDM modulation (convert data to electrical signals improving spectral efficiency and mitigates effects of multipath interference)

• MIMO (Multi-Input Multi-Output): improve communication performance, numerous antennas at transmitter and receiver ends boost spectral efficiency, data throughput and network capacity

• Beamforming: concentrating radio waves in direction, narrow beams improve signal coverage and quality, lower interference uplink and down link, stronger

• DSS (Dynamic Spectrum Sharing): distributing spectrum rescores dynamically to coesits with existing 5G-LITE for seamless transition to 5G

• mmWave (milimeterWave): high frequency above 24GHz used to deliever ultra high data rates, limmited coverage and blocked by obstacles,large bandwidth and capacity, fast

-- Core network & Services --

• scalability cloud native and cloud network

• SDN - software defined networking: programmability and automation, real time response efficency and responsiveness

• NFV - Network Functions Virtualization: minimizes hardware dependency, improve resource ultlisation, speed up deployment

• O&M - opperations and mantainence: admin cloud streamlines opperations provisioning and configuration

• Edge cloud: integration ofcloudd capabilities at network edge @ end users lowers latency boost performance

• Regional Cloud & core cloud: hierical cloud deployment approaches. RC (goegraphic location only to relieve traffic on edge), CC (centralised control management, optimising resources and services and avalibility decrease latency)

-- 5G Service Oriented Core (SOC) --

• flexible architecture

• programability

• smart pipes (transfers data securely, quality management, deep packet inspection)

• Control & User Plane Separation (CUPS): sepperates userplane data transfer and control plane network and signaling

• Serviced based architecture (SBA): network as service to be reused ad agile

• Network slicing: several networks tbcreated on singale infrastructure for URLLC, mMTC, eMBB

• Native Cloud: microservices & VENV & containerisation

• 
  

• ierih

-- Arrays --

breakpoints, functions, debugging, header files

-- ARM Hardware Architecture --

• ARM processors --> Advanced RISC (Reduced Instruction Set Computer) Machine

• Raspberry Pi (Rpi) ARM cortex A manufactured by Broadcom (

  1. benefits: small size & runs op sys e.g. Linux & low cost

  2. note: cortex A = application, R = real time high performance, M = micro-controller

• Rpi parts

  1. ARM processor (CPU + on-chip GPU)

  2. RAM

  3. HDMI x2

  4. USB ports x4

  5. Ethernet

  6. can add modules with carrier board through surface pins

• Industrial --> compute Module 3 (CM3) ARMv8-A

  1. benefit: custom more peripherals, form factor, flexible easy update replace on board

  2. parts: GPU, Memory, Timers, DMA, Interrupt Controllers, GPIO, USB, PCM, Serial Communication (I2C, PWM, UART)

• SoC --> System on chip

• bus architecture based on AMBA --> Advanced Microcontroller Bus Architecture

• allows interface to external devices

• e.g. AXI --> Advanced extendable Interface for flash memory NAND type

• NAND type Broadcom type SMI --> secondary Memory Interface requires

  • smaller number of address pins

  • 6 requisite control pins

  • "8080", "6800" simple, modest hardware

• 8080 mode bus

  • 16 bit address bus

  • 8 bit data bus

  • separate read write signals control bus

  • LCD & flash drive 8080 mode

  • adr data bus size vary

  • control specific signal sequence

• mapped memory

  • IO/M signal to differentiate mem & I/O

    1. use interfacing device, load store move instructions or high-level assignment

    2. input output instructions comparision

• ROM: Non-volatile, store unchanging data, no write by processor

  1. EPROM --> Erasable Programable ROM: erase all data stored, for testing development

  2. PROM: data stored after manufacturing, data never changed, non-reversible, small scale production

  3. EEPROM --> Electrically Erasable PROM: parallel data rewritten easily, storing data slower than reading, updates per byte, store settings and configuration

  4. FLASH: updates parallel data in grps of bytes, NOR gate tech to store programm for firmware updates flash chips, NAND gate sequential access of data flash devices e.g. me cards, SD, USB, CF or hard disk large data storage

• RAMs: volatile

  1. Static:

  2. Dynamic:

-- Tutorial --

• ROM only written by manufacturer called masking

• Flash device (faster e.g. hard disk ssd) = EEPROM

• EEPROM = electrical erasable programable rom

• CMOS same as BIOs

• buffer for input

• latch output

• tri-state buffer input to CPU

• Bi-directional buffers --> data bus signal

• latch regular lvl trigger, changes according to input --> clocks trigger

• edge trigger, changes --> positive low to high, or negative high to low

1. Networking & security (base layer)

2. Compute & storage (scalable independently)

   • Compute resources: (5)

   • (1) Compute engine --> Iaas flexibility to manage server instances on virtual machine

   • (2) Google Kubernetes Engine GKE --> containerized (code packed with dependencies) applications on cloud (Iaas & Paas)

   • (3) Google App Engine --> Paas bind code to libraries access to infrastructure to focus resources on app logic

   • (4) Cloud functions: functions as a service --> serverless events-based execution

   • (5) Cloud run --> scales from 0, full managed compute platform that runs requests or events driven workloads

   • Tensor Processing unit --> scalable processing power to accelerate machine learning workloads, faster, energy efficent

   • Storage resources, full managed database: (6)

   • (1) Cloud Storage --> unstructured data (preferred), standard (hot/ frequently accessed/ store briefly) data, nearline storage (infrequent / backups), coldline storage rarelyy modified), archive storage (disaster recovery)

   • (2) Cloud Bigtable --> structured analytical NoSQL

   • (3) Cloud SQL --> structured transactional SQL 4 regional scalability

   • (4) Cloud Spanner --> structured transactional SQL 4 global scalability

   • (5) Firestore --> structured transactional NoSQL document oriented

   • (6) BigQuery --> structured analytical SQL (training model) full managed data warehouse& unstructured (eh), mange tabular data and execute ML models by tunning parameter 4 u

3. Data & AI ML products

   • Data 2 AI workflow seamless transition from data to AI

     1. Ingestion & process: gather real time & batch data from source

        • e.g. Pub/sub, dataflow, dataproc, cloud data fusion

     2. Storage: save data in storage

        • e.g cloud storage, cloud sql, cloud big table, cloud spanner, firestore

     3. Analytics: analyze data & visualize result

        • e.g. BigQuery, Bussiness intelligence (BI), looker

     4. AI/ML: train model 4 prediction --> use SQL commands

        • e.g. vertex AI, AutoML (no code), workbench (custom), colab (custom), generative AI studio, model garden

        • Vertical market needs: specifically 4 industry problem --> AI for healthcare, Discovery AI for retail

        • Horizontal market needs: across industries --> contact center AI CCAI, Document AI

Data types

• structured: info stored in tables, rows & columns

  1. transactional workloads: fast inserts & updates for online transactions to maintain snapshots

  2. analytical workloads: entire datasets read for processing requiring complex queries

• unstructured: non-tabular form such as documents, images, and audio files

Model Cat

• AI --> comp mimicking human intelligence

• ML --> subset of AI where comp learn without explicit programming

• Deep learning & deep neural networks --> subset of ML

• Supervised learning: labelled data & task-driven, identify goal

  1. Classification: predict categorical variable true false (e.g. cat dog based on labelled pictures)

     • logistic regression model

  2. Regression: forecast numeric future data based on past values (e.g. sales based on past sales)

     • linear regression model

• unsupervised learning: unlabeled data & data-driven, identify pattern

  1. Clustering: groups similar data (e.g. customer info to determine customer segmentation)

     • k-means clustering

  2. Association: identifies underlying relationships (e.g. correlation between product distance & sales)

     •  association rule learning techniques Apriori

  3. Dimensionality reduction: combining record characteristics to reduce features & improve model efficiency with simplified rules (e.g. combine characteristics 4 simplified rule 4 insurance quote)

     • principal component analysis

BigQuery

• few steps, saves time

• supports ML models & MLOps (experiment to production and helps deploy, monitor, and manage the ML models)

1. extract, transform, load data

   • enrich data with other sources --> SQL joins

2. select preprocess features

   • SQL training dataset

   • one-hot encoding --> categorical data into numeric data

3. create model & evaluate performance

   • choose one of the models above based on goals

   • define label col

4. Deploy, use model to predict

5. moitor

6. release

Google cloud AI tool

1. pre-trained APIs (application programming interface) --> no experienced personnel Paas

   • cloud natural language API: parts of speech as entities & sentiment

   • Speech 2 text API: audio 2 text 4 data processing

   • Cloud translation API: text to another language

   • Vision API: image content to text

   • Video intelligence API: recognizes motion and action in video

   • Dialogflow API: builds conversational interfaces

   • v generative AI

   • PaLM API: pre-trained large language model

   • Imagen: create & edit images

   • Chirp 4 speech: build voice enabled application

   • codey 4 code generation: reduces & debugs code

   • Natural language API: 4 customer feedback, comments etc

     • Entity analysis: identifies subject, automatic tagging, document classification, information extraction

     • Sentiment analysis: identify emotions with scores between -1 and 1 to indicate strength of emotion (positive negative neutral)

     • Syntax: Linguistics

     • Categories: what is the text about

2. low / no-code --> BigQuery (low code SQL) , AutoML (no code point & click)

3. custom training --> code based ML environment, training & deployment 4 flexibility & control over ML pipeline

Vertex AI

• unified platform to build ML projects end-2-end pipeline

  1. data readiness: upload from wherever

  2. feature readiness: process data features to put into model

  3. training & hyperparameter tuning: experiment with different models

  4. deployment & model monitoring: set up pipeline to turn model into production by monitoring performance & updating improvements

• benefts: seamless, scalable, sustainable, speed

• uses: autoML (no code), custom training (code based)

AutoML

• no code

• AutoML supports four types of data: image, tabular, text, and video.

• automate development & deployment of ML model

• Phases

  1. Data processing: convert data into format

     • numbers: z-score, quantiles

     • text: n-grams, embeddings

     • extract year day month

     • categories: one-hot encoding, grouping, embedding

     • Array of categories: convert 2 lookup index, embedding

     • nested fields: flatten, apply type transformation

  2. Architecture search & tunning:

     • neural architect search finds the best optimal model and tune parameters automatically by comparing performnce

     • transfer learning which speed the searching by using the pre-trained models e.g. LLM if say u have small data set or less computational power can use this 4 better results and accuracy

  3. Bagging ensemble: a few best models are assembled from phase 2 and prepared for prediction

  4. Prediction

Custom training

• coding, more control

• phrases

  1. choose environment: pre-built (TensorFlow, scikit-learn, ptorch) / custom container

     • TensorFlow: end-2-end open platform 4 ML support, multiple abstraction layers with highest built on low-lvl APIs, runs on diff hardware

       1. high lvl API: keras, data

       2. model lib: layers, losses, metrics

       3. low-lvl API: core tensorflow on C++ / Py

  2. create model: layers of neural network

     • model = tf.keras.Sequential

     • tf.keras.layers.dense()

  3. compile model: config evaluation metrics & optimize method

     • model .compile(loss = ..., optimizer = ..., metrics = ...)

  4. fit model: train 4 best fit model

     • model.fit(tf.expand_dims(x, ...), y, epochs = ...)

• Neural network - input layer, hidden layer, output layer
  

• e.g. DeepNN, ConvolutionalNN, RecurrentNN, LargeLanguageModel, ArtificalNN
  

• node = neuron, line connecting nodes = synopses weights
  

• training steps how ANN learns
  

  1. Calculate the weighted sum: tt sum of (input * weight)
     

     • weights & bias r parameters only set initial values
       

     • what the model learns ^
       

  2. Apply activation function to weighted sum
     

     • activation function converts linear network to non-linear
       

     • ReLU --> rectified linear unit function: turns input to zero if negative or original if positive
       

     • Sigmoid function: turns input to value between 0 to 1 for binary classification in logistic regression models (spam / not spam)
       

     • Hyperbolic tangent (Tanh): shifts sigmoid curve to -1 to +1,
       

     • Softmax: multi-class classification, probability distribution
       

  3. cal weighted sum for output
     

  4. apply activation function to weighted sum
     

  5. loss / cost function to see performance: compare predicted y & actual y
     

     • loss function used to calculate errors for a single training instance
       

     • cost function calculates errors from entire training set between predicted and actual value
       

       1. MSE mean squared --> regression problem
          

       2. cross-entropy --> classification problems, probability distribution
          

  6. Backpropagation: adjusting weights and bias
     

     • take cost function & turn into search strategy
       

     • gradient descent --> process of walking down the surface formed by the cost function and finding the bottom (till gradient = 0 flat line)
       

     • direction (derivative of cost function + slide left) & step size (learning rate set beforehand as a hyperparameter)
       

  7. Iteration: setting hyperparameters like learning rate & epoch (step 1 to 6)
     

     • set weights adjustment till optimum
       

     • till value stops decreasing
       

     • hyperparameters decided before training, determines how machine learns: e.g. num of layers, neurons, activation function, learning rate, epochs
       

     • ^ how the model learns
       

     
     

ML workflow

• steps:
  

  1. data prep --> input data (real time streaming vs batch data), engineer features
     

  2. model development --> train & evaluate
     

     • train: method, details
       

     • evaluate: recall & precision use confusion matrix 4 performance measurement
       

     • recall --> true positive / (true positive + false negatives)
       

     • precision --> True positive / (true positives + false positives)
       

     • recall & precision are trade-offs
       

     • feature importance
       

  3. model serving --> predict used, deploy & monitored
     

     • deployment:
       

       1. endpoint --> low-latency deployed 4 real time predictions
          

       2. batch prediction --> no immediate response needed, accumulated data procssed by single request (e.g. targeted ads weekly)
          

       3. offline prediction --> preloaded deployed model for off the cloud specific environment (e.g. sensitive data, internal data)
          

     • monitoring: Vertex AI Pipelines --> automate, monitors & governs workflow in serverless manner, triggers warning based on predefined trashhold
       

     • Vertex AI workbench --> define own pipeline with SDKs
       

• code workflow with: vertax AI, Colab
  

MLOps

1. plan pipeline components
   

2. build custom components
   

3. add prebuilt component
   

4. compile pipeline
   

5. define & run job
   

Generative AI

• generates content by learning from existing content of foundational model
  

• Create descriptions, create images, summerize videos etc, generate Q&A, discover code bugs, search document, automate classification of feedback & ticket, automate extraction & labeling of contacts
  

• generated content can be multi-modal, including text, code, images, speech, video, and even 3D
  

• creating marketing campaigns, generating code, drafting text, extracting information, summarizing documents, providing virtual assistance, and operating call center bots
  

• e.g. LLM
  

Model Garden

• search engine for models on vertex AI
  

• pre-trained and open-source models:
  

  1. foundation models:  large models that can be tuned or customized for specific tasks by using Generative AI Studio, Vertex AI APIs and SDKs --> PaLM, Imagen, embedding API, Chirp Codey
     

  2. task-specific solutions: pre-trained models 4 specific problems --> entity analysis, syntax analysis
     

  3. fine-tunable or open source models: through custom notebook / pipeline
     

• filter 4 proj requirements
  

  1. Modalities: language, vision, speech
     

  2. Tasks: generation, classification, detection
     

  3. Features: pipeline, notebook & detection
     

• Tune model --> deploy model 2 notebook --> customize open source model
  

• 2K storage = 2048 --> 0x00 to 0x07FF

• 4K storage = 4096 --> 0x00 to 0x0FFF

• don't care connect to ground

• base address can also be used for the device components' address

• Viability: profitable & sustainable value chain

• Desirability: customers needs

• Feasibility: ability to carry it out, logistics

Blue vs Red Ocean

• Blue: new market, vast opportunities, no other competitors mistakes to learn from

• Red: cut-throat competition, known market, plenty of examples of how to run a business

LEAN framework

1. Reactive Machine AI (basic): no learning no past experiences, repeated behavior

2. Limited memory AI: learns from past, limited mem, data ultilized to build reference model

3. Theory of mind AI: comprehends enities interacts with by recognising needs, emotions, beliefs, thoughts.

4. Self-aware AI: theoratical, understands its characteristic and who they are to percieve emotions and comprehend various situtations

AI - ability to think like people. These processes include reasoning, self-correction, and learning

ML - automated search for significant patterns in data. AI that enables a system to learn from data instead of explicit programming is called machine learning. (improves, describes predicts data)

DL - Massive amounts of data are usually ingested by deep learning algorithms, which then use supervised or even unsupervised learning to generalise the characteristics/features and categories associated with that data. Neural networks are the foundation of deep learning systems.

• data(observations) + model(expections from prior data) * compute prediction (classification or quality rating)

• Supervised

  • regression

  • classification

  Alggorithums

  • linear regression:

  • random forest: classification & regression

  • support vector machines: classification

• Unsupervised

  • clustering: split into similar grps

  • anomally detection: usual points

  • Association Mining: sets of items that occur tgt in data (corelated)

  • latent varible models: data pre-processing to reduce dimensions features

• Reinforecement learning (RL)

  • "Agent" in reinforcement learning (RL) learns how to accomplish its objectives/goals in an uncertain and sometimes complex environment. For instance, the agent may be placed in a simulation where it is rewarded or penalised for the actions that it executes\

  • agent maximases rewards

  • appropriate simulation env

-- Neural Networks --

• edges connections like synapses

• signals are numbers

• nodes are like neurons

• weights increase or decrease signal strength

• layers

• ANN (artifical neural network): just NN, 1 layer

• DNN - multiple layers

• SNN (simulated NN)

• CNN (Convolutional): multiple pooled or fully interlocked layers where the integrals that overlaps those is CNN, input operation applyed per layer for deep but less parameters, NLP, image and videos

  1. convolution: linear opperation elementwise matrix multiplication and addition, extract features from input image preserving spational relationship

  2. Non-linearity ReLU ()Rectified Linear unit): operation applied after convolution introducing non-linearity

  3. spatial Pooling / down-sampling: reduce dimensionality while retaining important info

  4. classification: fully connected layer from first 3

• Generative AI: subset of DL creates content based on existing learnt content through statistical models

• LLM: core based on deep learning model transformer architecture, results in vector of word embedding for correlations between words

• Computer vision: generative AI creates new images by learning structure and features of existing images, GAN (generative adversal network) has a discriminator to seperate real images from fake for realistic images

-- AIOT --

The seamless integration of AI technology into IoT systems is the core aspect of AIoT. AI gives Internet of Things (IoT) devices the ability to think, allowing them to analyse data, spot trends, and decide for themselves. Usually, the architecture goes through these stages:

1. Data Collection: Massive volumes of data are gathered by Internet of Things devices like wearables, cameras, and sensors from their environment.

2. Data Transmission: For additional processing, the gathered data is sent to a cloud platform or centralised server.

3. Data Processing and Analysis: In order to extract insights, identify anomalies, and forecast trends, artificial intelligence (AI) algorithms, including machine learning and deep learning, process and analyse the data.

4. Decision Making: Decisions are made with the help of data analysis insights. AI gives the system the ability to make these decisions on its own, without assistance from a person.

5. Action and Control: Based on the decisions made, the AIoT system can set off actions or control other devices, resulting in automated and optimised operations.

6. Continuous Learning: AIoT systems are made to continuously learn from new data and user interactions, which increases their efficiency and accuracy over time.

Benifits

• real-time data processing

• predictive mantainence

• enhanced user experience

• energy efficency

• security annomoly detection

• new bussiness opportunities

• reduce downtime

-- Cloud based AIOT --

• device layer: hardware devices

• connectivity layer : cloud storage to controllers sensors etc

• Cloud layer: data storage, visualistaion, analystics, processing and API access

• User communication layer: web portals and apps

-- edge based AIOT --

• collection terminal: hardware (sensors and embedded devices) connected to gateway

• connectivity: field gateways via powerlines

• edge layer: facilitates data processing and insight generation

• AI applications on end devices

• 
  

• ML model only consume numeric data (binary)

• Messy data --> missing attributes, noise, outliers, wrong, duplicates, differing case col names, not ready for ingestion

• import libraries and csv data file into pandas data frame

  df_filename = pd.read_csv('../datafilelocation/filename.csv')
  
  df_filename.head(X) #first X num of rows to show
  df_filename.tail(X) #last X num of rows to show

• Analyse & understand data

  df_filename.info() #show data frame and data types
  #note string appears as obj 
  
  print(df_filename, X) #prints last and first 5 rows of data
  
  df_filename.describe() # summary statistics mean, std, count, min max, quartiles
  
  grouped_data = df_filename.groupby(['criteria col', 'criteria col']) # groupby splits data into cols
  df_filename.groupby('Fuel').first() #groupby first entry each month
  
  df_filename.shape[0] #num of rows
  df_filename.shape[1] # num of cols
  df_filename.columns.tolist() #list of col feature names
  df_filename.nunique() # num of possible data variations per feature
  df_filename.isnull().sum().values.sum() #total null missing values

• Solve missing values

  df_filename.isnull().sum # shows all null values per feature col in data Frame
  print(df_filename['Date'].isnull()) #shows rows of Date feature col (no is null row show Nan, isnull() row with Nan = True)
  
  #replace missing with most common value using lambd
  df_filename = df_filename.apply(lambda x:x.fillna(x.value_counts().index[0]))
  

• Converting date cols to date time format

  df_filename['Date'] = pd.to_datetime(df_filename['Date'], format='%m/%d/%Y) #convert string to datetime format
  
  df_filename['year'] = df_filename['Date'].dt.year #creating col year and casting date col's year data in

• renaming feature col

  #all cols either all upper or all lower, no space
  df_filename.rename(columns = {'Date':'date', 'Zip Code':'zipcode', 'Model Year':'modelyear'})

• remove value from feature

  #unusable data e.g. col  type num int64 has <2006 remove rows
  df = df_filename.loc[df_filename.modelyear != '<2006'].copy()

• one-hot encoding

  # Categorical cols
  
  data_dummy = pd.get_dummies(df[['zipcode','modelyear','fuel', 'make']], drop_first=True) #dummy variables based on categorical
  
  # makes a 0 or 1 col based on each category
  # increases num of cols in df
  
  #concatenate original file df with dummy df
  df = pd.concat([df, data_dummy], axis=1)
  #drop original categorical cols
  df = df.drop(['date','zipcode','modelyear','fuel','make'], axis=1)

• Analyse Pd df

  # relationship between features
  figure out goal label
  read in csv file using pd.read_csv('..\df_filename')
  use head, describe, info, etc

• Seabornplots for data analysis

  # import matplotlib.pyplot as plt
  # import seaborn as sns
  
  # graphs reflecting varibles paired together
  sns.pairplot(df_filename)
  
  sns.displot(df_filename['Pricing']) #histogram count of one col
  
  sns.heatmap(df_filename.corr()) #lighter better correlation, dark black not correlated, confusion matrix

• Train Linear Regression Model using scikit-learn

  #scikit-learn ML library stats modeling, clustering, dims reduction, regression, classification, define model obj to fit set of points / predict value
  
  1. set features (only numbers no categorical  )
  X = df_filename[['Avg. Area Income', 'etc', 'etc2']]
  2. set label
  Y = df_filename['Price']
  
  3. Train Test Split
  from sklearn.model_selection import train_test_split
  X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.4, random_state=101) #same split always set via random_state
  
  4. Create model & Training
  from sklearn.linear_model import LinearRegression
  lm = LinearRegression() # create obj to be LR model
  lm.fit(X_train, y_train) #training model
  
  5. Evaluation
  print(lm.intercept_) # intercept is the value of the bias
  coeff_df = pd.DataFrame(lm.coef_, X.columns, columns = ['Coefficent']) # see weights corrosponding to different features
  #if all other features set, and only that feature changes 1 unit label price will increase the weight unit amount
  
  6. Predictions
  predictions = lm.predict(X_test) # predict values on linear model obj
  plt.scatter(y_test, prediction) # scatter plot represent relation between predicted and actual value (woring = good correlation = plots form linear shape)
  sns.displot((y_Test-predictions), bin=50) # residuals (diff between observed vs predicted) working = normal distribution histogram
  
  7. Performance matrix
  
  Mean Absolute Error (MAE) --> mean absolute oferrors
  metrics.mean_absolute_error(y_test, predictions)
  
  Mean Square Error (MSE) --> larger errors more obvi
  metrics.mean_square_error(y_test, predictions)
  
  Root Mean Square Error (RMSE) --> interpretable "y" units (most popular)
  np.sqrt.(metrics.mean_square_error(y_test, predictions))

IMPROVE DATA THROUGH DATA ANALYSIS

• ML phrases

  1. training

  2. inference

• Define

  1. best use cases

  2. success criteria

  3. process t production (below)

• Preprocessing raw data

  1. Data extraction: structured, unstructured, from sources

  2. Data analysis:

  df_filename

• encapsulation --> putting things in classes (boxing data fields relating to each other)

• abstraction --> methods

• inheritance

• polymorphism

to fix no implicit parent class no argument constructor

1. add no arg constructor in parent

2. explicitly add parent data to constructor

Inheritance:

• subclass can only inherit from one superclass

• subclass cannot see private of superclass

Object:

• root of the hierarchy

• override: (only in parent child context) changing the method

• overload: same method, diff parameter signature

• equals(): no override, then it will compare both memory reference

• super(): calling no-arg constructor

• ths(): calling its own no arg constructor

Polymorphism:

• require superclass object, can pass in a subclass

ArrayList:

• generic class container

• must state what datatype in <>

Abstraction:

• abstract class --> at least 1 abstract method

• interface --> all methods abstract

• data quality

• linear regression

• tensorflow playground

• neural networks

link ref for product dev: https://www.youtube.com/watch?v=TbGzhmElEus

mst - ch1-3

CHATBOT

• NLP, intent recognition

• preprocessing data

• predicts intents based on user input and confidence threshold used to determine appropriate response

• used in customer service, healthcare, finance

1. download zip folder & unzip, launch VS code with code folder

2. python extension

3. ctlr shift p

4. (@ code search top) Python select interpreter > +Create Virtual Environment >Venv >Python 3.10.5

5. Open terminal (top 3 dots) > ".\.venv\Scripts\activate"

6. "Set-ExecutionPolicy Unrestricted -Scope Process"

7. if error do ".\.venv\Scripts\activate" again

8. (ensure u see .venv) pip install scikit-learn

9. pip install nltk

10. pip install colorama

11. pip install tensorflow==2.15

12. Run model > .\.venv\Scripts\python training.py (replace training.py with py file)

13. train till accuracy = 1, model trained finish

14. .\.venv\Scripts\python inference.py (for user response)

15. add new tag, patterns (questions) and responses in intents to make chatbot respond better

16. data loaded and preprocessed in training.py?

    with open("intents.json") as file:

    data = json.load(file)

17. model = Sequential(): Creates a sequential neural network model. This type of model allows the definition of a linear stack of layers.

18. update driver see lab1 step 3.4

     

    model.add(Embedding(vocab_size, embedding_dim, input_length=max_len)): The main purpose of the Embedding layer is to map each word in the vocabulary to a unique vector in a high-dimensional space

1. casting

   • done with constructor function

   • x = int(1)

   • z = float("4.3")

2. collections(arrays)

   • list [ordered and changeable]

     -> my_list = ["apple", "pear"]

   • tuple [ordered and unchangeable]

     -> my_tuple = ("apple", "pear")

   • set [unordered and unindexed]

     -> my_set = {"apple", "pear"}

   • dictionary [unordered and changeable]

     -> my_dict = {"fruit": "apple, "color": "red"}

   • looping through: for x in arr: print(x)

3. Class and objects

   • class MyClass: x=5;

   • method: defined inside class, used to define behaviour of object.

   • init(): executed when class is initiated. assign value to object properties. (similar to a constructor) def init(self, name, age): self.name = name, self.age = age

4. Inheritance

   • private variable: __var = 0

   • public variable: var = 0

5. creating environment

   • conda create -n my_env numpy

   • coda create -n my_env python=3.3

   • conda activate env_name

   • conda deactivate

6. python libraries

   • numpy: numeric computing

     -> import numpy as np

     -> arr = np.array([])

     -> create sequence number: print(np.arange(1, 10, 2)) <2nd parameter defines max number>, 3rd defines +3>

     -> array slicing: arr_name[from:to]

     -> array stacking

     -> matrix multiplication: np.dot(a,b)

   
   

   • pandas: data structures working with labelled date

     -> import pandas as pd

     -> dataframe: pd.DataFrame

     -> reading data from cv: pd.read_csv("csvname.csv")

   
   

   • matplotlib: plotting library for 2D graphics

     -> import matplotlib.pyplot as plt

     -> plt.plot(y axis) or plt.plot(x, y)

commands/keywords:

1. range() -> loop through a set of code a specified no. of times, start by 0. e.g: for x in range(2, 30, 3): print(x)

2. def -> to define a function. e.g: def my_function():

3. reshape() -> reshape 1d array to multi d array, reshape(2,5) -> no. of dimensions, no. of elements

ArrayList<String> lis = new ArrayList<>();
lis.add();
lis.get();
for (String num : lis){
     System.out.println(num)
}

String name[] = new String[3];
int age[] = new int[3];

for (i=0;i<3;i++){
  int x = i+1;
  System.out.print("Student "+x+": "+name[i]+", "+age[i]+", "+school[i]+"\n");
}

casting

(int), (String), (float), (double), (byte)

classes

public static void main(String[] args){
}

public class ArrayList<String> numbersdis(String b){
}

exceptions

throws

random

double rand = Math.random();
int rolled = (int) (rand*max+min);
// int rolled = (int) (rand*max+min); returns 0-max

comparable

String a = a[i];
String b = a[n];
if (a.compareTo(b) >= 0) {
   //a is greater than b
}

Holders

System.out.printf("%10d%10.4f%10.4f\n", d, (float) s, (float) c);
//%10d == speciefies width for int
//%f is float place holder
//%10.4f == 10 spaces with float to 4 dp

Scanner

Scanner sc = new  Scanner(System.in);
string name = sc.nextLine();
int n = sc.nextInt();

• address bus: unidirectional, used by CPU to send out address of memory location

• data bus: bidirectional, road for data to travel, to and fro the cpu and other areas. device connected must have 3 state buffers

• control bus: tells computer R/WR activity. read: read memory location. write: CPU store memory or output data

-- increase in performace

1. processing power

   -> faster processing thus older programs can be used

2. address

   -> 16 bit address only allows 2^16 bytes of data

3. data

   -> fetch and manipulate in units of 32 bits at a time

intel processor: desktop

arm risc: mobile devices

-- embedded system characteristics

• fixued use

• small and limited resources

• failure tolerance

• real time control

• guaranteed response time

• reliable

• simple i/o

• low power consumption

• low cost

-- hardware

1. interrupts: handles slower i/o devices without processor idling

2. timers: precise timing for control tasks

   >> software delay loops does not allow processor to perform other tasks, execution time of an instruction can vary, and multitasking os cannot work on sw loop

   >> on or off chip are essential

   >> possible to read values of timer register for short duration events

3. DMA: peripheral devices transfer data to and fro memory without disrupting processor

   >> main processor goes into HOLD state where address and data lines are high impedance

   >> will generate addresses needed and keep track of bytes transferred

   >> address & data bus taken over by DMA

4. power management: modern mobile phones run on analog signals

   >> standby: memory content present

   >> hibernate: memory content move to non volatile memory

my_list = ["apple", "banana", "cherry"]

i = int(0)

for i in range(3):

print("fruit = " + my_list[i])

#8.5.2 Tuple --> ordered & unchangable

my_tuple = ("apple", "banana", "cherry")

i = int(0)

for i in range(3):

print("fruit = " + my_tuple[i])

#8.5.3 Set --> unordered (cannot be called by index) & unchangable

my_set = {"apple", "banana", "cherry"}

for x in my_set:

print(x)

#8.5.4 Dictionary --> collection unordered, changeable, indexed

#key: value

my_dict = {"brand": "Ford", "model": "Mustang", "year": 1964}

# access value items using key

x = my_dict["model"]

print(x)

for x in my_dict:

print("key = " + x) #gets keys

print("value = " + str(my_dict[x])) #gets keys

#8.7.3 continue

i = int(0)

while i<10:

i+=1

if i==3:

continue #go back to start of loop skips rest of code below

print(i) #print 1-10 skips 3

for x in range(2,30,3): # range(start, end when less than, increment value)

print(x)

#customizing parameters

# %s --> holder for string, %d --> holder for number

print('%s, %s!' % ("lol","ducks"))

def hello(greeting='hello',name='world'): # Default parameters.

print('%s, %s!' % (greeting, name)) # Format the output.

hello() # hello，world Default parameter.

hello('Greetings') # Greetings，world Position parameter.

hello ('Greetings', 'universe') # Greetings, universe Position parameter.

hello (name= 'Gumby') # hello, Gumby Keyword parameter

#9.1 class

# --> grp tgt related data and functions which act on that data

class MyClass:

x = 5

#9.2 class object

# --> unique instance of a data structure defined by its class

obj1 = MyClass()

print("obj1 = " + str(obj1.x))

#9.3 Methods

# --> functions defined inside a class body

# --> behaviors of an obj

class HiClass:

me = "celine" #variable

def say_hi(self):

print('Hello %s, how are you?' % (self))

def name(self):

print('Hello %s, how are you?'% (self.me))

p = HiClass()

p.say_hi()

p.name()

#9.4 init function

# --> Initalise function assign values to object properties

# --> like constructor

class Person: # create class Person

def init(self, name, age): #initialise

self.name = name #attributes

self.age = age

p = Person("John", 36) #create object p in class Person

#Person obj no attribute self

print(p.name)

print(p.age)

#9.5 using classes

# string.title() --> first letter in each word upper case

class Dog():

"""A simple attempt to simulate a dog"""

def init (self,name,age):

"""Initialize the name and age attributes."""

self.name = name #atributes

self.age = age

def sit(self):

"""Simulate a dog sitting when ordered."""

print(self.name.title()+" is now sitting")

def roll_over(self):

"""Simulate a dog rolling over when ordered."""

print(self.name.title()+" rolled over!")

def info(self):

print(self.name + "is %d years old!" % (self.age))

#def testing():

#print()

# Instantiate a class.

dog = Dog ("Husky",2)

dog.sit()

dog.roll_over()

dog.info()

# dog.testing() --> Type error 0 arguments but 1 self was given

• Launching into ML

• Computer Vision

• NLP

• Example project

• proj pictures

side note:

• geminni

• open government products (see linkedin Kee Wei Lam)

• RNN and Transformers

• Diffusion and GAN

• Trailblazing with Streamlit

• pytorch

• Orange and Pycaret

• Steps:

  1. data extraction (structured CSV, a text, JSON, or XML format or unstructured)

  2. data analysis (use EDA exploratory data analysis, CDA classical data analysis and Bayesian)

  3. data preparation (data transformation, which is the process of changing or converting the format, structure, or values of data you've extracted into another format or structure e.g cleansing)

• Cleansing

  • remove superfluous and repeated records from log data

  • alter data types where a data feature was mistyped / mis-converted

  • convert categorical data to numerical data

• Data Quality through data asset inventory --> relative accuracy, uniqueness, and validity of data

  • established baseline ratings for data sets compared against the data and systems to identify new data quality issues

  • Accurate == correct values that match up with the real world or events it describes for correct conclusions

  • Consistent == represented in a consistent and unambiguous form that correlates to different dataset instances

  • Complete --> Timely == expected vs ready to use production to processing time

• Ways to improve data quality (cleaning data)

  1. Resolve missing values (null values skew data)

  2. Convert date time features to a date/time format (seasonality)

  3. Parse date time features (trends)

  4. Remove unwanted values from a feature column (e.g. < , make bucket or remove it from row)

  5. Convert categorical data to One-hot encoding (create a boolean column for each category)

     
     

     
     

     Data analysis:

• EDA (exploratory data analysis)

  • problem > data > analysis > model > conclusion

  • infer what model is best fit for data

  • analyzing data sets to summarize their main characteristics, often with visual methods for graphics & simple stats

  • Outliers or anomalies, trends, and data distributions to identify those features that can aid in increasing the predictive power of ML model

  • obtain theories that can later be tested in the modeling step

  • graphics and basic sample statistics such as mean and median or standard deviation through scatterplots, boxplots, histograms, etc

• CDA (classical data analysis)

  • problem > data > model > analysis > conclusion

  • analysis of estimation and testing are focused on parameters of model

  • data collection is followed by the imposition of a model, normality, linearity

• Bayesian analysis

  • problem > data > model > prior distribution > analysis > conclusion

  • answer research questions about unknown parameters using probability statements based on prior data

EDA in ML

• focus is on the data, its structure, outliers and models suggested by the data

• Univariate analysis describes data via summaries & patterns

  • categorical

    • numerical - pd.crosstab

    • visual - sns.CountPlot()

  • continuous

    • numerical - dataframe.describe()

    • visual - boxplot(), distplot(), kdeplot()

• Bivariate analysis finds relationship between two sets of values

  • seaborn - ability to build conditional plots

    • jointplot function draws a plot of two variables with bivariate and univariate graphs

    • factorplot map method can map a factorplot onto a KDE, distribution or boxplot chart

  • category 2 category - sns.factorplot()

  • category 2 continuous - sns.jointplot()

  • continous 2 category - sns.factorplot.map(sns.kde/dist/box0)

Data analysis & visualisation

• find influential features

• EDA help find insights for data cleaning, preparation or transformation

• at every step data exploration, data cleaning, model building, conclusion

• Histogram - spread of continous sampled data - sns.set_style('whitegrid')df_file['col_name'].hist(bins=30) plt.xlabel('median_house_value)

• Scatter plot - bivariate correlation presence -

  x = df_file['col_name1']

  y = df_file['col_name2']

  plt.scatter(x,y)

  plt.show()

• Heatmap - multivariant correlation, lighter shade = stronger - sns.heatmap(df_filename.corr())

-- ML IN PRACTICE --

Supervised learning

• modelled labelled examples to predict future values using predictors

• relation or key feature known

• row = example, columns = characteristic label / features

  • regression - continuous

  • classification - discrete value / classes

Unsupervised learning

• unlabeled data

• discovering raw data groups by looking for clusters etc

Linear regression

• (continuous) 1 group that spans across range

  • (1) get linear slope to find rate of relation to multiply features for future prediction

  • (many) generalisation of a line to get continuous value for lable

  • minimize error between label predicted and actual using mean-squared error (reduce euclidian distance between predicted & actual)

• (categorial) - classification linear groups linearly seperate

  • decision boundary to separate different classes

  • hyperplane in higher dimension with classes on either sides

  • minimize misclassification error between predicted and label class through cross-entropy

  • can discretize by forming groups using the ranges as classes

  • BigQuery stores data and retrieved with sql

Both use 2D hyperplane to get better splits

Logistic regression

• predict probability instead of binary yes or no answers

• take linear model and squish it through a sigmoid activation function for probably 0 to 1 with a calibrated probability estimate of training data occurrence

• gradient descent seeks to minimize cross entropy, it pushes output values closer to one for positive labels and closer to zero for negative labels

• regulation is important to prevent driving loss to zero

• sigmoid function asymptotes to zero when the logit is negative infinity and to one when the logit is positive infinity

• weights is increased and increased, leading to numerical-stability problems, overflows and underflows bad for training

• saturation when gradient becomes 0 and training stops, vanishing gradient makes training difficult --> overfitting

• regulation in logistic regression helps stops weights being driver to +- infinity & help logistics avoid asymptotes that halt training

• counter overfitting via regularization & early stopping (loss epoach graph)

• overfitting hurts generalization & regularization increases bias underfitting hurts generalization

logistic regression predictions should be unbiased

• ave predictions == ave observations (no shift of probability)

Receiver Operating Characteristic curve (ROC)

• chooses decision threshold based on decision criteria

• binary decisions based on TP / FP rates

Area Under Curve (AUC)

• average measure of performance across possible classification thresholds

• help choose between models for model that scores in correct relative order

• scale-invarriant & classification threshold invariant & precision recall curve

Calibration scatter plot

• bucketized bias to find slices where model poor performance of over or under predicted (locate area that cmi and fix it)

• true or false for real world but prediction is 0 to 1 so ave a large grp sum for more accurate probability

-- TRAINING AUTOML THROUGH VERTEX AI --

• ML starts with requirement or problem to solve

• ML pipline workflow

• ML uses lots of data (train test validating) and keep outlier, builds models (more holistic picture of data)

• Statistics throw away outliers

• Statistics for testing hypothesis

• Deep learning is supervised learning, large data, GPU, high accuracy, longer to train, tunning parameters

Loss functions (model performance)

• models are defined as math functions using parameters & hyperparameters

• Analytical methods for finding the best set of model parameters do not scale

• optimizing parameters searching through parameter-space

• loss functions take training set composed single number for model quality evaluation

• Measure of quality:

  • Error difference -> sign difference between actual value - predicted

  • Compose a sum -> put error values together

  • Root Mean Square Error (RMSE) -> minimize RMSE better quality predictions (continuous, not classification)

  • loss functions help find best fit linear model predictions for underlying data for better model (weight and bias choice)

  • Cross entropy -> penalizes bad predictions very strongly by finding best parameters (classification best)

Gradient Descent

• turn loss function (topographic / contour map) into search strategy to find minima by descending(like climbing down a mountain)

• Which direction should I head?

• How large or small step?

  • too small training take forever

  • too large, overshoots, not guaranteed to converge

• Epsilon -> tiny constant

• while loss > epsilon, parameter = old value + (step size * direction), recompute loss

• loss function slope (finding minima where loss function slope = 0)

  • direction -> slope negative go right

  • Size -> magnitude slope

• empirical performance problem theorized good but dose not work as well

  
  

Troubleshooting loss curve

• Loss/time curve, big drop then smaller steps closer to minima

  • looks like seismicseismic activity -> overshoot

  • looks like going no where -> step too small

• learning rate set before beginning -> hyperparameter

• hyperparameter tunning ->method to set best hyperparameters

• learning rate is fraction <1

  
  

ML model pitfalls

• running same hyperparameters model twice resulting parameter different

• could have more than 1 bottom -> convexity -> non-convex

• 1 bottom -> convex

• too slow

  1. train loop (calculate derivative, data points & parameters)

     • cost of the calculation is proportional to the number of data & number of parameters

  2. Take Step (num of model parameters)

  3. Check Loss (num of data points, model parameters & freq)

• improve training time by

  1. turning knob of num of data points to cal derivative of

  2. frequency of checking loss - readyToUpdateLoss function are time-based and step-based

• samples == batches = sampling with uniform probability

• mini-batch = batch size

• model training: changing our model's parameters and checking to see when we've made the right changes

  
  

Performance metrics

• Loss functions problems: long training times, suboptimal minima and inappropriate minima

  • inappropriate minima -> points in parameter space that reflect strategies either that won't generalize well, that don't reflect the true relationship being modeled or both -> e.g. predict every category available not possible

  • gap between the metrics we care about and the metrics that work well with gradient descent

  • perfect loss function would minimize the number of incorrect predictions where it takes integers not real world values

  • need differentiability with respect to loss

• performance metrics easier to understand & directly connected to business

• e.g. confusion matrices, precision and recall

Confusion Matrix -> quantifiably assess the performance of our classification model

• Precision - how it did with true positive predictions but could miss false neg

• Recall - how it did with false positives

-- SAMPLING & GENERALISATION --

Generalization & ML models

• accurate ML model is not always your best choice

• mean square error tells us how close a regression line is to the set of points from it

• A more complex model could have more free parameters -> overfit

  • to solve - split data, done with training, compare its performance against an independent siloed validation dataset

  • train till not perform well against validation, loss metrics start to increase or creep up, it's time to stop

• overly simplistic linear model that doesn't fit the relationships true to the data -> underfitting

• add data test set for validation to loss function

When to stop model training

Implementing Linear Models for image classification:

keras -> high level API, core data structure consists of layers and models

simplest type: sequential model which groups a linear stack of layers: output of 1 layer is input of another layer.

Cannot work with multiple input types (images, texts at the same time) or multiple outputs (regression, classification) at the same time. need to use functional API.

Keras model:

1. Define and create model

   Linear model: each image is flattened by unstacking pixels then produces 5 outputs. weights are only created when model is first exposed to input data.

   
   

   Flattening layers: transforms image format from 3D array to 1D array. No parameters to learn, and only reformats the data.

   
   

   softmax -> exponentiates its inputs and normalise them (higher values higher, lower values lower)

2. compile model

   Optimiser (updates model based on data it sees), loss function (how accurate model is during training) , metrics (monitor training and testing).

   
   

   optimisers: methods to update model weights to reduce losses, and adjusting learning rate

   
   

   loss function: tells optimiser when its in the right or wrong direction. classification taks: >=2 label classes

   
   

   metrics: identify areas for improvement in training.

3. print a string summary of NW (OPTIONAL)

4. call mode.fit() -> train model for no. of epochs

   training dataset is to train model, validation is for report accuracy metrics.

5. evaluate performance (OPTIONAL)

   callback: object that can perform actions at various stages of training

6. generate predictions

   model.predict() -> expects a batch of images, thus need to reshape image tensor as a batch consisting of only one image.

   
   

   predicted label from model is the label in which has most confidence. can convert logits to probabilities by softmax (tf.math.softnax)

7. plot predictions (OPTIONAL)

NN and DNN for image classification

linear model: single layer neural network (1 fully connected layer)

Y = softmax(B + W(B + WX))

need to connect many processing units into a neural network, each computes a linear function if its followed by a nonlinear

Activation funciton -> separates linear models from NN

• (ReLU helps NN to learn nonlinear states with real world info of visual inputs)

• returns 0 if -ve input

• returns value if +ve input

one hidden layer = one dense layer before output layer

evaluate model performance: plot loss and validation loss using training_plot function

-> if loss doesnt go down smoothly: training parameters can be improved

-> similar to what is obtained in linear model: model stuck searching for function to sufficiently map inputs to outputs (try improving)

Improve model performance:

• adding regularisation

• changing learning rate

• apply early stop

DNN dropout and Batch Normal

Universal approximation theorem: any continuous function defined in the n-dimensional unit hypercube can be approximated by a finite sum of the type. only 1 single hidden layer and approximate any function (meaning theoretically, no ML task a network cant solve.)

DNN -> more than 1 hidden layer: each added layer, no. of trainable parameter increases

large: more memory, smaller batch size, slow optimisation, slower to decide, overfit training data

size of NN: implicit regularising effect, preventing large nets compared to the dataset.

prevent overfitting in DNN:

• regularisation

  • dropout (only used with NN)(use to avoid overfitting in DNN)

    only used during training

  • L1 - model sparser by dropping poor features

  • L2 - keep weight value small

collection - stores elements

map - stores key-value pairs

-- Collection--

• Set

  
  

• Sequenced Collection

• Queue

-- Array List --

ArrayList a = new ArrayList<>(); 
a.add("Ang Mo Kio"); 
a.add("Dover"); 
a.add("Changi"); 
a.forEach(e->System.out.println(e.toUpperCase()));

-- Linked List --

-- Doubly Linked List --

-- Stack --

-- Methods --

-- Iterators --

-- Queue & Dequeue --

-- PiorityQueue --

-- Set --

• HashSet

• TreeSet

• LinkedHashSet

conda create --name myenv python=3.10

conda activate myenv

pip install streamlit

pip install tensorflow

pip install streamlit tensorflow

streamlit run rps_app.py

or just

cd C:\Users\celin\OneDrive - Singapore Polytechnic\Desktop\AIML\Pancakes_doughnuts

conda activate myenv

streamlit run pandou_app.py

OCR -> Optical Character Recognition

Types:

• Image classification

  • single image with 1 label for whole image

• Semantic segmentation

  • no labels, partitions into multiple regions and labels pixels with different labels based on category

• Instance segmentation

  • identifies boundaries of object from image segmentation

• Image classification with localization

  • assigns bounding box around single object and labels it

• Object recognition

  • outputs class labels & class probabilities of object prediction if object is present but not where the object is

• Object detection

  • assigns bounding box around multiple object and labels it

• Pattern recognition

  • detects and identifies repeated shapes colors other visual indicators

  • movement, OCR, medical imaging recognition etc

• Facial recognition

  • identify multiple faces, key facial attributes, emotional state, headwear etc

• Edge detection

  • initial step for object recognition

  • identifies boundaries of objects using changes in brightness and intensity levels

• Feature matching

  • compares features of images that differ orientation perspective lighting sizes and color

  • auto object tracking, £D object reconstruction, robot navigation, indexing use case etc.

Vision API-pre-built ML models

• image classifiers use DNN or CNN

• pre-built ML API

  • fastest -> great starter

  • uses less new inputed data

    • Vision API - google image dataset (logo, face detection, object, printed hadwritten text and builds metadata into imgae catalog)

    • Speech-2-Text - Youtube captions

    • Translation API - parallel text language translation

    • images videos & text data already there just need to "mine" the text

    • video intelligence API - run videos through for labeling

• BigQuery

  • structured data already stored in Big Query -> second fasted

  • built in SQL, linear logistic regression models

  • cannot help with computer visions tasks

• AutoML Vision

  • codeless -> google cloud console

  • own defined labels to train ML models

  • link or upload training set for custom

  • human labeling program

• Custom models

  • custom lol needs a buck ton of data

• start simple -> add complexity when needed

Linear Model

• work well for both binary and multiclass classification

• SoftMax or sigmoid layers help ensure linearity

• predicted Y = Weights * sum of inputs + bias

• bias -> difference between average prediction & correct value

• images represented as pixels of 2D arrays of numbers

• image class model use matrix-vector product taking multiple outputs inputs and parameters

• linear model every input is connected to every output

• fully connected -> dense layer

• input layer not counted

• paramters: height x Width x Nclasses + Nclasses

Reading Data

• JPEG or PNG standard format decoding by:

  1. Read -> construct dataset from data in memory contents as are (no parsing) use img = tf.io.read_file('filename')

  2. Decode -> convert input bytes string into uint8 tensor using tf.image.decode_jpeg(img, channels = X)

     • encoded images represented by scalar string tensors

     • decoded -> 3-D uint8 tensors of shape [height, width, channels]

     • grey scale -> 1 channel, RGB -> 3 channels

  3. Convert -> 3D uint8 of image RGB intensities to float in the [0,1] range using img = tf.image.convert_image_dtype(img, tf.float32)

  4. Resize ->use new_img = tf.image.resize(img, new_width, new_height) or crop to desired size or pad data with 0s

• Tensor - mathematical objects defined by num of directions & dimensionality

  • scalar -> zero-rank tensor

  • vector -> first rank tensor

• visualising image data exploration using Matplotlib to understand data

  • display image use img = read_and_decode(filename, [height, width]) function and plt.imshow(img.numpy())

  • imshow() creates 2D numpy array need numpy to convert tensor image into numpy array

• efficient input pipeline

  1. Reading entire dataset use tf.data.Dataset

  2. dataset transformation pre-processing

  3. iteration dataset to process elements -> streaming fashion, memory efficent (dataset no need fit into memory)

• list of all images use tf.io.gfile.glob("file_location")

• label images

  1. tf.strings.regex_replace function to remove the prefix

  2. label = tf.strings.split(basename, '/')[0] function to get the class name.

• process records TFRecordDataset

• create dataset of files matching a pattern use tf.data.Dataset.list_files

• read fixed-length records from binary files and create each record an element of the dataset, use tf.data.FixedLengthRecordDataset()

• decode_csv(csv_row) -> extract filename to read img, return img & label representation (not hot-encoded)

• map(decode_csv).batch(10) -> handles each read line

• batched as optimizer class expects it (batch both training and evaluation)

Implementing linear models for image classification

• increase complexity till performance criteria are met

• simpler models -> less likely overfitting

• train NN opp direction by choosing a large model to interpolate data then do regularization

• high level keras instead of low level tensorflow more convenient

• keras

  • highly productive focus on DL

  • core structure layers & models

  • tf.keras -> sequential model (list of layers outputs connected to next layer input) grps linear stack of layers -> no sequential if multiple input types concurrently

  1. Define & create model

     • Sequential([ tf.keras.layers. Flatten(input_shape=(IMG_HEIGHT, IMG_WIDTH, 3)), tf.keras.layers. Dense(len(CLASS_NAMES)) ]) -> linear model, each image is flattened by unstacking its pixels.

     • outputs are a weighted sum of input pixel values

     • no weights when model is created, initialised only when built & exposed to input data then have weights or pass input_shape in first layer

     • hence implicit input layer expects 3D img tensor

     • model created initialized no weights, until built & exposed to input data

     • Flatten layer (reformat) -> 3D image tensor as an input & flattens (unstacking rows of pixels in the image and lining them up) to same number of values 1D array

     • Dense layer -> dense operation cal output as activation of weighted ave of flattened input pixel added by bias -> activation function used to determin output of equation

     • Activation function -> determine output of equation by maping resulting logistic values between range probability

     • Softmax -> converts vector to probability distribution (exponentiates its inputs then normalisation) keeping relative order for valid probability distribution

  2. compile model to configure training (optional print string summary of network)

     • Optimizer -> how the model is updated based on data & lossfunction

       • e.g. Adam optimisation: stochastic gradient descent

     • Loss function -> measures accuracy of model during training, minimize this for accurate

       • tells optimizer when its moving in right or wrong direction

       • computes quantity to minimize in keras model

       • 2 or more label class classification -> use tf.keras.losses.SparseCategoricalCrossentropy() -> provide labels as integer

       • one-hot representation to provide labels, use the CategoricalCrossentropy loss

       • translate for business user meangingful

     • Metrics -> monitor training & testing steps

       • areas for improvement in training data

     • model.complie() for configuration

     • model.summary() -> display model contents

  3. Call model.fit -> train model for a fixed epoch (optional evaluate the performance)

     • training and validation datasets

     • evaluate model by plotting history of loss & error metrics

     • callback -> object that perform action at various stages of training

       • e.g. History callback, history.history.keys() -> records training metrics per epoch as loss, accuracy, validation loss, or validation accuracy

       • looking for smooth curve decent validation loss curve, else, training parameters (batch size, optimizer, learning rate, loss function) needs improving

       • insight for slope -> speed of convergence over epoch, status of convergence & overtraining

  4. Generate prediction for input samples using model.predict()

     • model.predict() expects a batch of images, so you need to reshape the image tensor as a batch consisting of only one image

     • Logits -> predicted confidence values -infinity to +infinity

     • predicted label is the one with most confidence

     • convert logits to probability by applying softmax, tf.math.softmax

     • evaluate:

       • accuracy - fails when one of the classes in the dataset is represented by a very rare example

       • Plotting predictions -> what the model has learned, plot predictions of model on a few images from the evaluation dataset by defining a custom plot_predictions() function must be batch data

• Functional API -> handle model with multiple inputs or outputs shared layers, non-linear topology

• DNN -> many hidden layers stacked together for activation to get better results

  • requires more data points to improve accuracy

  • improve by adding regularization, increasing amount of dataset, change learning rate or apply early stop

  • Improve with dropout layers

  • Improve with batch normalisation

  • not affected by the pixel randomization, because the data is not structured hierarchically

  • DNN has convolutional & pooling - sequencing, combining them with traditional neural network architecture improves the performance of image recognition systems

• NN -> input layer > hidden layer > output layer = linear network + more dense layers

  • feature engineering responsibility of model

• layer = nodes a bunch = nodes in one layer connected to another node in the next layer with its own weight = each node own linear regression model, composed of input data, weights, a bias (or threshold), and an output.

• input layer’s -> nodes take the values of the input features

• output layer -> one unit for each value that the network outputs

• hidden layer -> trainable weights not i/o layer

• wrapping multiple layers - creates more linear equation, need non-linera connected as well for neural network

• Activation function separates linear from NN

  • non-linear component in the form of an activation function identifies non-linear relationship

• Rectified Linear Unit (ReLU)

  • help NN learn non-linear states with visual inputs

  • return 0 if input negative but returns the value of positive magnitude

-- CNN --

Convolutional Neural Network

• multilayered NN called the neocognitron inspired by brain's simple (extracts local feature) and complex cells (extracts deformation / local shift) -> recognize patterns by learning shapes

• LeNet (CCN structure) -> handwriting recognition by aggregating simpler features into progressively more complicated features

• AlexNet won annual ImageNet competition for labeling pictures with increased accuracy

• CNN standard for computer vision

  • three layers -> convolution, pooling, fully connected

  • Convolutional layer detect patterns in multiple subregions of the input field by using receptive fields to learn parameters

  • to learn a hierarchical structure based on locally correlated elements on visual inputs & audio, time series, signal data

  • hierarchy -> how pixels are placed next to each other vital in CNN

  • reduces need for feature engineering and increase performance

  • Weights of general-purpose filters to be learned during training itself

  • Input > Convolution > Pooling > Convolution > Pooling > Fully connected > Output

• MPs = megapixels -> smartphone screen 12-16 MPs, too many weights cos of DNN density

Understanding Convolutions

• convolution -> sliding kernel across an image -> mathematical combination of two functions to produce a third function -> merge 2 input sets for collection of input elements

  • where the model learns

  • detect horizontal edges or brighter or darker spots

  • 0 ignore, +num brighter bigger magnitude, -num darker bigger magnitude

• processing a group of pixels that are located close to each other

• 1-dimensional CNNs are used on time series, audio, and text data,

• 3-dimensional CNNs are mostly, 3D image data such as video, magnetic resonance imaging (MRI), and computerized tomography (CT) data

• 2D CNN image classification (greyscale) 3D for GGB stacked

• Kernels -> building blocks of CNN to extract right and relevant features from inputted data, parameter-sharing concept to CNNs

  • weights or numbers in kernel represent the feature to detect

• 2D CNN -> slides along 2 dimensions of the data

• each square 5x5 so 25 input values dataset or 25 weights associated with kernel, each pixel value is multiplied by the corresponding weight

• convolved feature -> convolution operation sum of products

• feature map -> weight each square from shared kernel's sliding window across image in both directions

• if square kernels are used, expect output shape smaller

• TPU GPU - fast - compute the values of the convolution operation for all positions of the convolution kernel in parallel

  
  

CNN Model Parameters

• ML model parameters -> values learned from training data during training used to determine input data transformation into predicted output

  • output image width = ( (input image width - kernels size width + 2*Padding )/ stride ) + 1 -> O = ( (I - K + 2P) / S ) + 1

  • Number of filters - number of independent filters to apply to the input

    • number of output image channels = number of filters

  • Number of input channels - number of input image channels

  • Kernel size - is dimension of a filter, pick a kernel size that is used across all filters

  • Strides - the size of step the filter slides across the input image (default size is 1 pixel both directions)

    • larger step will skip input pixels produce Fewer output values, Reduce computation required

    • too high lose data, reduce output dimension

  • Padding - maintain the same size across the input and the output of the convolutional layer by adding border around input value using 0s

    • thickness depends on size of kernel

    • keras has bulit in calculations

    • padding has same or valid type

  • Activation - specifies the name of the activation function you want to apply after performing the convolution for non-linearity

• trainable parameters increase as complexity increases

  • Convolution layer parameters -> calculate learnable para by multiplying

    • ((width of filter x height of filter x prev layer num of filters) + 1 the bias term ) x current num of filters

  • Pooling layer ->0 parameters, calculates specific num no para

  • Fully connected layer -> every node is fully connected to every other neuron on layer most parameters

    • ((current layer neurons (c) previous layer neurons (p))+1 current layer neurons (c))

• tf.keras.layers.Conv2D() -> convolutional layer use relu for activation function

  • relu popular as it does not activate all nodes at the same time

• 4D tensors are expected as inputs and output of convolutional layer [batch, height, width, channels]

Working with Pooling layers

• Pooling -> reduce the dimensionality of the previous input or hidden layer

• v num of para to learn, v the amount of computation network performs

1. Max Pooling - returns the maximum value out of all the input data values passed to a kernel used in combination with the convolutional layer

   • maximizes reduction of dim, no weights, no parameters changed

   • tf.keras.layers.MaxPooling2D(pool_size(), ...)

2. Average pooling - calculate the average value for each block on the feature map

   • less extreme output than max pooling

3. Global average pooling

4. Min pooling

   
   

CNN on Vertax AI with pre-built TF Container using Vertex Workbench

• dense too many weights convolutional channels less weights

• separate an image’s features from its dimensions

• must have a method to stop recursive

• if recursive function is called in itself with other instructions below it, the unrun instructions below will be pushed into stack (first in last out)

• Factorial use recursive nature

• can stop using general (reducing to smaller verssion of itself) or base case (solution obtained directly & stop)

• Tower of Hanoi

• Fib num

• Big O notation

  • drop constants, only interested in trends

  • for loop is linear -> O(n)

  • only capture fasting growing trend

  • n^x , x being the number of nested for loops -> quadratic e.g. O(n^x)

  • constant e.g. O(1)

  • function of for loop log -> O(log2n)

Preprocessing Image Data

1. create Dataset - training dataset

   • collection method to get data - internal resources or 3rd party

   • validate & annotate dat

2. Reading the dataset - validate data into model

   • csv not recommended only standard img

   • slow reading each img file

   • use tf.records - fast I/O operation

   • tf.data.TFRecordDataset build complex input pipelines from simple, reusable pieces

3. Preprocessing dataset - img tensors have expected shape for input layer, keras preprocessing layers automatically applied durring prediction

   • Resizing - downsizing img increase efficency, convolution faster on less px, tf.image.resize(), must keep aspect ratio,avoid distortions use tf.image.resize_with_pad or learned resizers

   • Converting between color spaces

   • Cropping

   • Flipping

   • Rotating & Transposing

   • Segmentation

   • Compress

Model Parameters & Data scarcity Issue

• more complex model, more para, need more data

• insufficient data:

  1. Data Augmentation - creating more data

     • Instead of picking a random point in feature space, you can pick points strategically

     • points in regions that are homogeneous with respect to class

     • unstructured data, these sorts of neighborhoods don’t exist

     • determining which assumptions your data will support

     • think about how the transformation will affect the model’s ability to learn and the sources of information it requires to do its task

     • Dataset.map to make augmented datasets

     • input elements are independent of each other, the preprocessing can be parallelized across multiple processors

     • train the model longer whenever you augment data, due to the increase in data size

  2. Transfer learning - making the model more data efficient, thus reducing its need for data

     • initializing the parameters with better values

     • journey through weight space, optimizer looks for

       best values for the weights in the model to minimize the expense of data and time

       • by training from scratch - too long time

       • source model - abundance of data and a similar task and then transfer this model to our task

     • shape errors if your labels aren’t the same shape as the ones used to train the model

     • isn’t a 1:1 match between each of your classes and those on source model's prior training might actually undermine performance.

     • replace them with newly initialized parts that are related to your specific use case

     • layers near output most closely aligned to source task

     • make source model weights trainable if dataset small, NN not overfit

• text 2 speech / speech 2 text

• text classification (predict label to doc, sentiment analysis e.g. rating based on comments, spam identification, topic of content)

• entity extraction (relevent specifics like topic, date, location)

• machine translation (gg translate 133 languages)

• interactive conversation (google assistant interprets intentions to conduct conversation)

• end 2 end platform via VertexAI

  • autoML - no code

  • custom training - code based + TensorFlow libraries

• NLP onto gg search, 10 years developing and applying

  • word2vec breakthrough neural network in text representation

  • Attention mechanism - identified the most relevant section

  • Transformers -> Pre-trained language models

  • BERT (Bidirectional Encoder Representations from Transformers)

  • T5

  • PaLM (Pathways Language Models)

• Smart Reply response (Gmail)

• Auto summary (google docs , chat, meet, workspace)

• NLP = linguistics + AI

NLP is

• computer to understand human language

• using unlimited mem (cloud store) & comp power (Tensor Processor Unit)

• input (natural human language text or speech) > process (text representation > Process method) > output (Text classification, entity extraction, machine translate, interactive conversation)

  1. text representation -> translating HumLang to CompLang

  2. Process method

     • Rule-based reasoning -> action from hand-coded rules e.g. dictionary lookup

     • ML -> learn the rules by itself (more scalable)

• NLP is multidisiplinary

  • linguistics

  • comp sci

  • AIML

  • stats

• NLP uses sequential models

  • one to sequence -> one input is passed in, sequence output generated (e.g. image captioning - image in sentence out)

  • sequence to one -> sequence in, one output generated (e.g. sentiment analysis - comment in 1 rating out)

  • sequence to sequence -> sequence both in & output (e.g. google translate)

History

• Before ML era 1940-2001 > statistical reasoning / hard-coded

• After ML era

  • NLP models

    • NN to predict next word given previous word

      • RNN recurrent neural network

      • LSTM long-short term memory - variant of RNN

      • attention mechanism - solves blockage problem of fixed-length encoding vector from prev models, ^ model performance by focusing on relevant task info only

      • large pre-trained language models - from attention mech architecture to pre-train large data and parameters to be fine-tunned after 4 specific tasks

      • triggered libraries like Transformers, BERT T5, GPT3, PaLM

    • word embedding -> vector to rep given words

    • text representation

  • computer hardware -> multi-tasked learning (solve>1 task e.g. entity recognition, topic classification based on shared parameters set)

NLP architecture has 3 layers

1. NLP solutions

   • Horizontal solutions - any industry same problem

     • Document AI (Doc AI) - NLP, OCR comp vision, pre-trained ML models for high-value, high-volume documents (processor is key)

       • Input: unstructured data, emails, images, docs & PDF files

       • Output: provides structures (identifies data fields e.g. name) to make the data easier to understand, analyze & consume as JSON file

       • Can: convert img 2 text, classify documents, analyze extract entities, produce new insights

       • Doc (detailed instructions send processing request)> Processor (choose/create)> Human review (2 evaluate output accuracy/interpret result/solve overlooked issues) > Database/analytics (further insights)

       • Processor existing

         • general - starting point ready 2 use, extract simple information, low quality in comparison, most affordable

         • specialised - ready 2 use, handle 1 type of document (e.g. parse W2 forms/rental contracts), best quality most expensive

       • Processor create (custom ~ build yourself/with GG) - parse specific format, prep data > GG manage remaining work, tedious, Intellectual Property (IP) ownership only build it yourself

       • prediction end-point 2 send the doc

       • see NLP solutions @ 08:02

     • Contact Centre AI (CCAI) - ^ operational efficiency with minimum AI skills by integrating 2 existing infra, key features:

       • Automatic Speech Recognition (ASR) / Speech2Text (SST) -> capture speech & process text

       • NLU -> interpret human words & identify intent

       • Talks via TTS (Text2Speech) from text response 2 audio stream or playback file

       • Personalization -> branding, local region etc, generate diff voices, accents

       • one-click integration -> telephony providers

       • Agents: Dialogflow, Agent Assist (AI help human agent), agent Insights (analyse convo uncover meaning 2 suggest what needs improvements)

   • Vertical industry solutions - 1 industry

     • Lending DocAI (auto mortgage doc processing 4 home loans)

     • Retail Search (retailers platform ^ search quality retain sales conversion instead of search abandonment)

2. NLP AI development platform

   1. pre-built API

      • low expertise required, no data required, no tuneable hyperparameters, no training needed

      1. Dialogflow API - platform conversational user interface using NLUnderstanding 2 analyse multiple inputs (text/audio) for an integratable human-like-agent engagement response (text/synthetic speech) on bots, apps, etc trained via past conversation/existing knowledge database deployed with 2 services Dialogflow:

         1. ~ Essentials (ES) - standard small simple virtual call centre agent

            • only 1 agent per proj

            • building blocks: flat structure of intents

            • console: text forms format mostly

            • quick & affordable

         2. ~ Customer experience (CX) - advanced large/complex agent

            • up to 100 agents per proj, diff types like sales agents, marketing agents & customer service

            • building blocks: graph structure flow & pages

            • console: visual graph conversation path & text forms 4 configuration

            • better scaling across multi dev teams

         • Parts of Dialogflow

           1. Intent -> topic e.g. reminder, weather, check acc bal - data from convo to build popular intents

              • matches end user expression to intent via ~v simultaneousl, choosing best result

                • rule-based reasoning - decision reasoning

                • ML matching - use model 2 learn from alot of convo to identify & match keywords intents

           2. The entities -> specific details of topic, 4Ws, types ~v dictate how data is extracted

              • system entities - pre-defined common data types like date, time, color, email

              • custom entities - match custom data e.g. vegtable entity 4 grocery store agent

           3. Flow of conversation -> NL context e.g. they (find context on what the end user is referring 2)

              • configure context by setting I/O context

              • Input context - control intent matching while context active likely match

              • Output context - control active contexts

         • End user (I/O) > Dialogflow (integration/API services) > Fulfilment (external API/webhook service/database)

           • default agent response is static response

           • integration option more dynamic response through fulfilment of self defined actions

      2. Natural Language API - NLU 4 sentiment analysis, entity analysis, entity sentiment analysis, content classification & syntax analysis

      3. Healthcare NL API - real time analysis of insights stored in unstructured medical text to machine readable data

         • AutoML Entity Extraction 4 Healthcare - easy custom knowledge extraction models without needing coding skills

   2. Vertex AI - AutoML

      • low expertise required, small 2 med data set required, no tuneable hyperparameters, long training needed

   3. Vertex AI - custom training with Workbench

      • high expertise required, med 2 large data set required, tuneable hyperparameters, med 2 long training needed

3. NLP AI foundations (base layer)

   • google cloud infrastructure - auto, no need to manage and scale below tasks

     1. compute & storage - decouples 4 independent scaling of resources

        • compute breakthrough - TPU gg's Application-Specific-Integrated-Circuits (ASIC)

          • accelerate ML workloads

          • ^ efficency by tailored archie 2 meet computation needs in a domain (matrix multiplication ML)

          • cloud TPUs - integration across gg products 4 supercomputing

     2. Networking & security - base base

   • Data engineering

     • BigQuery - analyse uncover data insights

     • Dataflow - ingest process batch & real-time data

VERTAX AI

• production challenges (only 1/2 projects pass pilot phase)

  1. scalability

  2. monitoring

  3. continuous integration & deployment

• ease of use challenges of current tools

  1. requires advancing coding skills

  2. requires ML NLP expertise

  3. No unified workflow (tools not compatible with each other)

• Vertex AI -> the solution unified model creation, deployment, manage, over time & @ scale

• Pipeline

  1. Data readiness - upload data from wherever

  2. Feature readiness - create features, for the processed data 2 put into model sharable via feature store

  3. Training & tunning hyperparameter - once data is ready, users experiment with diff models and adjust hyperparameters

  4. Depolyment & model monitoring - pipeline 4 model 2 production by auto monitoring & performing continuous improvements

• Benefits

  1. Seamless - user experience from uploading, prep, train 2 production

  2. Scalable - MLOps 2 monitor & manage ML production, auto scale storage comp power

  3. Sustainable - artifacts & features cand be reused & shared

  4. Speedy - 80% fewer codes

• Vertex AI Feature Store - centralised repository for org, storing, serving features 2 feed training models

• Vizier - tune hyperparameters in complex ML models

• Explainable AI - interpret training performance & pipeline 2 monitor ML production line

NLP WITH AUTOML

• no code, automate tunning hyperparameters & comparing multiple models

• quickly prototype models & explore new features of dataset before investing in dev

• diff data type diff sol problem objective, accepts data type of

  1. img

  2. tabular,

  3. text (NLP most)

     • list categories -> Classification Model

     • label entities -> Entity Extraction

     • identify prevailing emotion -> Sentiment Analysis

  4. video

• AutoML can combine diff objectives to sol bussiness prob

• using:

  1. Transfer learning -> use pre-trained model with current smaller dataset or less computational power, higher accuracy

  2. Vertex AI Neural Architecture Search -> find optimal model through multiple comparision

     
     

NLP WITH CUSTOM TRAINING (AI Workbench)

• full control of training & deployment

• Workbench - notebook tool has libraries

• end2end workflow upload>train>deployment

• determine env ML code uses

  1. pre-built container - fully built dependencies & libraries (TensorFlow, PyTorch, Scikit-learn, XGBoost & py code 2 work with the platform)

  2. custom container - user defines exact tools needed

NLP END2END WORKFLOW

• 1 unified env 2 build custom ML models

1. Data Prep - upload new labelled or unlabelled data (NLP text from cloud storage or local machine) > feature engineering (process data)

   • label is a training target

   • Labelled

     • Sentiment - manually added / paid label service Vertex console

   • Unlabelled

     • underlying pattern of texts and group similar documents - unlabelled, CLUSTER ANALYSIS

     • co-occurrence of pairs of words or phrases - LATENT SEMANTIC INDEXING (LSI)

2. Model Training - iterative training & evaluation

3. Model Serving - deployment & monitoring to move into production

   • deployment types

     1. Endpoint - low latency, real-time results needed, deployed 2 endpoint bef use

     2. Batch prediction - process accumulated data with single request, no immediate response

     3. Offline prediction - outside cloud, model deployed in specific env

• NLP workflow is iterative

  • investigate the raw data

  • generate more useful features

  • data drifting -> accuracy of your predictions suddenly drop

    • check the data source & adjust model parameters

  • automated with MLOps

-- TEXT REPRESENTATION --

Tokenization

• represent text in numeric format while retain meaning

  • tabular data > binary code > num

  • img > px > num

  • audio > wave > amplitude

• feature engineering

  • equipped with libraries like TensorFlow & Transformers hide details of this step

  • no unified way to name steps - relate terms to specific functions

  1. Raw Text -> read in chunk of text

  2. Tokenization -> divide text chunk into language units / tokens

     • strategies for diff lang

     • how comp read lang

     • types: character tokens, subword tokens (og word & tense), word tokens, phrase tokens, sentence token (by punctuation)

     • use type based on model & libraries

  3. Pre-processing -> keep the main tokens & remove punctuation

     • Lowercasing - all text data to lower case

     • Stemming - root form remove tense (e.g. running to run)

     • Stopword removal - remove low information word (e.g. a, the)

     • Normalisation - 2 standard form (tmr to tomorrow)

     • API TensorFlow tf.keras.layers.TextVectorization

       • maps text features to integer sequences\

       • includes preprocessing, tokenisation, vectorization

  4. Text representation -> transform tokens into representative num as vector, comp understands input

     • Basic Vectorization

     • Word Embeddings

     • Transfer Learning

  5. NLP model Training

• problems:

  • retain text meaning in num format through indication of words relation numerically (convey meaning)

  • text > num > ML model where input needs dense matrix vector or overfitting (efficient)

• Basic Vectorization

  • one-hot encoding -> encode each word by order into vector

    • vector size = vocab_length x vocab_length

    • 1 corresponds to word position rest 0

    • Pro: intuitive, easy 2 implement

    • Con: no relationship conveyed, sparse vector

  • Bag-of-words -> bag of vocab, vector of freq / occurrence token appears in sentence

    • vector size = 1 x sentence_length

    • Pro: intuitive 2 implement, captures some semantic similarity (similar vocab likely similar meaning), matrix less sparse

    • Con: high dimensional sparse vector, no capture word relations

Word Embeddings -> convert word into its properties into dimensions

• convey meaning through vector space - where dis between similarity of word meanings are dimensions

• low dimensional dense vectors

• uses word2vec, GloVe & FastText

• Pro: low dimension dense vector, captures distribution & meaning, vectors trained by NN no manual work

Word2Vec -> context to predict words

• family of model architectures & optimizations 2 learn word embeddings from large datasets

• corpus - collection / body of text

• expensive 2 use

• only need import embedding from tensorflow keras

  • embedding layer (lookup table maps one-hot encoded specific words 2 dense vector embeddings)

• Hyperparameters

  • dimensionality (width)

1. Continuous bag-of-word (CBOW) -> predicts centre word given surrounding context

   1. prepare data

      • slide a window to find each token's surrounding context

      • ((surrounding, word), centre_labelWord)

      • till centre word is last word

      • takes 2 words bfr & aft centre word

   2. train a NN

      • narrow NN -> only 1 hidden layer

      • embedding matrix -> E subscript(v*d)

      • E = weights

      • v = size of vocab

      • d = num of dimension (hyperparameter between 1-4, larger ^ refined, cost, required comp power)

      • embedded vector = input(one-hot encoding) * embedding matrix(word2Vec)

      • embedded vector has semantic meaning

      • sum of all embedded vector used for hidden layer (needs 1xd, E is randomly assigned to start)

      • feed result 2 softmax function for probability y (1xv vector) output layer prediction of the word being in that position

      • compare output layer with actual result & backpropagation to adjust weights E till minimal diff between results

2. Skip-gram -> predicts context (likelyhood of words appearing around centre word) based on centre word

   • Data prep

     • format (centre, surrounding), (centre, surrounding2)

   • training

     • skip-gram technique

     • input layer (one-hot encoded)

     • embedding matrix applied then feed into hidden layer

     • another E' weights layer get output layer (vector 2 predict 2000 surrounding context words)

Transfer Learning & Reusable Embeddings

• TF Hub - tensorflow hub

  • add pre-trained embedding 2 model via TF Hub

  • library of reuseable ml parts called modules

  • parts of ml model

    • publish

    • discover

    • reuse

  • pass url of the module to hub constructor (e.g. NNLM)

  • NNLM - Neural Probabilistic Language Model, a pre-trained word embedding

    • TF hub has 50 dimensional NNLM = nnlm-en-dim50

  • can be used as keras layer in a sequential or functional model

  • pass sentence in argument to get embedded vector

  • decide to make pre-trained embeddings trainable or not

    • small dataset -> freeze trainable (prevent overfitting)

    • large dataset -> allow embedding, trainable = true

• Re-useable embeddings (pre-trained embeddings)

-- NLP MODEL --

ANN

• Artificial Neural Network / shallow NN

• input layer (text) > hidden layer > output (probability)

• calculate input weighted sum > activation function > output weighted sum > activation function > cost function > gradient descent & backpropagation

• activation function

  • Sigmoid (0-1 binary)

  • Relu (neg to 0, pos stay)

  • Tanh (-1 to 1)

  • Softmax (probability distribution multi-class)

• Cost function

  • MSE -> regression problem

  • cross-entropy -> classification

• gradient descent - search function, learn rate determines speed, epoch hyperparameter

TensorFlow

• open source high performance

• end 2 end platform

• Architecture

  1. tf.keras, tf.data -> high-level, API distributed training

  2. tf.layers, tf.losses, tf.metrics -> NN API components

  3. Core TF (py) -> numeric processing

  4. Core TF (C++) -> wrapping containers & custom operations

  5. CPU, GPU, TPU, Android -> diff hardware

DNN

• more hidden layers than ANN

• feed forward only 1 direction -> no memory

• use keras more convenient, has abstractions, building blocks, launch with high iteratin velocity

  • core data structure - layers & models

• sequential - output of prev layer input of nex

  1. define & create model

     • pre-processing layer - TextVectorization

     • word embedding - input to dense layers, define TF what embedding vector

       • vocab_size + 1 -> num of tokens

       • vocab_size -> account for unknown words

       • Max_len -> max words of article title in dataset

     • Create a hidden layer -> calculate the average of the embedded vectors and use it to represent a sentence BOW to lambda

  2. Compile the model

     • optimizer

     • loss function

     • evaluation metrics

  3. Call model.fit()

     • hyperparameters: epoch, embed dimension (num of features to represent word), patience (no of epoch aft no improve stop)

  4. evaluate the performance

     • plot accuracy & loss graphs

  5. model.summary() - model architecture

  6. Generate a prediction

     • returns probabilities of each class

       
       

RNN

• recurrent NN -> recursive multiple DNN - has memory

• loop carries the hidden states, which contain the memory, to the next time step of learning

• repeat input / output / hidden till mirrors time t

• inside each RNN Cell

  • 1 word at a time

  • 1 pipline

  • has input (concatenation of ht-1), tanh (activation for relevant data filter), hidden layer (ht) & output (feed to linear/dense layers)

  • hidden state ht is messenger carrying info to nex cell

• uses rmb prev words to predict nex word

• Hidden States -> retain infomation from past 2 futures

• short term memory

LSTM

• long short-termmemory

• vanishing gradients -> closer words have stronger effect on prediction than further words (RNN short term memproblem)

• LSTM cell

  • 2 pipelines

  • 1 short term, 1 long term

  • NN layer, pointwise operation, vector transfer, concatenate, copy

  • Gate -> regulates flow of information - sigmoid function (0 forget the info entirely - 1 rmb info entirely) & pointwise multiplication function to decide how much og pipline info to pass on to the nex iteration

    1. forget gate -> get ht-1 + xt to decide what should pass on, then outcome vector * cell state Ct-1 updated to delete not needed info

    2. input gate -> 1st NN layer has 2 sigmoid functions filter info that must be added to the cell state, 2nd NN layer use tanh to create new candidate of cell state to long term mem

    3. output gate -> combines input, hidden state (short term mem) & cell state (pipeline that allows data in a straight forward way) to decide what to output

GRU

• gated recurrent unit

• a variant of LSTM

• merges cell state & hidden state pipeline into 1 (combining short & long term mem)

• Reset Gate -> determines how much info from past carried over

• Update Gate -> combines forget and input gate in LSTM to decide how much info should be forgotten & rmb to pass to time step

Encoder-Decoder architecture

• encoder - unrolled RNN/LSTM/GRU -> enter 1st language

• decoder -> output translation of second language

• softmax layer -> model predictions too many weights

• model training - gradients in affordable approximate way -> sampled_softmax_loss() method to compute and return the training loss

• architecture

  1. GO marks the beginning of the sentence

  2. predictions - call embedding layer 2 convert words into embedding

  3. unroll the RNN inputs - GRU

  4. Y is a dense layer - conditional probability of each word in vocab

  5. Beam search decoder -> considers multiple alternatives at a time

Attention Mechanism

• to improve translation use ^

  1. passes more data (all hidden states) to the encoder

  2. extra step bef producing output

     • foucus on most relevant parts of input

     • multiples encoder's hidden state with its softmax score

     • amplifying high score hidden states vice visa

• attention step - pass the context vector (hidden states & H4 vector) feedforward NN trained jointly with the model indicating output word of time step

• repeat

Transformer

• input sentence > embed each word > multi-headed attention with respective weight metrics > calculate attention QKV matrices > concatenate for output same dims as final matrix

• Transformers pre-trained models that represent words as vectors

• vectors remain constant & don’t change based on context

• model learns how certain words are aligned to each other

• built based on attention mechanism improves the performance

• encoder (input sequence) & decoder (representation for relevant task)5x

• hyperparameters

  • no of encoders stacks

• Encoders

  • has different weights

  1. self-attention:

     • inputs first flow through encoder

     • look @ relevant words as encodes centre word in sentence

     • dependencies exist

  2. feedforward:

     • expects 1 matrix per word - use concatenate * extra weights

     • same feedforward NN independently applied to each position

     • dependencies do not exist allowing parallelism

     • focus on different positions

• Decoders

  1. self-attention

  2. encoder-decoder attention

     • helps decoder focus on relevant parts of input sentence

  3. feedforward

• after embedding the words in the input sequence the embedding vector flows through the 2 layers of encoder

• each word create QKV matrices by multiplying the embedding by three matrices trained during training process

  1. Query vector

  2. key vector

  3. value vector

  • vectors dims - 64

  • embedding & encoder input/output dims - 512

• constant computation of multi-headed attention constant -> architecture vectors don't have to be smaller

• multiply by the softmax score to prep for sum up - keep intact focus words & remove irrelevant words e.g 0.001

• positional encoding -> specific pattern 2 determine position, dist of each word in seqn

BERT

• Bidirectional Encoder Representations from Transformers

  • trained in 2 variations (12 base & 24 large, encoder stacks), using wiki & bookcorpus, one million steps, TPU

• used in gg search

• benefits

  • handle long input context

  • multi-tasked objective

  • both sentence-lvl & token-lvl

  • fine-tunned for diff tasks

• BERT has more layers feedforward networks & attention heads than transformers

• trained on task:

  1. Masked language model

     • sentences masked and model trained 2 predicts masked words

     • let k = 15% balance masking (too much limits learning, too little expensive)

  2. predict next sentence

     • learn relationship between sentences & predict next sentence (given a new sentence is that the nex one)

     • binary classification

     • Tokens fed into the model

       1. CLS - classification task objective

       2. token embeddings - transform into vector representation of dims

       3. segment embeddings [SEP] - separator distinguish inputs in a given pair to split sentences

• consists of a stack of transformers

• fine-tune BERT for downstream tasks (quicker not from scratch)

  1. format input sentence as a single-packed sequence (e.g. classification or similarity)

  2. add new output layer to pretrained model ()

  3. choose fine-tunned model parameters or leave be based on amt of data

  • used for single sentence classification, sentence pair classification, question answering, tagging tasks, etc

Large Language Models

• large general-purpose pre-trained models, fine-tunned for specific purposes (e.g. lack resources, skills)

• solve common problem tailored to fit industry

  • text classifications

  • question answering

  • doc summarization

  • text generalisation

• benefits

  1. single model usable for diff tasks

  2. fine-tune requires minimal filled data

  3. performance continuously growing with data & parameters

• PaLM - 540 billion parameters, leverages new pathway sys, orchestrates distributed computation for accelerators

• 29 common english NLP tasks

  • NL interference

  • common sense reasoning

  • in-context reading comprehension

  • Question answering

  • winograd-style tasks

  • close and completion

• future trends of NLP:

  • large language models fundamental for applications

  • scaling law (scaling model parameters most impt, then scaling data, training iterations)

  • business model change to rental downstream from upstream providers (cos training is resource intensive)

  • take adv of scalability, accessibility, mobility, security & cost

• 
  

1. Breadth First Search

   • queue

   • visited array

   • polling root's neighbours to find path

2. Depth First Search

   • stack (first in last out)

   • visited array

   • poll neighbours according to top of stack

key words:

adjust, pop, poll, add, isEmpty, contains, size, Vertex, ArrayList, push, forEach(), ArrayDeque(), get(), label

• supported on java collections framework

ArrayList

LinkList

Stack

Queue

-- BINARY TREE --

-- BINARY TREE SEARCH --

• General Statement - calling recursive function further

• Base Statement - condition where it stop further calling itself, solution obtained directly

• all recursive can be implemented using iteration

• takes up significant resources -> incomplete ops stored in stack

• only use when it's easier to understand

  • tower of hanoi

  • Fibonacci num

class TestRecursive {
 public static void main(String[] args) {
   shout(5);
 }
 public static void shout(int x) {
   System.out.println(x + “.AHHHHH!"); //out 5 to 1 
   if (x>1) shout(x-1); //reducing version of self
   System.out.println(x + “.Cheeeew!");//1-5 aft ah
 }
}

-- BIG-O NOTATION --

• more iteration more time complexity

O(n) -> linear

• unnested loops

• xn, x = no. of loops

O(n^2) -> Quadratic

• nested loops

• ^x , x = no. of nested loops

O(log2(n)) -> Log

• for loop where i = i*2

• binary Search worst case

O(1) -> Constant

• no loop same discreet no of operations

btn for links

• run-time error unexpected event causing termination

• try-catch-finally handles exceptions

  try{
    //intended operation code
  }catch (ArithmeticException e){ //api exception
    //output error message
    //can have more than 1 catch
  } finally {
   //will always execute
  }

• ArithmeticException -> math error things

• ArrayIndexOutOfBoundsException -> index non-existent

• NumberFormatException -> wrong number format

-- THROW vs THROWS --

• throw used inside method to throw an explicit exception

  e.g. MyClass() {throw new ArithmeticException("error msg"); }

• throws used in method signature

  e.g. MyClass() throws IOException{}

• Hierarchy subclasses: Object > Throwable > Error, Exception

-- CHECKED UNCHECKED EXCEPTIONS --

• unchecked ~ -> no compilation error

  • RuntimeException

    1. NullPointException

    2. ArrayIndexOutOfBoundsException

    3. IllegalArgumentException

    4. NumberFormatException

    5. ArithmeticException

  • Error

    1. LinkageError

    2. VirtualMachineError

    3. AWTError

• checked ~ -> compiler checks it

  1. SQLException

  2. IOException

  3. ClassNotFoundException

  4. InvocationTargetException

  5. AWTException

-- ABSTRACT CLASS --

• abstract -> method with signature only, no logic/implementation

• used in related classes

• contains @ least 1 abstract method

• extend ->

  public abstract class MyAbstract {
    MyAbstract() {}
    public void MyAbsMethod() {}
  }
  
  public class MyClass extends MyAbstract {
    public void MyAbsMethod();
    MyClass(){}
  }

• can provide abstract/non-abstract methods

• if class has >=1 abstract class must declare abstract

• can have final, non-final, static, non-static variables unlike interface

• abstract method are methods in abstract class

-- INTERFACE --

• have methods & data

• can only have constants like static & final variables, nothing else

• usually static/default methods abstract (only signature)

• cannot be instantiated

• returm type methods using interface method must return the implemented object interface

• enforces consistency - must provide implentation method

• Define

  public interface MyInterface { 
    String MyMethod(); //method no logic
  }

• implement

  public class MyClass implements MyInterface {
    String name;
    public String MyMethod(){
      return (name) //details
    }
  }

• using interface implemented method to print name

  public class Test {
    public static void test(String[] args){
      MyClass x = new MyClass();
      x.name = "Celine";
      System.out.println(x.MyMethod()); //prints 
    }
  }
  // can implement MyInterface in other classes as well

• Default method - classes implementing interface can override default method

  default void methodName(){}

• implement can use multiple inheritance (while class can only extend 1 parent class)

• extends

-- POLYMORPHISM --

-- CLASS OBJ --

-- METHOD OVERRIDING --

-- ARRAYS --

-- GENERIC CLASS --

• DO -> 5G topic finish

• DO -> 5G slides

• DO -> IENT Reflections

Tue

• DO -> ECS Paper the second 1

• DO -> ECS Report

Wed - 5GAIOT Presentation & EST, ECS lab (test animation)

• DO -> ECS Report

Thur - ECS Demo, ECS paper

• DO -> ECS Report

Friday - IENT reflections, ECS Report

• DO -> give Wen En Bday present

map

Binary search tree

recurssive

0x0A -> A being the index so 17 = 0x11

0xcA -> c second row, A ~^

1. Quantum Computing -> complex parallel process

2. AIML -> learn from data

3. Extended Reality XR -> blending digital-physical world

4. IOT -> interconnected devices collecting data

5. BlockChain -> secure Decentralised connections

6. Edge Computing -> process data closer to the source

7. Robotic Process Automation RPA -> Automate repetition

8. 5G -> network speed bandwidth communication

9. MetaVerse -> digital world digital objects / places

10. Digital Twin -> replica of systems predictive

Integrate trends that align and scale with business model

-- 10 5G Trends --

1. OpenRAN -> interoperability network slicing & virtualisation

2. Network Slicing -> multiple virtual networks on a single physical comp

3. Beamforming -> strength signal & reduce interference by directing wireless signals to specific devices

4. 5G SA -> with 5G core independently enhanced performance & flexibility

5. Ultra Reliable Low Latency Communication URLLC -> time critical applications

6. Phased Arrays Antennas -> directs signal through multiple antennas improve strength capacity

7. Massive Machine Type Communication MMTC -> many IOT devices with low power sparse transmission

8. Enhanced Mobile Broadband EMBB -> increased data rate, performace

9. Carrier Aggregation -> combines multiple freq bands increase data rate & network capacity

10. Dynamic Spectrum Sharing DSS -> simultaneous 4G 5G freq band optimising spectrum usage

-- Beyond 5G --

1. Intelligent mobility systems

2. mobility as a Service

3. dependable internet services

• 6G for machines efficiency

• use AIML to plan & optimise networks

• support digitalisation virtualisation experience

• 6G at 110 Terahertz THz faster than 5G in gigahertz

  • high freq broadband communication -> high precision sensing capacity

  • problems:

  • rfic performence

  • path loss to atmosphere

  • high speed adc/dac

• Freq Divsion Duplex

  • Freq DD - 3G

  • Time DD - 4G

    • XDD -> increase uplink

• Full Duplexing -> increase capacity

• Network topology

  • continuity of network connection services

  • optimization of network configuration

  • enhanced mobility support for network entities

  • NTN (non terrestrial network) & HAPS (high altitude platform systems)

-- 5G Impact --

1. Enhanced Connectivity -> seamless network for devices to communicate, collaborate AIOT

2. Smart City -> AIOT & 5G respond to real time data optimize operations & improve quality of life

3. Industry 4.0 -> data from sensors, centralised AI, optimize production line efficiency, predict maintenance & reduce downtime

4. Retail -> immerse targeted marketing supply chain efficency, real time data smart shelves RFID & customer behaviour analysis

5. Transit -> integration of 5G AIOT, v2v communication, optimize infrastructure less congestion, safer, sustainable

6. Healthcare -> wearable devices send continous real time data to AI for early detection, proactive treaments & personalised treament plans

-- Regulations --

1. Spectrum Allocation -> policies & exisiting spectrum usage to harmonise integration across regions

2. Infrastructure Regularisations -> small cells & base stations deployment based on zoning etc

3. Data Privacy & Security -> laws to prevent cyber risks

4. Interoperability -> compatibility between exisisting devices need standardisation of protocols & interfaces

-- Considerations --

1. Privacy & survillence

2. Bias & discrimination

3. Job Displacement

4. Health & environmental impacts

   • electromagnetic radiation

   • end of life waste management

-- Benefits --

-- Edge Computing --

-- Network Slicing --

-- 4G limitations --

-- Massive IOT --

6Cs

1. Collect data - sensors & microcontrollers

2. Communicate - wifi module & Acess Point

3. Connect - wired / wireless

4. Cloud - storage, server, aws, etc

5. Comprehend - analytics

6. Create - interfaces

-- AIML --

--

1. Beamforming

2. MIMO

3. OFDM

4. DSS

5. mmWave

-- 3GPP --

• ALL Cloud & service oriented core

• session management, security, end devices, authentication & traffic

-- ALL CLOUD - Core Network Architecture --

1. Software Defined Network

2. Network Functions Virtualization

3. Operations & Mantainence

4. Edge Cloud

5. Regional & Core cloud

-- Service Oriented Core --

3 key factors

1. Programmability

2. Flexible Architecture

3. Smart Pipes

4 features

1. Control User Plane Separation (CUPS)

2. Serviced-Based Architecture

3. Network Slicing

4. Cloud Native

1. Self-Aware Machines

2. Theory of Mind

3. Limited Memory Machines

4. Reactive Machines

-- subsets --

AI>ML>DL>CNN & DNN & Generative AI

-- ML methods --

1. Reinforced Learning -> agent max out reward in uncertain simulation based on actions

2. Unsupervised Learning -> patterns in unlabelled data

   • clustering, anomaly detection, association mining, latent varible model

3. Supervised Learning -> classification & regression

   • linear regression, random forest, support vector machines

-- Cloud AIOT Integration Architecture --

Device Layer > Connectivity Layer > Cloud Layer > User Communication layer

-- Edge AIOT Integration Architecture --

Collection Terminal Layer > Connectivity Layer > Edge Layer

benefits:

• reduced network workload

• Privacy

• Real Time Response / early response

• continous

• low latency

--