ml, machine learning, ml, python, artificial intelligence, ai, computer science, data science, data analytics, technology training at bay area, algorithm, deep learning, supervised learning, unsupervised learning, database, logistic regression, polynomial regression, rnns
CSE590/MB590 Special Topics
  - Advanced Machine Learning

This course is focuses on in-depth understanding of Deep learning applications and introduces some key concepts in reinforcement learning. Training Deep learning networks can be a challenging task and requires a good understanding of the nature of gradient descent and its variant. Students will learn about different forms of loss functions and hyper parameters and regularization in conv nets, RNNs and others.
  • » 23 hours (8 weeks) in class lecturing plus dedicated mentoring sessions from our faculty of industry experts
  • » 1.5 semester credits for both certificate and master’s degree
  • » Access to high-quality live class recording
  • » Online live classroom available for all classes
  • » Lifetime learning resources for our students
  • $ 990
Course Description

This course is focuses on in-depth understanding of Deep learning applications and introduces some key concepts in reinforcement learning. Training Deep learning networks can be a challenging task and requires a good understanding of the nature of gradient descent and its variant. Students will learn about different forms of loss functions and hyper parameters and regularization in conv nets, RNNs and others. The focus then turns into reinforcement learning as an alternative to supervised learning. OpenAI Gym is introduced as a tool to simulate the agent’s environment and interaction. We will use Keras as a key framework to model different neural network architectures.

Prerequisite: Work experience, basic math knowledge, basic understanding of neural networks and programming experience with Python.

Measurable Course Objectives
  • Demonstrate a basic understanding of machine learning and deep learning concepts and reinforcement learning.
  • Explain the key elements of training a deep neural network.
  • Describe the major concepts of machine learning, its main framework derived from traditional data analytics
  • Explain the reinforcement learning and its relationship to deep learning frameworks.
  • Demonstrate ability to build model, train model, tune model using deep reinforcement learning technology on specific projects.
University-wide Student Learning Outcome

The University Student Learning Outcomes assessed and reinforced in this course include but are not limited to the following:

  • Communication
  • Critical Thinking
  • Information Literacy
Course Topics
Week 1-2: Unsupervised Learning and Clustering
  • Clustering
  • K-means and its limitations
  • Using clustering for image segmentation
  • Using clustering for preprocessing
  • Using clustering for semi-supervised learning
  • DBSCAN
  • Other clustering techniques
  • Gaussian Mixtures
  • Anomaly Detection using Gaussian Mixtures
  • Selecting the number of clusters
  • Bayesian Gaussian Mixtures
  • Other algorithms for Anomaly detection and Novelty Detection
  • An overview of Keras based on TensorFlow
  • Sequential and Functional Neural Network architecture definition
  • Project 1: Olivetti faces dataset K-Means clustering/model reduction and classification (due in two weeks after announcement)
Week 3-4: Training Deep Neural Networks (practical aspects)
  • Vanishing/Exploding Gradients Problems
  • Reusing pretrained models (transfer learning)
  • Faster Optimizers
  • Regularization in Deep NN
  • Project 2: Train a deep neural network on CIFAR10 dataset (due in two weeks after announcement)
Week 5-8: Deep Reinforcement Learning
  • Introduction to reinforcement Learning
  • Concept and applications (Agent, environment, observations)
  • Markov Decision Process (MDP)
  • Introduction to OpenAI Gym
  • Q-Learning
  • Value of action (Q-function)
  • Bellman Equation
  • SARSA vs. Q-learning (table driven approach)
  • Deep Q-Learning (DQN)
  • Application of CNN in Q-learning
  • Limitations of using NN in RL
  • Replay buffer and target network
  • Project 3: DQN project (stock market project)
About the Instructor

Jahan Ghofraniha

Expert in pattern recognition, advanced machine learning and Deep Learning with application to medical, financial data. Reinforcement learning and DQN. Deep learning implementation with parallel algorithms and GPUs using Pytorch, TensorFlow, and Keras.

Specialties: Reinforcement Learning, Deep Learning, Convolutional Neural Network, GANs, supervised and unsupervised learning algorithms, data clustering, model reduction techniques, advanced machine learning and business analytics.