会员服务

【斯坦福CS205L新课程】聚焦机器学习的连续数学方法

专知出品

【导读】斯坦福大学最近新开设一门课程《Continuous Mathematical Methods with an Emphasis on Machine Learning》，以机器和深度学习为重点的计算机视觉和机器人学中连续数学方法的综述。虽然从机器学习的角度出发，但本课程将侧重于计算线性代数和优化等基础数学方法，以及反向传播自动微分、常微分方程动量法、CNNs、RNNs等专题。


课程主页：

http://web.stanford.edu/class/cs205l/lectures.html


课程安排：

  • Unit 1: Introductory Material [1A slides] [1B notes] [1A,B CA notes] [1C notes] [1D slides]

    • Knowledge Based System(KBS) versus Machine Learning(ML) (Discrete versus Continuous Math)

    • Addition/Multiplication with KBS vs ML

    • Polynomial Interpolation and Overfitting

    • Monomial Basis and Error Sources

    • Condition Number

    • Polynomial Interpolation (Lagrange/Newton basis functions)

    • Representation Theory, CNNs for cloth

  • Unit 2: Solving Linear Systems [2 notes] [2A CA notes] [2B,3A CA notes]

    • Systems of linear equations

    • Normalization

    • Rank and Solvability

    • Matrices: Square, Diagonal, Upper Triangular, and Lower Triangular Matrices

    • Gaussian ELimination

    • LU Factorization

  • Unit 3: Understanding Matrices [3A notes] [3B notes] [2B,3A CA notes] [3B,4,5A CA notes]

    • Eigenvalues and eigenvectors

    • Singular Value Decomposition

    • Eigenvector Decomposition

    • Preconditioning

    • Vector and Matrix Norms

    • Condition Number

  • Unit 4: Special Linear Systems [4 notes] [3B,4,5A CA notes]

    • Diagonally Dominant Matrices

    • Symmetric Positive Definite (SPD) Matrices

    • Cholesky Factorization

    • Symmetric Approximation

  • Unit 5: Iterative Methods [5A slides] [5B notes]

  • Unit 6: Local Approximations [6A notes] [6B slides]

  • Unit 7: Curse of Dimensionality [7 notes]

  • Unit 8: Introduction to Least Squares [8 notes] [8 CA notes]

    • Overfitting and Underfitting

    • Overdetermined Systems

    • Least Squares Formulation and Common Mistakes

    • Residuals and Minimization

  • Unit 9: Basic Optimization [9 notes] [9,10A CA notes]

    • Critical Points

    • Classifying Critical Points

    • Quadratic Form

    • Least Squares

  • Unit 10: Solving Least Squares [10 notes]

    • Normal Equations

    • Condition Number

    • Summary

    • Understanding Least Squares

    • Orthogonal Matrices

    • Gram-Schmidt

    • QR Factorization

    • Householder

  • Unit 11: Zero Singular Values [11 notes] [11,12A CA notes]

    • Example

    • Solving Linear Systems

    • Minimum Norm Solution

    • Sum of Rank One Matrices

    • Approximating a Matrix

    • Principal Component Analysis (PCA)

    • Finding Low Rank Approximations

    • Computing Eigenvalues

    • Condition Number

    • QR Iteration

    • Power Method

  • Unit 12: Regularization [12A notes] [12B slides] [12B,13A CA notes]

    • Adding an Identity Matrix

    • Full Rank Scenario

    • Rank Deficient Scenario

    • Initial Guess

    • Iterative Approach

    • Adding a Diagonal Matrix

    • Column Space Search Method

  • Unit 13: Optimization [13 notes]

    • Function Approximation

    • Choice of Norm

    • Optimization: Overview

    • Conditioning

    • Nonlinear Systems: Overview

  • Unit 14: Nonlinear Systems [14 notes]

    • Jacobian Matrix

    • Linearization

    • Iterative Solver

    • Line Search with Search Directions

  • Unit 15: Root finding [15 notes]

    • Fixed Point Iteration

    • Convergence Rate

    • Newton's Method, Secant Method, Bisection Method, and Mixed Methods

    • Nonlinear Systems Problems and Optimization

  • Unit 16: 1D Optimization [16 notes]

    • Golden Section Search

    • Unimodal and Successive Parabolic Interpolation

    • Root Finding

    • Nonlinear Systems Problems and Optimization

  • Unit 17: Computing Derivatives [17A notes] [17B slides] [17B supplementary reading]

    • Differentiability

    • Activation Functions

    • Symbolic Differentiation

    • Finite Differences

    • Automatic Differentiation

    • Network Cost Functions



-END-
专 · 知


专知，专业可信的人工智能知识分发，让认知协作更快更好！欢迎注册登录专知www.zhuanzhi.ai，获取5000+AI主题干货知识资料！
欢迎微信扫一扫加入专知人工智能知识星球群，获取最新AI专业干货知识教程视频资料和与专家交流咨询！
请加专知小助手微信（扫一扫如下二维码添加）， 获取专知VIP会员码， 加入专知人工智能主题群，咨询技术商务合作~
点击“阅读原文”，了解使用专知，查看5000+AI主题知识资料
展开全文
Top
微信扫码咨询专知VIP会员
Top