Lecture 25 - Supervised Learning I

#notes#cs471

Recap §

Approximate Q-Learning

• Q-Learning with linear Q-Functions
• 
• transition = $(s,a,r,s^{\prime })$
• difference = $[r+\gamma ma{x}_{a^{\prime }}Q(s^{\prime },a^{\prime })]-Q(s,a)$
• $Q(s,a)\leftarrow Q(s,a)+a[difference]$

Learning §

• Essential for unknown environments
• Learning is useful as a system construction methods

Supervised Learning §

• Task: Learn a mappin/model $f_{\theta }$ from imputs $x\in X$ to outputs $y\in Y$

  • Inputs $x$ are also called features
  • Outputs $y$ are also called targets
  • If y is categorical: classification model
  • If y is real-valued: regression model
  • $f_{\theta }$ has learnable parameters $\theta$

• Experience: Given in the form of input-output pairs

  • Training Set D = ${x,y}^N$

• Examples

  • Email spam detector
    • Input: Words
    • Output: Spam or not
    • Classification model
  • Digit Classification
    • Input: image
    • Output: Digits
    • Classification Model
  • Stock Forecasting
    • Input: Price History
    • Output: Future Price
    • Regression Model

K-Nearest Neighbor Algorithm §

• Memorize training set
• Given distance metric, find K closest sample to x, and pick average of y.
  • So basically count K closest sample, and pick most frequent one.

Probabilistic Modeling §

• Coin Flipping: Want to predict whether a coinflip will be a head or tail

Maximum Likelihood for Learning §

• Training Set: D = $\{y^{(n)}{\}}_{n=1}^N$
• Likelihood of D
  • Assume samples are independent and identically distributed
  • L(D;$\theta$) = ${\Pi }_{n}P(Y=y^{(n)})={\Pi }_n{\theta }^{y^{(}n)}\ast (1-\theta {)}^{(1-y^{(n)})}\ast$
• Maximize $L(D;\theta )\leftrightarrow$ Maximize $\log L(D;\theta )$
• To maximize find $\theta$