Loss Function

Cost function SSR [Sum of Squared Residual] function to find the optimal parameters values.

Dataset:

x=[2,4,5]\\ y=[1.2,2.8,5.3]\\

Initial parameter values:

LR=0.001\\ \beta_{0}= 0\\ \beta_{1}=1

Cost function : Sum of Squared Residuals (SSR)

cost = J = \sum(y-y_{pred})^2

Prediction function:

y_{pred} = \beta_0 - \beta_1x

Predicted value:

y_{pred1} = 0 + 1(2) = 2 \\ y_{pred2} = 0 + 1(4) = 4 \\ y_{pred3} = 0 + 1(5) = 5

Calculating the cost:

cost = J_1 = (1.2-2)^2= -0.8^2 = 0.64 \\ J_2=(2.8-4)^2=-1.2^2=1.44\\ J_3=(5.3-5)^2= 0.3^2=0.09\\ total\space cost= 2.17

Finding derivations:

\frac{\partial}{\partial \beta0 }J = \frac{\partial}{\partial \beta0 }\big(\sum(y-y_{pred})^2\big) \\ =\frac{\partial}{\partial \beta0 }\big((u)^2\big)