And the equation of a line can be written as y=\u04e8.x.<\/p>\n
If there are more than one independent variable, this representation can be extended.<\/p>\n
x=[<\/span>x<\/span>1<\/span><\/sub>\u00a0 <\/span>x<\/span>2<\/span><\/sub>\u00a0 <\/span>x<\/span>3<\/span><\/sub>\u00a0 ………..\u00a0 <\/span>x<\/span>n<\/span><\/sub>]<\/span>T<\/span><\/sup> \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u04e8=[<\/span>\u04e8<\/span>1<\/sub>\u00a0 <\/span>\u04e8<\/span>2<\/span><\/sub>\u00a0 <\/span>\u04e8<\/span>3<\/span><\/sub>\u00a0 …………….\u00a0 <\/span>\u04e8<\/span>n<\/span><\/sub>]<\/span>T<\/span><\/sup> ,here <\/span>x<\/span>1<\/span><\/sub>=1<\/span><\/p>\n And the equation of the line remains the same.<\/p>\n \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 y=\u04e8\u00a0 . x<\/span><\/p>\n Suppose we have three (x,y) pairs. {(2,5), (4,9), (6,13)}. It is a simple example. To train our model, we need to find the vector \u04e8. Now, for any given x, we can easily compute the value of y. We plot the three training examples, which fall in a straight line in this case. If the given x is 5, we can find the y value using this plot.<\/p>\n The parameters of this model can easily be calculated to be \u04e80<\/sub>=1 and \u04e81<\/sub>=2. Using this learned information we can compute (predict) the output of any new input data. This was a simple example, but actual tasks are not so easy. The training examples may be scattered yet they appear linear, and we will have to find the best fit.<\/p>\n In this picture, the examples do not lie in a straight line. However, they seem to follow a line closely. This line can be the one that can fit the dataset with the least loss and hence be the required model.<\/p>\n The parameter vector \u03b8 can be computed by plotting a loss curve, which will always be a parabola with its opening in the upward direction, then picking a point on the curve and moving towards the minimum of the curve to find the parameters\u2019 optimal value.<\/p>\n We upgrade the value of the vector \u03b8 as follows.<\/p>\n \u03b8<\/span> j<\/span><\/sub>:=\u03b8<\/span> j<\/span><\/sub>-(\u03b1\/m)*<\/span>\u2211<\/span>i=1<\/span><\/sub>m<\/span><\/sup> (<\/span>h<\/span>\u03b8<\/span><\/sub>x<\/span>i<\/span><\/sup>–<\/span>y<\/span>i<\/span><\/sup>)<\/span>x<\/span>j<\/span><\/sub><\/p>\n The normal equation method is a straightforward method of computing theta without requiring any loop and is, therefore, asymptotically faster.<\/p>\n \u04e8=(<\/span>X<\/span>T<\/span><\/sup>*X<\/span>)<\/span>-1<\/span><\/sup>.<\/span>X<\/span>T<\/span><\/sup>*Y<\/span><\/p>\n We will try to build a linear regression model which uses the normal equation method to find the value of \u04e8.<\/p>\n We will build our own dataset to train a regression model.<\/p>\n Output-<\/strong><\/p>\n Output-<\/strong><\/p>\n The dataset we have created is a straight line. To make it a bit more challenging, we will add some noise to it. The data thus created will resemble that in practical examples in terms of being scattered.<\/p>\n Output-<\/strong><\/p>\n Output-<\/strong><\/p>\n As we can see the vector y is no longer precisely linear to the input.<\/p>\n Every other pair of points has a different gradient and intercept. Our job is to find a line so that the distance betweeb these points from it is minimum.<\/p>\n For this example, we will use the normal equation to calculate theta.<\/p>\n Output-<\/strong><\/p>\n Firstly, we will plot the initial x and y. Then, plot the x and the y computed using the calculated theta (prediction). Finally, we will plot both of these graphs together to see if our prediction is good or not.<\/p>\n Output-<\/strong><\/p>\n Output-<\/strong><\/p>\n Output-<\/strong><\/p>\n As we can see that the prediction curve follows the initial data points very closely, we can rest assured that our model is working properly.<\/p>\n Now that we have built a model using numpy, we will try to build one using PyTorch. This time we will use Gradient Descent as the optimisation algorithm.<\/p>\n Before doing anything, we have to import torch and other required libraries.<\/p>\n #Now we will create random data to train our model<\/p>\n #We have taken the transpose of the input and output tensors to transform its shape from 1 x 10 to 10 x 1.<\/p>\n![\"linear](\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2023\/01\/linear-regression-from-scratch-graph-1.webp\")
<\/a><\/p>\n
\n\u0b83 y=1 + 2x.<\/p>\n
\nFor x=5, y=1+2*5=11.<\/p>\n<\/a><\/p>\n<\/a><\/p>\nMethods of finding the parameters of a model in Linear Regression:<\/h3>\n
1. Gradient Descent Method:<\/h4>\n
2. Normal Equation Method:<\/h4>\n
Linear Regression using numpy:<\/h3>\n
a. Importing the required libraries<\/h4>\n
import numpy as np\r\nimport matplotlib.pyplot as plt\r\n%matplotlib inline<\/pre>\n
b. Making dummy dataset<\/span><\/h4>\n
x=np.linspace(-5,5,30)\r\ny=12*x\r\n\r\nprint(x)<\/pre>\n
<\/a><\/p>\nprint(y)\r\n<\/pre>\n
<\/a><\/p>\nc. Adding noise to the output vector and reshaping the vectors<\/h4>\n
noise = np.random.normal(0, 5, y.shape)\r\ny=y+noise\r\n\r\nprint(y)\r\n<\/pre>\n
<\/a><\/p>\nx=x.reshape(-1,1)\r\ny=y.reshape(-1,1)\r\n \r\nprint(x)\r\n<\/pre>\n
<\/a><\/p>\nprint(y)\r\n<\/pre>\n
<\/a><\/p>\nd. Using the normal equation method to compute the parameters<\/h4>\n
theta=np.dot((np.linalg.inv(np.dot(x.T,x))),(np.dot(x.T,y)))\r\n\r\nprint(theta)\r\n<\/pre>\n
<\/a><\/p>\ne. Visualising the prediction along with the input initial input and outputs<\/h4>\n
plt.scatter(x,y)<\/pre>\n
<\/a><\/p>\nplt.plot(x,x*theta)\r\n<\/pre>\n
<\/a><\/p>\nplt.scatter(x,y)\r\nplt.plot(x,x*theta)\r\n<\/pre>\n
<\/a><\/p>\nBuilding Linear Regression Model Using PyTorch:<\/h3>\n
a. Importing the required modules<\/h4>\n
import torch\r\nfrom torch.utils.data import TensorDataset,DataLoader\r\nimport torch.nn as nn\r\nimport numpy\r\nimport matplotlib.pyplot as plt\r\n<\/pre>\n
b. Building the model<\/h4>\n
class Model(torch.nn.Module):\r\n def __init__(self):\r\n \tsuper(Model, self).__init__()\r\n \tself.linear = torch.nn.Linear(1, 1)\r\n\r\n def forward(self, x):\r\n \ty_pred = self.linear(x)\r\n \treturn y_pred\r\n\r\nlreg=Model()\r\n#Creating an instance of the model we have created above\r\n\r\ncriterion = torch.nn.MSELoss()\r\n#specifying the loss function. Here we have used Mean Squared Error Loss\r\n\r\noptimizer = torch.optim.SGD(lreg.parameters(), lr = 0.01)\r\n#Optimising the model using Stochastic gradient descent.\r\n<\/pre>\n
c. Preparing data<\/h4>\n
x = torch.tensor(range(-5,5)).float()\r\ny = 12*x\r\n\r\nx_train = x[:,None]\r\ny_train = y[:,None]\r\n<\/pre>\n
d. Training the model<\/h4>\n
for i in range(500):\r\n \t \r\n \toptimizer.zero_grad()\r\n \t#If we do not set grad to zero then the old gradients will add up in the current gradients making our model incorrect.\r\n \t \r\n \ty_pred = lreg(x_train)\r\n\r\n \t \r\n \tloss = criterion(y_pred, y_train)\r\n \t \r\n \tloss.backward()\r\n<\/pre>\n
#Back propagating the neural network to find the gradient of the output with respect to the given parameters.\r\n\r\n \t \r\n\toptimizer.step()\r\n\t#Optimising the weights of the neurons depending on the gradient calculated during backpropagation.\r\n\r\n\tprint(i,loss)\r\n<\/pre>\n