17 Multiple Linear Regression

Most practical applications of regression analysis employ models that are more complex than the simple straight-line model. For example, a realistic model for monthly sales revenue would include more than just the amount of advertising expenditure. Factors such as last month´s sales, number of sales competitors, and advertising medium are some of the many variables that might be related to sales revenue. Thus, we would want to incorporate these and other potentially important independent variables into the model in order to make accurate predictions.

  • A general model for a multiple linear regression model that relates a dependent variable, \(y\), to \(k\) independent \(x\)-variables is :

\[ \begin{equation} y_{i}=\beta_{0}+\beta_{1}x_{i,1}+\beta_{2}x_{i,2}+\ldots+\beta_{k}x_{i,k}+\epsilon_{i}. \end{equation} \]

  • \(k\) is the number of predictors or independent variables, that is, the model has \(k+1\) regression parameters (the \(\beta\) coefficients).
  • \(\epsilon_{i}\) have a normal distribution with mean \(0\) and constant variance \(\sigma^2\). The same assumption of simple linear regression.
  • \(i\) refers to the \(i\)th individual or unit in the population.
  • The word linear refers to the fact that the model is linear in the parameters, \(\beta_0, \beta_1, \ldots, \beta_k.\) This means that each parameter multiplies an independent variable, while the regression function is a sum of these parameter times x-variable terms. Each \(x\) can be a predictor variable or a transformation of predictor variables.
  • For example, if
    • \(x_1\) = Advertising expenditure
    • \(x_2\) = Number of sales competitors
    • \(x_3\) = \(x_{i,1}^2\)
    • \(x_4\) = 1 if TV advertising, 0 if not. We can fit the model:

\[ \begin{equation} y_{i}=\beta_{0}+\beta_{1}x_{i,1}+\beta_{2}x_{i,2} + \beta_3x_{i,1}^2+ \beta_4x_4 + \epsilon_{i}. \end{equation} \]

where \(x_3 = x_{i}^2\) is called a higher-order term because it is the value of a quantitative variable (\(x_1\)) squared (i.e., raised to the second power).

References: