Learning outcomes - Describe a simple linear regression model, least squares estimation, and coefficient interpretation. - Explain the four assumptions and use residual plots to spot violations. - Calculate and interpret fit measures, coefficient tests, ANOVA output, forecasts, prediction intervals, and functional forms.

Core Idea

1. Simple linear regression asks one clean question: can variation in one variable help explain variation in another variable?
2. The variable being explained is the dependent variable. What is dependent variable: the output you are trying to understand or predict.
3. The variable doing the explaining is the independent variable. What is independent variable: the input you use to explain movements in the dependent variable.
4. If there is only one independent variable, the model is simple linear regression. If there are many, you have moved into multiple regression.
5. Regression helps in two ways. Why is regression used: to test whether a relationship exists and to use that relationship for prediction.
6. The straight-line model is:

1. What is intercept: the predicted value of \(Y\) when \(X\) equals zero. This matters only if zero for \(X\) is economically meaningful.
2. What is slope: the change in predicted \(Y\) for a one-unit change in \(X\). Positive slope means both move together; negative slope means they move opposite ways.
3. What is error term: the gap between observed \(Y\) and the true population relationship. This is the part the line cannot perfectly explain.
4. What is residual: the gap between observed \(Y\) and fitted \(Y\) from the sample line. The error term belongs to the population; the residual belongs to your estimated sample line.
5. A scatter plot is your first truth serum. If the points already lean upward or downward, the line has a fighting chance.

Estimating the Line

1. Least squares chooses the fitted line by minimizing the sum of squared residuals. Why is least squares used: it punishes large misses heavily, so the chosen line is the tightest straight-line fit.
2. The fitted value for observation \(i\) is:

1. The residual for observation \(i\) is:

1. Residuals are vertical distances from observed points to the fitted line, so they are measured in the same units as the dependent variable.
2. The least-squares objective is:

1. What is SSE: sum of squares error, the part of variation the line still fails to explain after fitting.
2. The slope estimate is covariance over variation in \(X\):

1. Why is this formula built this way: the numerator asks whether \(X\) and \(Y\) move together, and the denominator scales that co-movement by how much \(X\) itself varies.
2. The intercept estimate is:

1. The fitted line always passes through the sample means. Casual shortcut: if you plug in \(\bar{X}\), you get \(\bar{Y}\).
2. The sample correlation and slope share the same numerator, so covariance decides both signs. Positive covariance gives positive slope and positive correlation.
3. In simple regression, the sum of residuals is zero by construction. The line balances the misses above and below itself.
4. Cross-sectional regression uses many entities at one time. Time-series regression uses one entity across many dates. Same machinery, different data layout.

Assumptions

1. Four assumptions carry the model: linearity, homoskedasticity, independence, and normality.
2. Linearity means the true relationship between \(X\) and \(Y\) is linear in the parameters. What is linearity: the straight-line form is genuinely appropriate.
3. If the real relationship is curved but you force a line, the model underestimates in some zones and overestimates in others. That is bias, not bad luck.
4. A residual plot against \(X\) should look random. If it bends like a smile or a bowl, the line is missing the shape.
5. The source also warns that the independent variable should not be random. Why is non-random \(X\) used: the model treats \(X\) as the anchor used to explain \(Y\).
6. Homoskedasticity means residual variance stays constant across observations. What is homoskedasticity: the misses are equally noisy across the sample.
7. If residual variance changes across observations, you have heteroskedasticity. Think of one calm regime and one chaotic regime stitched into one dataset.
8. Residuals clustering into groups with very different widths is a visual red flag for heteroskedasticity and possibly multiple regimes.
9. Independence means observation pairs are uncorrelated with one another. What is independence: today's miss should not mechanically predict tomorrow's miss.
10. In time series, seasonality often breaks independence. Residuals that jump every fourth quarter are basically waving a flag saying, "you ignored a pattern."
11. Normality means residuals are normally distributed. What is normality: the misses around the line form a bell-shaped distribution.
12. Normality is about residuals, not the raw data. The data themselves do not have to be normally distributed.
13. Non-normal residuals matter more in small samples. In large samples, the central limit theorem can make the usual test statistics more usable.
14. Outliers in either \(X\) or \(Y\) can yank the line around. One dirty data point can fake a beautiful \(R^2\) and still make the model less trustworthy.

Breaking Down Variation

1. Regression tries to explain total variation in \(Y\), called sum of squares total:

1. The explained part is sum of squares regression:

1. The unexplained part is sum of squares error:

1. The whole decomposition is:

1. This identity is the emotional core of regression: total mess equals explained mess plus leftover mess.
2. A good model pushes more variation into \(SSR\) and leaves less sitting in \(SSE\).

Goodness of Fit and Tests

1. The coefficient of determination is:

1. What is \(R^2\): the percentage of dependent-variable variation explained by the independent variable. Higher means more explained, not automatically better economics.
2. In simple linear regression only, \(R^2 = r^2\). That shortcut dies once you leave the one-\(X\) world.
3. The F-test asks whether the slope is zero. In simple regression:

1. The mean square regression is \(MSR = SSR/1\), and the mean square error is \(MSE = SSE/(n-2)\). One slope and one intercept already consumed two degrees of freedom.
2. The test statistic is:

1. The F-test is right-tailed. Why is F one-sided: you care whether explained variation is large relative to unexplained variation.
2. The standard error of estimate is:

1. What is standard error of estimate: the typical distance between actual \(Y\) values and fitted \(Y\) values. Smaller means the line hugs the data more tightly.
2. To test a slope against any hypothesized value \(B_1\), use:

1. The slope standard error is:

1. More variation in \(X\) lowers the slope's standard error. Intuition: when \(X\) spreads out nicely, the slope has more room to reveal itself.
2. To test whether correlation is zero, use:

1. In simple regression, the slope-zero test and the correlation-zero test give the same t-statistic. Also, for slope-zero tests, \(t^2 = F\).
2. The intercept can also be tested. Be careful, though: intercept interpretation can be useless when \(X=0\) is economically impossible.
3. The p-value is the smallest significance level at which you reject the null. Smaller p-value means stronger evidence against the null, not bigger economic importance.

Indicator Variable Case

1. An indicator variable takes only 0 or 1. What is indicator variable: a switch showing whether a condition is absent or present.
2. If \(X\) is an indicator, the intercept is the mean of the zero group, and the slope is the difference in means between the one group and the zero group.
3. This is just regression wearing a very practical costume. You are testing whether two group averages differ, but through the regression framework.

ANOVA, Forecasting, and Prediction Intervals

1. The ANOVA table organizes sums of squares, degrees of freedom, mean squares, and the F-statistic in one place.
2. In simple regression, ANOVA is not decoration. It is the machine room where you read fit, error size, and the ingredients for later tests.
3. A forecasted dependent value is:

1. This gives the line's best predicted value, not a guaranteed realized value. Real observations still wander around the line.
2. That is why you need the standard error of forecast:

$$ s_f = s_e \sqrt{1 + \frac{1}{n} + \frac{(X_f-\bar{X})^2}{\sum (X_i-\bar{X})^2}}

$$ 69. Forecast uncertainty rises when \(s_e\) is high, sample size is small, or the forecasted \(X_f\) sits far from the sample mean \(\bar{X}\). 70. The prediction interval is:

1. Why is the interval wider far from \(\bar{X}\): you are extrapolating toward the edges, where the line has less support from the data.
2. If two models give the same fitted value but one has lower \(s_f\), that lower-\(s_f\) model gives the tighter and more useful prediction interval.

Functional Forms

1. Not every relationship is naturally linear. Sometimes the rescue move is not a new theory but a transformation.
2. A log-lin model logs the dependent variable only:

1. In log-lin, the slope measures the relative change in \(Y\) for an absolute change in \(X\). You must antilog forecasts to get back to plain \(Y\).
2. A lin-log model logs the independent variable only:

1. In lin-log, the slope measures the absolute change in \(Y\) for a relative change in \(X\).
2. A log-log model logs both variables:

1. In log-log, the slope is elasticity. What is elasticity here: the relative change in \(Y\) for a relative change in \(X\).
2. The correct functional form is chosen by fit measures and residual behavior together. Never marry a higher \(R^2\) if the residuals still look suspicious.

Final Exam Traps

1. If residuals are random, that is a good sign. If residuals show shape, grouping, or rhythm, the model is talking back and saying something is wrong.
2. Do not confuse residuals with error terms, \(R^2\) with significance, or a fitted value with a prediction interval.
3. If the intercept makes no economic sense, say so directly. The formula still exists even when the story at \(X=0\) is nonsense.
4. In simple regression, the exam loves these identities: \(R^2 = r^2\) and, for testing slope equal to zero, \(t^2 = F\).