Least Squares Regression Least Squares Regression Line of Best Fit. Imagine you have some points, and want to have a line that best fits them like this:. We can place the line "by eye": try to have the line as close as possible to all points, and a similar number of points above and below the line.