Problems tagged with "least squares"
Problem #03
Tags: linear regression, least squares
Suppose a line of the form \(H(x) = w_0 + w_1 x\) is fit to a data set of points \(\{(x_i, y_i)\}\) in \(\mathbb R^2\) by minimizing the mean squared error. Let the mean squared error of this predictor with respect to this data set be \(E_1\).
Next, create a new data set by adding a single new point to the original data set with the property that the new point lies exactly on the line \(H(x) = w_0 + w_1 x\) that was fit above. Let the mean squared error of \(H\) on this new data set be \(E_2\).
Which of the following is true?
Solution
\(E_1 > E_2\)
Problem #12
Tags: least squares
Suppose a data set \(\{\nvec{x}{i}, y_i\}\) is linearly-separable.
True or false: a least squares classifier trained on this data set is guaranteed to achieve a training error of zero.
Solution
False
Problem #20
Tags: least squares
In the following, let \(\mathcal D_1\) be a set of points \((x_1, y_1), \ldots, (x_n, y_n)\) in \(\mathbb R^2\). Suppose a straight line \(H_1(x) = a_1 x + a_0\) is fit to this data set by minimizing the mean squared error, and let \(R_1\) the be mean squared error of this line.
Now create a second data set, \(\mathcal D_2\), by doubling the \(x\)-coordinate of each of the original points, but leaving the \(y\)-coordinate unchanged. That is, \(\mathcal D_2\) consists of points \((2x_1, y_1), \ldots, (2x_n, y_n)\). Suppose a straight line \(H_2(x) = b_1 x + b_0\) is fit to this data set by minimizing the mean squared error, and let \(R_2\) be the mean squared error of this line.
You may assume that all of the \(x_i\) are unique, as are all of the \(y_i\).
Part 1)
Which one of the following is true about \(R_1\) and \(R_2\)?
Solution
\(R_1 = R_2\)
Part 2)
Which one of the following is true about \(a_0\) and \(b_0\)(the intercepts)?
Solution
\(a_0 = b_0\)
Part 3)
Which one of the following is true about \(a_1\) and \(b_1\)(the slopes)?
Solution
\(a_1 > b_1\)