[size=150][size=200][justify][/justify][/size][justify][/justify][/size]The ellipse is a level curve for the joint probability density function [math]f\left(x,y\right)[/math]. For convenience we chose the means as [math]\mu_X=\mu_Y=0[/math]. The [color=#f1c232]yellow line[/color] is supporting the major axis of the ellipse and joining the two points on the ellipse that are the furthest from each other (the points of tangency of the ellipse with its circumscribed circle). Note that in the case when [math]r=0[/math] and [math]\sigma_X=\sigma_Y[/math], every line through the origin is a line of best orthogonal fit, therefore there is no yellow line displayed.[br]The [color=#6aa84f]green line[/color] is the regression line [math]E(Y|X)=\alpha+\beta X[/math]. It minimises the sum of squared vertical distances. This regression line joins the two points where the ellipse is tangent to vertical lines. The simple explanation goes as follows: let's denote by [math](x_1,y_1)[/math] the coordinates of the rightmost point on the ellipse for instance. Then consider the restriction of the density function along the vertical line [math]X=x_1[/math]. This is a multiple of the density of the normal variable [math]Y|X=x_1[/math], (the conditional density) which takes every value twice except the value at its mode [math]y=\alpha+\beta x_1[/math]. Indeed, it is a unimodal density. Any other ellipse which is a level curve for [math]f(x,y)[/math] cuts the line [math]X=x_1[/math] in two points or not at all. So the point of tangency is precisely the point of coordinates [math](x_1,\alpha+\beta x_1)[/math].[br][br]Similarly, the [color=#ff00ff]magenta line[/color] is the regression line [math]E(X|Y)[/math] that minimises the sum of squared horizontal distances. This line joins the only two points where the ellipse is tangent to horizontal lines.[br][br][br]You may also observe that the line of best fit bisects the angle formed by the two regressions lines only in the case [math]\sigma_{X_{_{ }}}=\sigma_Y[/math].[br]