I use SVD (Singular Vector Decomposition) to solve eigenvalue problems (EVD) (note that you don't do so when the matrix is very large) often. But thru years, I gradually forget why we can do so.
I did some revisiting today and found a good explanation. Equation 11 on this webpage (http://mathworld.wolfram.com/EigenDecomposition.html) explains everything clearly. The right-hand side is the form of SVD. Eq. (11) also reveals the "remarkable relationship between a diagonalized matrix, eigenvalues, and eigenvectors follows from the beautiful mathematical identity (the eigen decomposition)" [See also]. And the reason Eq.(11) holds is from Equations 1 to 10.
For more info on the connection between SVD and EVD, you can read this section in Wikipedia:http://en.wikipedia.org/wiki/Singular_value_decomposition#Relation_to_eigenvalue_decomposition
See also: Matrix Diagonalization, Wolfram MathWorld, http://mathworld.wolfram.com/MatrixDiagonalization.html