This paper presents a novel Sparse truncated Singular Value Decomposition (SSVD) formulation that can select the best submatrix precisely up to a given size to maximize its truncated Ky Fan norm. The fact that the proposed SSVD problem is NP-hard motivates us to study effective algorithms. To do so, we first reformulate SSVD as a mixed-integer semidefinite program, which can be solved exactly for small- or medium-sized instances. We next develop three selection algorithms and two searching algorithms. We prove the approximation ratios for all algorithms. Our numerical study demonstrates the efficiency of the proposed algorithms. Finally, all our analysis can be extended to row-sparse PCA.