The development of next-generation sequencing technology has provided us with great convenience in genetic association studies and many effective analysis methods were proposed continuously. However, population stratification is still a major issue in current genetic association studies. Many existing methods have been developed to remove the bias due to population stratification for common variant association studies, but such methods may be not effective for rare variant, which will lead to power reduction. Therefore, in this paper, we develop a principal component analysis strategy (called PC-LLR) based on local linear regression method to eliminate population stratification effect in both rare variant and common variant association studies. Simulation results indicate that the new PC-LLR method can eliminate population stratification effect well. It has correct type I error rates in all cases and higher powers in most cases, while most existing methods have inflated type I error rates at least in some cases. We also demonstrate that the PC-LLR is more effective to eliminate population stratification effect through applying the PC-LLR to the whole-exome sequencing data set from genetic analysis workshop 19 (GAW19).