This paper presents two novel Lyapunov functionals for analyzing the stability of neural networks with time-varying delay. Based on our newly proposed Lyapunov functionals and a relaxed Wirtinger-based integral inequality, new stability criteria are derived in the form of linear matrix inequalities. A comprehensive comparison of results is given to illustrate the newly proposed stability criteria from both the conservative and computational complexity point of views.