Fuzzing is a technique of finding bugs by executing a target program recurrently with a large number of abnormal inputs. Most of the coverage-based fuzzers consider all parts of a program equally and pay too much attention to how to improve the code coverage. It is inefficient as the vulnerable code only takes a tiny fraction of the entire code. In this article, we design and implement an evolutionary fuzzing framework called V-Fuzz, which aims to find bugs efficiently and quickly in limited time for binary programs. V-Fuzz consists of two main components: 1) a vulnerability prediction model and 2) a vulnerability-oriented evolutionary fuzzer. Given a binary program to V-Fuzz, the vulnerability prediction model will give a prior estimation on which parts of a program are more likely to be vulnerable. Then, the fuzzer leverages an evolutionary algorithm to generate inputs which are more likely to arrive at the vulnerable locations, guided by the vulnerability prediction result. The experimental results demonstrate that V-Fuzz can find bugs efficiently with the assistance of vulnerability prediction. Moreover, V-Fuzz has discovered ten common vulnerabilities and exposures (CVEs), and three of them are newly discovered.