HIV-1 Nef is an important pathogenic factor for HIV/AIDS pathogenesis. Studies have shown that the association of Nef with the inner leaflet of the plasma membrane and with endocytic and perinuclear vesicles is essential for most activities of Nef. Using purified recombinant proteins in pull-down assays and by co-immunoprecipitation assays we demonstrate that Nef binds directly and specifically to all GABARAP family members, but not to LC3 family members. Based on nuclear magnetic resonance (NMR) experiments we showed that Nef binds to GABARAP via two surface exposed hydrophobic pockets. S53 and F62 of GABARAP were identified as key residues for the interaction with Nef. During live-cell fluorescence microscopy an accumulation of Nef and all GABARAP family members in vesicular structures throughout the cytoplasm and at the plasma membrane was observed. This plasma membrane accumulation was significantly reduced after knocking down GABARAP, GABARAPL1 and GABARAPL2 with respective siRNAs. We identified GABARAPs as the first known direct interaction partners of Nef that are essential for its plasma membrane localization.