Motor brain-computer interfaces (BCIs) have gained growing research interest in motor rehabilitation, restoration, and prostheses control. Decoding upper-limb movement direction with noninvasive BCIs has been extensively investigated. However, few of them address the intervention of cognitive distraction that impairs decoding performance in practice. In this study, we propose a novel decoding model with invariant patterns in embedding manifold on a mixed dataset pooled from electroencephalograph (EEG) signals under different attentional states. We reconstruct an embedding low-dimensional manifold that intrinsically characterizes movements of the upper limb and transfer patterns of neural activities decomposed from brain functional connectivity (FC) to the manifold subspace to further preserve movement-related information. Experimental results showed that the proposed decoding model had higher robustness on the mixed dataset of attentive and distracted states compared to the baseline method. Our research provides insights into modeling a uniform underlying mechanism of movement-related EEG signals and can help enhance the practicability of BCI systems under real-world situations.