Wing Kinematics-Based Flight Control Strategy in Insect-Inspired Flight Systems: Deep Reinforcement Learning Gives Solutions and Inspires Controller Design in Flapping MAVs

Flying insects exhibit outperforming stability and control via continuous wing flapping even under severe disturbances in various conditions of wind gust and turbulence. While conventional linear proportional derivative (PD)-based controllers are widely employed in insect-inspired flight systems, they usually fail to deal with large perturbation conditions in terms of the 6-DoF nonlinear control strategy. Here we propose a novel wing kinematics-based controller, which is optimized based on deep reinforcement learning (DRL) to stabilize bumblebee hovering under large perturbations. A high-fidelity Open AI Gym environment is established through coupling a CFD data-driven aerodynamic model and a 6-DoF flight dynamic model. The control policy with an action space of 4 is optimized using the off-policy Soft Actor-Critic (SAC) algorithm with automating entropy adjustment, which is verified to be of feasibility and robustness to achieve fast stabilization of the bumblebee hovering flight under full 6-DoF large disturbances. The 6-DoF wing kinematics-based DRL control strategy may provide an efficient autonomous controller design for bioinspired flapping-wing micro air vehicles.