A new heuristic optimization algorithm is presented to solve the nonlinear optimization problems. The proposed algorithm utilizes a stochastic method to achieve the optimal point based on simplex techniques. A dual simplex is distributed stochastically in the search space to find the best optimal point. Simplexes share the best and worst vertices of one another to move better through search space. The proposed algorithm is applied to 25 well-known benchmarks, and its performance is compared with grey wolf optimizer (GWO), particle swarm optimization (PSO), Nelder-Mead simplex algorithm, hybrid GWO combined with pattern search (hGWO-PS), and hybrid GWO algorithm combined with random exploratory search algorithm (hGWO-RES). The numerical results show that the proposed algorithm, called stochastic dual simplex algorithm (SDSA), has a competitive performance in terms of accuracy and complexity.