Bone regenerative therapies have been explored using various biomaterial systems. Notably, collagen biomineralisation is believed to be essential for promoting bone regeneration. However, ideal bone repair materials with an appropriate mineralised matrix, superior osteogenic activity with early vascularisation, and recellularisation properties are still needed. This study aimed to develop a method to subject the decellularised cancellous bone matrix (DCBM) to ultrasound to obtain specific demineralisation to investigate the effects of DCBM with different degrees of mineralisation on proliferation and osteogenic differentiation in bone marrow-derived mesenchymal stem cells (BMSCs) and in repairing femoral bone defects in rabbits. We established an optimised native DCBM mineralisation ECM scaffold for bone regeneration. Upon complete decellularisation of the cancellous bone matrix, DCBMs with specific degrees of mineralisation were obtained. We comprehensively evaluated their bioactive components, minimal immunogenicity, ultra-micro-structural mechanical properties, and degree of mineralisation. Furthermore, specific mineralised DCBMs (obtained by low-temperature rapid ultrasound for 4 and 8 h) had prominent effects in promoting the osteogenic differentiation of BMSCs in vitro. Moreover, more newly formed trabeculae, vessels, and endochondral bone were also detected in the aforementioned groups during early-stage bone repair in vivo. The underlying mechanism might be mineralisation-related regulation and ultra-micro-structural mechanical properties. Thus, the present study shows that specific demineralised DCBM obtained under optimal conditions had superior properties to those of unmineralised or completely demineralised DCBM by promoting MSC osteogenic differentiation and initiating endochondral bone formation and de novo osteogenesis.