Random number generation is an important task in modern science. A variety of quantum random number generation protocols have been proposed and realized. These protocols, however, are all based on two parties. Based on the weak measurement technique, we propose and realize a quantum random number generator among three observers. The violation of a double classical dimension witness based on the determinant value is first observed in experiment. With the heralding single-photon source, our experimental setup attains the independent assumption and the dimension assumption, which means our setup is semi-device-independent (DI). This Letter sheds new light on generating DI-type random number among multi-user and it has potential application prospect on the quantum cryptography and quantum random number in network environment.