Digital phantoms continue to play a significant role in modeling and characterizing medical imaging. The currently available XCAT phantom incorporates both the flexibility of mathematical phantoms and the realistic nature of voxelized phantoms. This phantom generates images based on a regular breathing pattern and can include arbitrary lung tumor trajectories. In this work, we present an algorithm that modifies the current XCAT phantom to generate 4D imaging data based on irregular breathing. First, a parameter is added to the existing XCAT phantom to include any arbitrary tumor motion. This modification introduces the desired tumor motion but, comes at the cost of decoupled diaphragm, chest wall and lung motion. To remedy this problem diaphragm and chest wall motion is first modified based on initial tumor location and then input to the XCAT phantom. This generates a phantom with synchronized respiratory motion. Mapping of tumor motion trajectories to diaphragm and chest wall motion is done by adaptively calculating a scale factor based on tumor to lung contour distance. The distance is calculated by projecting the initial tumor location to lung edge contours characterized by quadratic polynomials. Data from ten patients were used to evaluate the accuracy between actual independent tumor location and the location obtained from the modified XCAT phantom. The RMSE and standard deviations for ten patients in x, y, and z directions are: (0.29 ± 0.04, 0.54 ± 0.17, and0.39 ± 0.06) mm. To demonstrate the utility of the phantom, we use the new phantom to simulate a 4DCT acquisition as well as a recently published method for phase sorting. The modified XCAT phantom can be used to generate more realistic imaging data for enhanced testing of algorithms for CT reconstruction, tumor tracking, and dose reconstruction.