We have previously formulated an abstract dynamical system for networks of spiking neurons and derived a formal result that identifies the criterion for its dynamics, without inputs, to be "sensitive to initial conditions". Since formal results are applicable only to the extent to which their assumptions are valid, we begin this article by demonstrating that the assumptions are indeed reasonable for a wide range of networks, particularly those that lack overarching structure. A notable aspect of the criterion is the finding that sensitivity does not necessarily arise from randomness of connectivity or of connection strengths, in networks. The criterion guides us to cases that decouple these aspects: we present two instructive examples of networks, one with random connectivity and connection strengths, yet whose dynamics is insensitive, and another with structured connectivity and connection strengths, yet whose dynamics is sensitive. We then argue based on the criterion and the gross electrophysiology of the cortex that the dynamics of cortical networks ought to be almost surely sensitive under conditions typically found there. We supplement this with two examples of networks modeling cortical columns with widely differing qualitative dynamics, yet with both exhibiting sensitive dependence. Next, we use the criterion to construct a network that undergoes bifurcation from sensitive dynamics to insensitive dynamics when the value of a control parameter is varied. Finally, we extend the formal result to networks driven by stationary input spike trains, deriving a superior criterion than previously reported.