For large-scale simulations, the data sets are so massive that it is sometimes not feasible to view the data with basic visualization methods, let alone explore all time steps in detail. Automated tools are necessary for knowledge discovery, i.e., to help sift through the data and isolate specific time steps that can then be further explored. Scientists study patterns and interactions and want to know when and where interesting things happen. Activity detection, the detection of specific interactions of objects which span a limited duration of time, has been an active research area in the computer vision community. In this paper, we introduce activity detection to scientific simulations and show how it can be utilized in scientific visualization. We show how activity detection allows a scientist to model an activity and can then validate their hypothesis on the underlying processes. Three case studies are presented.