Consent processes have attracted significant research attention over the last decade, including in the global south. Although relevant studies suggest consent is a complex negotiated process involving multiple actors, most guidelines assume consent is a one-off encounter with a clear 'yes' or 'no' decision. In this paper we explore the concept of 'silent refusals', a situation where it is not clear whether potential participants want to join studies or those in studies want to withdraw from research, as they were not actively saying no. We draw on participant observation, in-depth interviews and group discussions conducted with a range of stakeholders in two large community based studies conducted by the KEMRI Wellcome Trust programme in coastal Kenya. We identified three broad inter-related rationales for silent refusals: 1) a strategy to avoid conflicts and safeguard relations within households, - for young women in particular-to appear to conform to the wishes of elders; 2) an approach to maintain friendly, appreciative and reciprocal relationships with fieldworkers, and the broader research programme; and 3) an effort to retain study benefits, either for individuals, whole households or wider communities. That refusals and underlying rationales were silent posed multiple dilemmas for fieldworkers, who are increasingly recognised to play a key interface role between researchers and communities in many settings. Silent refusals reflect and reinforce complex power relations embedded in decisions about research participation, with important implications for consent processes and broader research ethics practice. Fieldworkers need support to reflect upon and respond to the ethically charged environment they work in.