Dissipative electromagnetic response and scattering geometry are potential sources for the appearance of a negative Casimir entropy. We show that the dissipative contribution familiar from the plane-plane geometry appears also in the plane-sphere and the sphere-sphere geometries and adds to the negative Casimir entropy known to exist in these geometries even for perfectly reflecting objects. Taking the sphere-sphere geometry as an example, we carry out a scattering-channel analysis, which allows us to distinguish between the contributions of different polarizations. We demonstrate that dissipation and geometry share a common feature making possible negative values of the Casimir entropy. In both cases there exists a scattering channel whose contribution to the Casimir free energy vanishes in the high-temperature limit. While the mode-mixing channel is associated with the geometric origin, the transverse electric channel is associated with the dissipative origin of the negative Casimir entropy. By going beyond the Rayleigh limit, we find even for large distances that negative Casimir entropies can occur also for Drude-type metals provided the dissipation strength is sufficiently small.