We report on the behavior of micron-sized prolate ellipsoids trapped at an oil-water interface. The particles experience strong, anisotropic, and long-ranged attractive capillary interactions which greatly exceed the thermal energy k(B)T. Depending on surface chemistry, the particles aggregate into open structures or chains. Using video microscopy, we extract the pair interaction potential between ellipsoids and show it exhibits a power law behavior over the length scales probed. Our observations can be explained using recent calculations, if we describe the interfacial ellipsoids as capillary quadrupoles.