I have heard a fairly distinct difference. It is subtle, and requires a good amp and system to pick out quickly. I have heard other things which I'd attribute to circuit design and execution( from both topologies ).

SE gets the 'that's quite different' reaction. It is often just such a difference which can be pointed to as reason to say, "better". I have seen it the other way, listening to a bunch of SE and then plugging my PP amp. Usually it is a combo of things like power and ( IMO ) putting in a better amp in. Clean headroom is not to be sneezed at. A SE design with 3-4 watts is going to be running too close to the edge on a lot ov the peaks...

I am not considerig SE for power just now. Been messing about with phono stage design...and of course the Amp project for Group Build. good news there BTW, some 20W Peerless 20-20 models are on the way to me( and then on to the Winder ).
regards,
Douglas