Excessive moisture and direct sunlight are the two biggest problems for speakers, except of course over-driving them in some way. As long as you don't push 'em too hard they'll last decades provided you take care of them.
Also, don't store them in a position where the cone faces up or down. The cones can sag after several years, especially subwoofers.
When you over-drive a speaker, it usually starts sounding very distorted.
As for the cone-up or cone-down thing, I was mostly talking about storage. If you lay speakers on their backs (cones facing up) and store them for a few years, you may find the suspension has sagged over time.
I have seen some subwoofer cabinets designed to face the woofer down though. It's an interesting design, and makes the woofer cone "invisible." But the disadvantage is that the cone weight will tend to shift it downwards, which will only get worse over time.
Curious fingers are usually the cause of dented voice coil covers. Something or someone has pushed them in. They do contribute to the sound in most speakers, but not subwoofers. Subs usually have a cap that's harder to dent too.
Fortunately, it's usually an inexpensive and easy fix. Take your dented speakers to a speaker repair shop and they'll replace those dented caps.
Positioning a driver "face-up" or "face-down" can make the suspension sag over time, especially subwoofers with heavy cones and loose suspensions.
Will this happen to the subwoofer even when it is used regularly and used in a normal position? How about when it is used regularly and the positioning is not correct, what will happen to the subwoofer?