They are extremely damaging to things they touch (obviously) but the damage is very localized. You'll see one house destroyed and the one right beside it has nothing changed, not even leaves blown off the trees.
We had a tornado a couple of years ago that struck (and destroyed) a family member's house. However, it left her neighbor's house completely intact. It's wild to me how powerful a tornado is, yet how concentrated the damage can be!