I think it's a regional thing. Because the modifieds are bigger up north, they generally have the better drivers, fields, etc. Around here in the Southeast, it's pretty much all late models, so that's where the short track talent is (with the exception of the Bowman Gray guys who are most certainly the exception to your "risk tearing up your stuff" argument as well as anything sensible about racing).
When the Whelen Southern tour has come to my local track, I have always been amazed by the cars themselves, and prefer them to late models. But the racing is always better there with late models because there is a higher level of competition. In fact, there was a year or two there where the Limited Late Models put on the best race every weekend, but most of the people that were in the class at the time have moved up to Late Models (or to other tracks) since then. It was hard racing every week, but it was also clean and respectful, much like I imagine you see with modifieds in the north.