I used to feel that way, but broadcasts of other sports get along without treating the entire audience like newbies. After a while, it gets annoying having tight and loose explained for the umpteenth time. Nobody finds it necessary to explain every week where the 'key' is on a basketball court, what 'icing' is during a hockey match, or what constitutes 'offsides' in soccer.
Incidentally, I don't know what any of those are, and I'm sure knowing would enhance my relative enjoyment, but not knowing doesn't stop me from watching soccer or hockey occasionally. I wouldn't watch basketball willingly no matter how much more I might learn about it.