At that time too in the early 80's we were less than a decade out of Vietnam and that war left a bad taste in a lot of people's mouths so it's possible the town kids resented the cadets, plus like another said, it's likely that a majority of the girls were from that town which of course wouldn't sit well with the local boys.

I remember asking a guy this same question he was a cadet like these boys and transferred to my high school shortly after this film came out. He said that a lot of these military academies buy land in in small towns because they can get it cheap. So you get a lot of lower-class "townie" types who resent the presence of these guys living there, stealing their girls, etc. Kinda like the townies in "An Officer and a Gentlemen" harassing Richard Gere in that one bar scene.