"Those flashbacks were done to establish Joe Buck as this damaged and emotionally and intellectually stunted person who learns to become a wiser and mature human being both through his often demeaning experiences in New York City and, more importantly, through his friendship with Ratso…"
I have seen this movie once or twice and I am not so sure I recognized, as you may have, where Buck changed or went through the character arc as you describe?
He injures or kills a man and steals his money before they get on the bus.
The only place where I can think of he might have "become wiser and mature" is where he throws away his cowboy outfit.
Is that what you are thinking of or something else?
"Please vote to preserve the unique character of Warren…" - Robert Duvall

Those flashbacks were done to establish Joe Buck as this damaged and emotionally and intellectually stunted person who learns to become a wiser and mature human being both through his often demeaning experiences in New York City and, more importantly, through his friendship with Ratso, who's probably the first person Joe Buck has ever had a substantial and meaningful relationship with.
You've seen Guy Standeven in something because the man was in everything.

I know this was discussed a couple of years back but the idea was that Crazy Annie was something of a town whore. For whatever reason, she attached herself to Joe Buck and the other guys who were scoring easy and regular sex from her were suddenly shut off. So, one night, the crew descended on the pair (while parking), gang rapes Annie - and, in a fit of sexual frenzy, rape Joe Buck while they're at it.
In Joe's world, as he later tells Ratso, that Crazy Annie had to be "taken away" because he was some kind of mega stud. In reality, she was already suffering from an undiagnosed mental distorter and was left in PTSD state by the attack. The haunting part was hearing her echo "He's the one, he's the only one" while being removed. While she did say that before all this happened, it took a whole new meaning now.

Joe too? I think I have to re-watch it sometime. Why would they even rape Joe? Gay attackers? And why were they being raped in the first placesome many questions lol. I think I didn't understand those flashbacks at all for the most part.

They both were raped. They pulled his pants down and the scene flashes to his grandmother smiling a devilish grin as she gives him an enema as a boy. Joe is associating that moment with getting penetrated.