In his memoirs entitled "Chronicles" (2004) Dylan wanted to set fire to them. He referred to the hippies as "gangs of drop-outs and druggies." They turned up at his family home in Woodstock and according to him, walked over his roof or tried to break in. Dylan went on to say: "I had very little in common with and knew even less about a generation that I was supposed to be the voice of."

Yeah I heard that as well, that the original site for the Woodstock Festival was right by his house, and he was mad that they were hanging around there. I also read that he said that one of the reasons he refused to play because he just didn't want to perform in front of "a bunch of unwashed hippies"
"Sometimes it's right to do the wrong thing, and right now is one of those times."

He certainly wasn't into the whole 'peace and love' vibe. Seem to remember him once mocking hippies-as a bunch of kids on acid with flowers in their hair and saying he couldn't understand why anyone would take them seriously.
There's also a feeling that the organisers of the Woodstock festival chose their venue because it was near to where Dylan was then living-the idea being to tempt him back onto the stage but he point blank refused to play. I know he also wrote in his autobiography about fans breaking into his house and following him around-seem to remember he wrote something about wan5b4ting to shoot them!