Don't start with House by the Cemetery. It's good, but you're REALLY gonna wanna be familiar with Fulci's style with that movie. He's not the most narrative focused filmmaker, but House by the Cemetery makes absolutely NO sense. Also it has an annoying dubbed child actors. Not a bad movie, but it's not for the uninitiated.
Zombi 2 was my first one, and it's mighty entertaining. It's a good introduction to his style, it's very outragous and it's sleazy without being too gross (unlike the New York Ripper). The Beyond is also really good, more successful at an artistic level over Zombi 2, but I prefer Zombi 2. And I never saw Don't Torture A Duckling, which I hear is amazing, but I can vouch for a Lizard in a Woman's Skin if you want a good giallo from him.
100 Greatest Action Movies -
http://www.imdb.com/list/ls000708268/

A LIZARD IN A WOMAN'S SKIN
DON'T TORTURE A DUCKLING
NEW YORK RIPPER
Those are Fulci's best gialli (violent murder mystery/thrillers).
However, if you're only looking for gore and care nothing about good writing, there's always:
ZOMBIE
THE BEYOND
CITY OF THE LIVING DEAD

I second jayt-26's answer.

The Beyond
City of the living dead
And
House by the cemetery
Many would argue that I should have included zombie.

This message has been deleted.