He tends to have a suspicion. But he definitely follows the evidence to confirm.
I think a few times he mentions something along the lines of "You know, I wasn't sure until…"
So he doesn't always "know," but he has an idea.
So many stories, so little time.

I think he suspects very quickly. Then he has to gather as much evidence as possible against the perpetrator, meanwhile pretending to bumble and stumble along and be great friends with them.

It is, when he whistles. But he actually only does it in a fairly small number of episodes.

I was watching
Make Me A Perfect Murder
last night and Columbo quite clearly suspects Kay from their first meeting, when Flanagan gives her Mark's old job while he is in the office with them.
But in
Greenhouse jungle
i am pretty sure he suspected Kathy and her lover Ken first before he suspected Jarvis.

I read somewhere that when you hear Colombo whistling "This Old Man", it's because he's just figured out who the murderer is.

Yeah, sometimes he has the suspect right off the bat after they explain their alleged alibi and something just doesn't add up. The Johnny Cash episode Is kind of an example of this, but a little different because Columbo investigated the crime scene a bit before meeting Cash/Tommy Brown.
Several cases he mentions interviewing other potential suspects and says why they were cleared or unlikely.

If you watch it closely you can tell sometimes he hones in very early and by the questions he asks he suspects the perpetrator when meeting them. Other times you can see the gears running in Columbos' mind and can tell he starts out with them being on the list to being sure they are the guilt party as he interacts with the perpetrator.