This probabaly didn't scan as well as it should have because the original story had Clemenza as the traitor but the actor wanted too much and wouldn't gain the weight required for an older fat clemenza. The Pentangeli character was a rewritten invention for the purpose and agrees to go to the meeting to send a signal to Roth that Michael's family is failing. Roth's plan is to make him think that Michael arranged the killing as part of the bigger deal with Roth (that Michael mentioned at the meeting at Pentangeli's house). Roth wanted Pentangeli alive and to fall into the hands of the FBI to turn him against Michael ("no one uses guns" at the house in Miami) hence "MC says hello" and the bartender saying "not here Carmine". The cop discovery was part of the plan ("is that something on the floor" was not natural). These remarks were said for Pentangeli to hear - they serve no purpose with him being dead.
The FBI probably is in on the scheme, to take Pentangeli into the witness program. Michael has no idea where he is until Tom reveals at the meeting in Tahoe that he is a star witness "Roth played that superbly". (Maybe a weakness in the script that Michael was not concerned in the interim about his disappearance or the lack of a funeral - maybe he was and we just don't know) Michael's only play is to convince Pentangeli that he didn't betray him by showing that he is solid with Pentangeli's older brother and his family which wouldn't have been the case had he set up Pentangeli.
The early scene where Pentangeli baulks at doing a deal with the brothers and appears to disrespect Michael is to his loyalty to Michael in accepting his request to play nice. Pentangeli conned by Roth feels guilty in having betrayed Michael (but not in testimony) and takes the honorable way out. Tom's visit gives him the security to do this knowing that his family would be taken care of.

I can see that. Pentangeli was a pawn in chess in Corleone's and Roth's eyes.
Maybe a big Mafioso in New York, but NOTHING compared to Michael Corleone or the seemingly middle class, Miami Fla residing Jewish big shot Hyman Roth. They were all SCUM, except for the prostitute that was murdered, Kay and the children. But Frankie "Five Angels" Pentangeli was NO angel himself.

I think the Rosato Bros. had every intention of killing Pentangeli in the barthe killer says 'Michael Corleone says hello', to let Frankie know it's Michael killing him.
BUTthe cop comes into the bar and interrupts the murder, and Frankie lives.
It's obvious that wasn't supposed to happen.
So it doesn't make sense that Roth wanted Frankie to think Michael approved the hit. Frankie was gonna DIE, the intention wasn't to scare Frankie but to kill him. Remember at the end, in that conversation you mention with Hagen? Michael says incredulously 'Pentangeli is alive?!' Why would Michael think he's dead if he hadn't ordered it? Pentangeli was a loose cannon. He publicly went against Michael, disrespectfully walking out on him. Michael wanted him dead.
You say that Roth wanted the murder to appear bothched, right? but if you watch,
Carmine and the bartender can't believe a cop comes into the barCarmine pulls out his gun, ready to kill the cop but the bartender screams 'Not here Carmine!'
OK, it's possible that if the cop never came in, then Coppola has to show how the Rosato's 'kill' Frankie how would they fake that? Maybe Coppola didn't think a faked murder would play well. The whole Rosato Bros. -Pentangeli part of the plot has always raised questions.

What makes it so tragic is that Frankie didn't even WANT to have a meeting with the Rosatos. He hated them. He went to this meeting because Michael wanted him to.
Never hate your enemies. It affects your judgment.
-Michael Corleone

I fully agree with what you're saying. Michael went too far out of line (and was condemned to life-long isolation, being his own worst enemy). A cold and paranoid prick, even to those who maintained their loyalty to him.
Obvious statement of the abuse of power and authority, and how weak dicks hide safely in its confines.
As to what would have occurred if Frankie wasn't dissuaded by his brother's appearance, I believe that Michael could have been convicted of at least a few of the perjury charges, which would have put him away for some time. He might have been killed in prison.
Can't recall all the details. Frankie's testimony would have screwed Michael.
And Roth also screwed Pentangelis, fooling him into believing that Michael pulled the attempted hit on him. "Michael Corleone says Hello"