The point of the scene was not only a class issue but the fact that what they were doing was so far reaching it could put even the cleaning lady on unemployment.

correct. That's how it is in NYC. People with money speak in front of maids and drivers etc as though they are not there.
Yup. I'm from NYC and as soon as I saw the cleaning lady, I said, "Lemme guess, they're not even going to say hello to her."

correct. That's how it is in NYC. People with money speak in front of maids and drivers etc as though they are not there.
Defender of the weak, and enemy of the weak minded.

I saw the scene as representing the "invisibility" of the working-class to those who have money.

You missed the whole point of the scene

If you're thinking of "condescending", you're way off point. Why would they want to alter their speech to include the cleaning lady in the conversation?

They speak sort of elliptically precisely because they don't want the cleaning lady to understand what exactly they're talking about.
"facts are stupid things" - Ronald Reagan