It's the composer's personal feeling. When the composer gets bored with a scene, or the scene feels dry emotionally and the composer feels the scene could use a little warm undercurrent, the composer wants to add music there. When a movie seems to be drowning in melodrama, the composer wants to withhold music, to simply let the dialogue and sound-effects speak for themselves, to make the scene feel more real.
Too little music and the movie is a documentary. Too much music and the movie drowns in melodrama. Just the right amount of music, strategically placed, is masterful, magical, perfectly accentuates just the right moments.
Thankfully, the director DOES know this, and helps the composer with it. Hopefully the director and the composer have a harmonious relationship, so their styles don't clash. Ultimately, the composer has to defer to the director, who is "the boss" in the creation of a movie.
John Williams wrote a whole bunch of music for the scenes of Han searching for Luke after Luke escapes from the Wampa's cave in the ice planet Hoth in The Empire Strikes Back that was not used in the film. Apparently Executive Producer George Lucas, who had final cut and was in charge of all aspects of the film's creation, felt the documentary approach, with just dialogue and sound-effects, worked better for this stretch of film, made Hoth feel like more of a frozen wasteland.
John Williams is one such composer who is a master at this process, called "spotting". Steven Spielberg commented on this about John's music in Jaws ("John didn't want music to be a red herring; he only wanted music to signal the actual arrival of the shark."), as did Midway's director, Jack Smight, where, in Midway, John's music had to take a back seat to the sound-effects of the Sensurround process. John has no ego about his music. He has an intelligence. As Jack Smight said, "He knows right where music should go.".

That may be a moot question. I am familiar with Bernstein's concert suite of his On The Waterfront. I finally saw the film and there were long sections where the music seemed way out of place. That may have not been Bernstein's fault. I doubt he was given final say in mixing levels or placement.

A film composer named Fred Karlin wrote a book including that topic, I believe:
In 1990 he wrote, with Rayburn Wright and John Williams, a textbook on film scoring called ''On the Track,'' which remains in print.
http://www.nytimes.com/2004/05/10/arts/fred-karlin-67-film-composer.html
https://en.wikipedia.org/wiki/Fred_Karlin
Relax in the safety of your own delusions.- J.R. Bob Dobbs