Although the aircraft had enough raw power to get into the air via its skids, it would have been a very difficult craft to taxi to its take off point. It had no wheels, which in any case may have bogged down in the sand. The usual method of steering an aircraft on the ground is by using the brakes individually - not possible as there were no wheels to brake, varying the power on each engine but again not possible as there is only one, or using the rudder, which can only be done when the craft is moving at some speed. Taxiing would have also used a great deal of fuel.

so what the people did was not so much "pull it" but to use force to turn the bird into the right direction, and in part make sure friction of the skids wouldn't cause it to fall on it's nose. - andy-1394
Yes. Dorfmann states this explicitly, that the men are just steering the Phoenix to the flat valley for take-off and not actually pulling it, as the engine does the actual locomotion. This is after Towns challenges him by saying that, at this point, the men barely have the strength to carry themselves.
Those are the headlines. Now for the rumors behind the news. - Firesign Theatre

Yes, the 'pulling' they did was just putting it in the right position from where a take-off was possible.

"Maybe I should go alone"

• Quint, Jaws.

the people didn't pull the plane, the power to move it came from the engine.
without enough airflow over the rudder there was not enough force to turn the bird. remember, it had some rudimentary skids and no steerable gear.
so what the people did was not so much "pull it" but to use force to turn the bird into the right direction, and in part make sure friction of the skids wouldn't cause it to fall on it's nose.
i could be wrong but i'm not