The Talmud tries to ascribe this rule to Rabbi Akiva, who considers being in the airspace the same as resting on the ground. By extension, an object in the hand is resting on the ground. But it considers such attribution too tenuous.
Then, it tries to say that it is the opinion of Rabbi Yehudah the Prince, who seemingly did not require the area of four by four. That does not work either because Rabbi Yehudah only talked about an area with a roof, and we have no roof in the street.
It tries to change the ruling, saying it should be "basket in his hand." The problem is that the rule needs to mention a basket, and it does not.
Finally, the Talmud concludes that because the hand can grasp things, it is considered by the person as if it were four by four "arms" (amot) - and that is why the poor man is liable.
Art: Breton Woman with a Basket by John Singer Sargent