The exchange is effected by saying “Let this animal become a sacrifice instead of that one.” “That one” animal, on which the exchange is based, must be his, otherwise, the declaration does not work. Therefore, Kohanim can make exchanges for animals that they sacrifice and that belong to them. However, they can't make exchanges for animals that were given over to them by a Jew to be sacrificed – despite the fact that after it is slaughtered, they can eat its meat. Regular Jew can make exchange for his sacrifices, even though by now the sacrifice really belong to the Temple – but he has some ownership in it, since the sacrifice achieves his atonement.
Even though a first-born animal is given over to a Kohen and is his property, the Kohen cannot make exchange for it. Rabbi Yochanan ben Nuri asked, “Why not?” Rabbi Akiva explained, “They are like all other sacrifices.” Rabbi Yochanan objected, “But a first-born is his property!” Rabbi Akiva replied, “When did the first-born become sanctified? When it was born, in the house of its owner! It and its exchange must be in the same domain, but now it is in the hand of the Kohen, thus exchange is not possible.”
Art: William-Adolphe Bouguereau - A Newborn Lamb