The first complication may arise if the now-deceased had two wives, one of which happens to be forbidden to the live brother. For example, she could be the brother's daughter, the dead man's niece. The brother cannot marry his daughter, and the other wife is automatically released from either yibum or chalitzah.
The second complication may arise when we are in doubt whether the deceased man married his niece or not. This doubtful marriage is the focus of our investigation here. How could it happen? Here is how. Instead of giving his bride the betrothal document, he threw it at her, and it fell to the ground. Now, if it fell closer to her, she would be married. If it fell closer to him, she is not. If it fell in the middle, we have a doubt.
The solution? The alive brother should release both wives and cannot marry either. Thus, he does the halitzah to both but cannot do a yibum.
Art: The Brothers Konrad und Franz Eberhard by Johann Anton Ramboux