We study two modifications of the Post Correspondence Problem (PCP), namely (1) the bi-infinite version, where it is asked whether there exists a bi-infinite word such that two given morphisms agree on it, and (2) the conjugate version, where we require the images of a solution for two given morphisms are conjugates of each other. For the conjugate PCP we give an undecidability proof by reducing it to the word problem for a special type of semi-Thue systems and for the bi-infinite PCP we give a simple argument that it is in the class Σ₂⁰ of the arithmetical hierarchy.