Matthew never mentions a manger and never mentions a journey from Nazareth. In fact, Matthew mentions Nazareth only when Joseph, upon returning from Egypt, fears to live in Archelaus' territory, and decides to go to Galilee on an angel's warning. The implication is that Bethlehem is Joseph's home, and the couple were living there when Jesus was born.
Luke, on the other hand, says quite distinctly that the couple never goes to Egypt. During Mary's period of purification they remain near Jerusalem, presumably in Bethlehem; after Jesus is presented at the temple, they go at once to Nazareth.
We can't have contradictions, so Christian tradition has fused the stories, ignoring the little discrepancies. But the written text retains them.