The reason you are not seeing 'alter' as a noun is because you did not read the lyrics closely enough. You copied them; it's there - altAr.
Altar, not alter.
An altar is a structure, often resembling a table, which is used in churches, temples and other houses of worship in various religions. It is used to lay out items for sacrifice rituals, to make offerings to deities. In some Christian churches, it is where the bread and wine are placed for communion.
In some religions, it is the place where a man and woman stand to receive the sacrament of marriage.