Regardless of religion, taking a life can be considered murder. Therefore, regardless of faith or lack of faith, an abortion can be considered infanticide. This could be argued to be wrong or amoral because it deprives the population of variety or potential to solve a major problem.
The concept of "life begins at conception or prior" is inherently religious in nature (because it ascribes value to a living creature beyond consciousness, usually but not always in a pseudo-spiritual sense), even if the people who follow it do not consider themselves religious.
As an unrelated aside, one of my first year law classes had an amusing discussion in it, where several students claimed that there were things considered 'wrong' that were universally and fundamentally wrong, regardless of particular moral code or society. The teacher said, "Okay, let's make a list of some." The first thing most students suggested was "Murder/killing someone."
The teacher then said: "Self-defence. Euthanasia." It transpired that we couldn't come up with a single thing that was fundamentally wrong across all civilisation.