Having one baby, or a baby and a toddler, or a baby and a toddler and a preschooler, is not a sufficient gauge* of whether one "can do this." Those years are wretched. They are not representative of the familial experience. I remember reading once in some Dobson book on parenting that the hypothetical mother whose personal well-being worried him most was the one home with two or three kids below preschool age. Quitting at that point is like calling your mom to pick you up from boot camp or giving up on getting in shape because your first time out you get a stitch in your side. You've just got to get past it, when there's the most work to do and no one to help.
*in most cases, in most cases, in most cases.