THE CASE FOR INFANT BAPTISM
1. One people
The people of God in the New Testament era are not a brand new entity; they are like branches grafted onto the 'olive tree' of the people of God before Jesus, most of whom were Jews (see Rom. 11:17). This is why believers in Christ are called the 'children of Abraham' (Gal. 3:7).
2. One Covenant
God's covenant with Abraham (a relationship with them which included wonderful promises) was mainly about spiritual and eternal blessings: In Gen. 17:7 God says that he is establishing an everlasting covenant between himself and Abraham and his descendants after him, to be his God and the God of his descendants after him. So it is about God being his God. This is just like Rev. 21:3 and Heb. 8:10 in the NT. So the 'new covenant' in Hebrews 8 is not brand new, but a sort of NT version of the covenant with Abraham.
3. Children in the Covenant
God included Abraham's children in this covenant. This did not mean that every single one of them would necessarily be saved, but it did mean that quite a number would be, and also that there was something special about being in a godly family. Indeed Paul says that until the NT era there was something special about being a Jew (Rom. 3:1-2).
4. The Covenant Sign
Furthermore, as an outward sign and seal of the blessings of salvation to Abraham and his descendants, the sons of the people of God were to be circumcised at 8 days old, Gen,17:10ff (and Rom. 4:11 shows that circumcision had to do with the righteousness of God, put to our account by faith, in other words to do with justification by faith!
5. Children Still in the Covenant
Now this principle of the children of the people of God being in some real sense included within the covenant people and having a promise that in general God will save them - this principle is not abolished with the coming of Jesus. After all, that would mean that after Jesus' coming things were in one way not as good for believing parents as before! Rather, this principle is added to: there are special promises that God is particularly willing to save the children of sincere Christians, and in addition everybody else from non-Christian backgrounds is invited too! This is what Peter says on the Day of Pentecost: “The promise is to you and to your children and to all who are far off, as many as the Lord our God shall call.” (Acts 2:39)
How else can we explain the incidental comment of Paul in 1 Cor. 7:14 that thge children of Christians, even of only on Christian parent – are 'holy'? (The sanctification of the non-Christian partner in that context no doubt has to do with the marriage being regarded as genuine and acceptable in the sight of God).
6. Baptism replaces Circumcision
Circumcision, like the feast of Passover, is not a Christian ordinance; it has passed away as a shadow (see Col. 2:17). They both involved the shedding of blood, pointing forward to Christ's sacrifice. The Lord's Supper has replaced the Passover - it was instituted at a Passover meal! What has replaced circumcision? Col. 2:11-12 speaks of circumcision and baptism as if they have the same spiritual meaning. And if, as seems likely that at least some of the households who were baptised in the book of Acts included small children or babies, this would also fit in with baptism taking the place of circumcision.
Conclusion
So, just as believing Israelites in OT days circumcised their boys in the faith and hope that God would be their God, so Christian parents today can baptise their children (the NT is better in every way: girls can be baptised as well!) in faith that sooner or later God will save their children, giving them a new heart and forgiving their sins through Jesus' blood. In doing this, God will often use the way the parents bring them up and the teaching they give, and will also be answering their prayers. The baptism is an outward sign of the promise of God to change their hearts, wash their sins away, and unite them to the Lord Jesus.
PS; If you were baptised as a child when your parents were not Christians, don't worry: you believe now, and by faith you can receive the encouragement of the sign that you know was applied to you. (After all, God's grace does in a general way precede faith, for faith itself is his gift). Yes, you should not have been baptised then, but the point is, you were! Baptism is an outward rite, and if it has been performed in a church that holds to the doctrines of the Trinity and of Jesus Christ being the living Saviour of sinners, it is right to assume that in God's sight you are baptised.
See also Bringing Up Children for God and The Mode and Meaning of Baptism