From my comments on Belgic Confession Article 34:
Baptism replaces circumcision in this respect- both ceremonies identified members of the covenant people. There has always been a nation, though in Old Testament times that nation was a political and ethnic body, and now it is a church. But there is still a nation which is the covenant people of God, and this covenant people include believers and their children. Acts 2:39 says that the promise of salvation in Christ is to “you and to your children”. 1 Peter 2:9 says that the church is a “holy nation”, just as Israel was supposed to be. Being the covenant people means being the people who possess the word of God and the promises of God.
In the Old Testament, if anything came into contact with unclean things, such as dead bodies, unclean animals like pigs, or lepers, they were unable to enter the temple until they were cleansed. This cleansing was accomplished by sprinkling water. There is a whole ritual associated with this, given to us in Numbers 19.
In Ezekiel 36:23-28, God talks about the restoration of Israel, which would come in the time of the Messiah. In that passage, God promises to “sprinkle clean water” on the people, to take away their idols and filthiness, and to give them a new heart. This promise is the promise given to us in baptism. When John the Baptist came, he taught the “baptism of repentance”, and told people that they needed to get baptized in order to prepare the way for the Lord. God (Jesus) was coming into their presence, and they were filthy, so they needed to be cleansed. John made it clear that their baptism had to be accompanied by repentance, because their problem was their internal sin, not outward ceremonies. If they turned away from their sins, then baptism promised they would be forgiven. We need to be careful to distinguish between the sign and the thing signified. Baptism is not the forgiveness of sins; baptism promises forgiveness of sins, if the promise is believed by faith.
Col. 2:11-12 makes the case that baptism points to a more comprehensive reality than circumcision, and therefore takes its place. Circumcision shows us our sinfulness and the need to cut off the sinfulness of our nature. Baptism shows us the reality of forgiveness, and our death and resurrection in Christ. ur baptism therefore shows us the “circumcision of Christ”; that Christ was cut off (killed) for sins, and we died in Him, and rose again in Him, and therefore we never have to worry about cutting off sin again.
Baptism is therefore something God says to us, not primarily something we say to God. It is the sign of a promise. That promise is given to the whole covenant people, which include children, just as the promise of circumcision was to the whole covenant people, including children.
Baptism is a promise of the covenant of grace, that God has freely offered forgiveness of sins in the death of Jesus. Receiving the sign means receiving the promise of the covenant, and marks me as a member of the covenant. Receiving the sign does not mean that I am saved. I must believe the promise, not just receive the sign, in order to be saved. The covenant people are the church, which is a holy nation, and includes our children, and therefore our children receive the sign of that covenant just as adults do.