But why in the form of a dove? The dove is a gentle and pure creature. Since then the Spirit too, is “a Spirit of gentleness,” he appears in the form of a dove, reminding us of Noah, to whom, when once a common disaster had overtaken the whole world and humanity was in danger of perishing, the dove appeared as a sign of deliverance from the tempest, and bearing an olive branch, published the good tidings of a serene presence over the whole world. All these things were given as a type of things to come. … In this case the dove also appeared, not bearing an olive branch, but pointing to our Deliverer from all evils, bringing hope filled with grace. For this dove does not simply lead one family out of an ark, but the whole world toward heaven at her appearing. And instead of a branch of peace from an olive tree, she conveys the possibility of adoption for all the world’s offspring in common.
John Chrysostom, The Gospel of St. Matthew, Homily 12.3quoted in Thomas C. Oden & Christopher A. Hall (ed), Mark, Ancient Christian Commentary on Scripture (Intervarsity Press, 2005) 13.