The Greenhill Church was founded by ex-slaves and local Indians just after the end of the Civil War in 1865. It was first established as the First Colored Baptist Church. Later, it came to be known as The Greenhill Missionary Baptist Church from the chant of ex-slaves that chanted: "goin' to th' green hill" when they were on their way to worship the Lord in their brush arbor. The brush arbor was used so that it would shield its worshippers from the elements of the weather, and so that they could not be seen by the whites who forbade them to assemble together.

The church gradually became known as The Greenhill Church, a non-denominational church under the pastorate of Pastor Raliegh A. Jones.