The Troy Catholic Male Orphan Asylum was founded in 1854 in Troy, NY. Its first location was in the Christian Brothers Academy, which was then called St. Joseph's Academy and located on Fourth St.. Later the Asylum was moved to its own site on Fifth Avenue. The Institution was called St. Mary's Orphan Asylum. On January 25, 1864, a Board of Trustees was organized and incorporated under the name of The Troy Catholic Male Orphan Asylum.
On May 5, 1866, the old building was destroyed by fire but another building was erected.
In 1927, the name was changed to Hillside School. Finally in 1953, Hillside School was closed. The name was changed to Hillside Hall when the building became the freshmen year scholasticate for the Brothers.