…When did the average pastor stop considering it his duty to regularly, at least annually, visit the homes of his parish?

Historically this happened no less than once a year. At Epiphany the priest(s) would make their way around the homes of the parish, drive-out darkness, bestow a blessing, hedge it with the Name of the Triune God, and meet with the family in the dwelling they called their own…