Anchorites tended to embrace a much more constrained physical stability so that they were  required to remain within a single small house or even a single room. Sometimes they were even locked or walled into such a place. Otherwise, however, both groups dealt with others (sometimes the degree of interaction was relatively extensive); similarly both were often approved to some extent by diocesan canons and local Bishops. Hermits (always men) who traveled from place to place were often granted the hermit tunic and permits to beg and preach by the local ordinary, for instance. But anchorites (who could include both women and men) lived their solitude within a fixed abode; hermits (who were, as you say, more marginal) could wander from town to town or otherwise live their solitude in less physically constrained ways. …

Because anchorites tended to live in the midst of villages with a window on the Church altar and one on the village square, they were often unofficial counselors, spiritual directors, a friendly pastoral ear, teachers, wisdom figures, preachers (as, again, were itinerant hermits), etc. Contrary to what you have concluded, while some were certainly secluded like the modern day Nazarena (oftentimes reforms were attempted by priests who wrote Rules for them limiting and regulating their contact with others) the very fact that such reforms were seen as necessary confirms that anchorites were, generally speaking, not so secluded as all that. 

Stillsong Hermitage

I think that is a pretty good starting point. While in modern Anglican terms both are Solitaries, the distinction is present in how life is lived.

I especially like the part about being “unofficial”. And being part of a eucharistic community and a secular community.