Well its used in 'Herbal' Medicine, so yes, I would call it a herb.
I would call anything a weed that is growing rampant and the owner doesn't want it. I think 'invasive' is the word many use for a 'non weed' though.
We purposely allow a certain amount of 'weeds' to grow for herbal use and for the wildlife such as daisies, nettles, great plaintain, but they get out of hand fast so then become 'weeds' that need to be 'pruned'.
Our neighbours have complained about our 'weeds' grown for medicinal use. Some people are not aware and then there are those that even when made aware don't care and would rather stick to what they know. Some people will always see weeds as a sign of an untidy neglected garden.