Can't blame them really. Many people come off the meds when they're not ready and end up having a torrid few weeks where problems resurface and then have go back on them. When I decided to come off them I was in a very good place and had not many external triggers (Such as stress at work, etc).
I am fairly comfortable now but that's not to say I won't need meds in the future.