I understand what you're saying... but I myself take refuge in the Buddha, the Dhamma and the Sangha. As such I am a Buddhist by most definitions I have seen. I'm not ashamed of the title 'Buddhist'. If I encounter someone else who also labels themselves Buddhist and who engages me in conversation about Buddhism, I say I'm an Upasaka. If they're 'fashion Buddhists' it usually becomes clear.
Another thought: if I met someone from a country where people generally hates folk from the UK (or the West in general) and they asked where I was from I wouldn't lie. By definition I'm British, I was born in Britian and I live in Britian. My parents are British etc. I label myself 'British' because conventionally I am! I'm not attached to being British, but I wouldn't lie about being British just because of the perceived behaviour of other British people.
By definition, I'm British. By definition, I'm Buddhist. I'm also a Scientist, a Man, a Husband, a Homeowner, a Scorpio (the list goes on and on).
I don't wish to antagonise here, my apologies if any insult is caused; that's not my intention... but maybe if more Dhamma-students / Triplegem refugees / vipassana Practitioners / whatever labelled themselves Buddhist, that 95% of 'fashion Buddhist' statistic would shrink?
On a personal note, the Dhamma saved my marriage. Whilst I know 'pride' is rooted in ego, I'm quite early on along the Buddhist path. At this time I can say honestly that I feel PROUD to have discovered Buddhism and PROUD to call myself Buddhist.
Maybe that means you who avoid the 'Buddhist' label have dispelled that pride and ARE further along the Buddhist path than I. In which case, well done
. This is 'where I'm at'.