Many have suggested the OP just explain his decision not to eat to his family. But I haven't heard the explanation myself. To the OP, why are you keeping the 6th precept? That precept for monks was made so as not to inconvenience the laity. That is, instead of going for all alms three times a day, it's easier for the laity if the monks just go once. So to me, I just don't see the rational for it as a lay person, especially when you are upsetting people by it.
Also a personal anecdote: I had been living at a Western Forest Monastery for three months keeping the 8 precepts and went home to visit family, with the intention to come back and ordain. I asked the abbott if he had any advice for me going home. He said keep the 5 precepts, not the 8! He said refusing food just creates unnecessary problems and in my opinion he's right.