I want to begin clarifying I'm not recriminating anyone with my words.
In some anime I just watched, there is a group of anime fans. In the show, you could see that this group (otakus) are recriminate by the society because the common sense is: they live watching their "cartoons ", that's all. Also, because hentai (I am not against the ones use to watch it) is the most common anime genre in Japan, the people think that anime is totally pervert (usual way of take it). Our society no use to talk so much about anime, they doesn't have plenty of information. The truth is that it's something "new" (because right now the fans out of Japan are increasing and people are hearing more often about) and don't get it as a hobby like read comics or watch movies. I don't negate that exist someone obsessed with anime and definitely lacks of social live, but it goes to other common hobbies too. The only aspect that applies here is tolerance and acceptation of others about your preferences, like another common stuffs in your daily life. Explain what it is to your friends or family, if they are with you I believe they will understand and maybe also find it fun :)