If atheists believe that they don't deserve to get religious labels attached to them because they never signed-up for that religion, then we non-liberals shouldn't be slapped with your labels since we never signed-up for them, either.
So when religious people say things like, "It's a sin in my religion to be gay/eat meat/masturbate/get tattoos, etc. You're being a sinner." etc., a common response from the people being attacked is, "I never signed-up for your religion. If it's a sin in islam to eat pork, then that's your choice but I'm not a muslim so that law doesn't apply to me. That sounds like a you problem."
So then if that's the logic then people who never signed-up for liberal beliefs should also be immune from your labels.
It's homophobic to dislike gays? It's islamophobic to dislike muslims? It's fatphobic to be hostile towards fat people? Well tough luck I never signed-up for liberal labels.
So it goes both ways.
If "I never signed-up for your belief system = Your labels don't apply to me", then...