I'm allergic to seafood, dust, and pollen. I take Zyrtac every morning to deal with the dust (I live in a very dusty apartment) and I stay inside on days when the pollen count is high. As for seafood, I just don't touch it. It's fine enough for me to be around, I've been out to eat with friends...
Then I don't really think that should be considered something you're terrible at it. Sounds like you're just vibing, tbh. What works for you is different for everyone else.
Most people are too busy to initiate conversation too. I've found that asking questions and giving small compliments is a good way to stick out in someone's mind.