I heard about being selfish for the first time from my Yoga teacher 30 years ago. The idea was simple: you have to take care of yourself enough to stay healthy and not to become a burden to your family when you get old. I know we cannot control many things, but everybody is capable of doing their best.
Loving the whole World is great! But nobody will come take care of your body or your mind for you. That is on your shoulders. Moreover, I am totally convinced that things like self-care, self-improvement, personal growth are our direct responsibility.