Having just sent maxemulien out of the house with both kids lest I start screaming and/or crying, I am totally with you on the importance of "me time."

How much is just enough is something of a balancing act that will depend on the personalities of the people involved and of course change over time as the child grows. The key, I think, for judging if you have the right balance is looking at your relationship with your child. If you are despondent, overwhelmed, or short tempered then your interactions with your child are going to reflect that and some "me time" is in order for both your sakes. If your child is clingy and insecure or you have no idea how to relate to her because you don't know what's going on in her life, then you probably need a little more "us time." The goal is to keep everybody happy and connected, so as long as your relationship is thriving and your time together is largely rewarding for both of you, then you're doing great.

I also don't think you should worry too much about being reluctant to switch gears (ie wanting to wait just a little longer to pick her up) All that shows is that you enjoy your work, which is a good thing. It is always a little hard to stop doing something we enjoy even if the next thing is also enjoyable. Transitions are hard.