I worried about passing in public when my hair was short and I had no breast to speak of. Now I only go out in public as a female. Of course with Covid restrictions that is usually just the grocery store or a sandwich shop. I agree with DeeAnn that most people are too absorbed with life issues to even focus on others.
Many people wear very unisex clothing these days. So when I come up behind someone in the store the first thing I usually observe is their hair; is it a female style. Then when I pass them do they have breasts. If a person fits those initial criteria then I assume they are female unless I need to talk to them for some reason and then I look closer. Also, if they look comfortable in their skin then they seldom get a second glance.
More often in public the only ones that I notice are the younger women wishing I was one of them.