The only moment I happen to notice this is if I open the doors outside the bus stop. For example, I've had too many beers, so I have a small crash, and call the police and lose 8 minutes. Then I'm in a hurry so I brake bad or do something wrong, so passengers get angry and want to leave the bus. After all that happened I get angry and frustrated too and decide to open doors on the first red light I have stop.
In this case, passengers start to get off the bus, and indeed they dissapear.
But in the rest of bus stops or normal situations I haven't experienced this issue... Just curious. Have you installed a mod, map, or something? It's just weird that if you stop in the bus stop correctly, passengers do what you say.
And when you say "various bus stops", you mean random or are some specific ones that always expericence this issue? I'm just curious so I can check it out to see if it happens to me too...