Hi there,
Yes, there are a lot of reasons to swap out the Glock factory plastic sights.
My current Glock 23 wears a pair of Meprolight Night Sights and I find the sight acquisition is a lot quicker with the 3 dot night sights as opposed to the standard white outline U and white dot configuration of the current sights.
While swapping out the the sights is relatively easy, Glockmeister.com of Phoenix, AZ offers a lot of options for sights including a sets of factory sights for different sight heights, night sights and all of the necessary installation tools.
I have spent a good deal of time around Glock Shooters and have not heard of any reports of the sights simply falling off from shooting. I have had at least one occasion when the front sight on my Springfield Armory 1911 shot loose on a factory gun.
Shoot, enjoy, customize, and repair...part of the pride of ownership...
Chris