I had trouble finding the answer to this question online and when I did the answers weren't right, so I decided to post this step-by-step list on how to change when and if you get the pop-up notifying you when friends come online. This refers only to Xbox 360.
1. Press XBox Guide button (the big button on the top center of the controller)
2. Navigate to the "Settings" tab (last one on the right)
3. Select "Preferences"
4. Select "Notifications"
5. Here's where you have the option of turning off all pop-up notifications by un-checking the box, or turning them off only when you're watching a video, and/or turning off the sound that plays when a notification pops up.