I read this helpful tip on the Forza forums.
Party Chat - Surprise, Kinect is Listening
Question: How did a friend in my Party hear my private phone conversation?
Issue: If the Xbox One controller powers off while you are in a Party chat, the Kinect microphone will activate and broadcast your voice to the Party. This will happen even if you have Muted the controller (to take a call, etc) before the controller powered off.
- If you don't want the Party to hear you, go to the Party app on the dashboard Home screen and select Mute mic, or Leave party.