Try using the above directions thevictors51 provided on your Xbox settings. This should fix your problem. Before I changed the console settings I was having to turn off the CC on my Netflix everytime also.
I am having this same problem with MLB.TV, I change the settings and restart the app but the Captioning is still on. I cannot get it to turn off. This seems to be the only app I have problems with this.