I found a post from a user with similar problems and it seems that Microsoft is aware of the issue and is working on a fix. This post is from Mister Gwyther:
We've heard of issues removing Pins once the content is deleted and no longer available. At this time, I don't have an answer on this, but we'd love to capture your feedback on this in our Feedback Forums (forums.xbox.com/.../default.aspx).
Please post it there and we'll make sure it's available for the right people to see. Thanks!
Did this solve your problem?
Sorry this didn't help.
Great! Thanks for marking this as the answer.
How satisfied are you with this reply?
Thanks for your feedback, it helps us improve the site.