There are three commands you have to run from PowerShell: (I only tried in admin mode)
Get-AppxPackage Microsoft.Windows.Photos -AllUsers | Reset-AppxPackage
Get-AppxPackage Microsoft.ZuneVideo -AllUsers | Reset-AppxPackage
Get-AppxPackage Microsoft.ZuneMusic -AllUsers | Reset-AppxPackage
First one (suggested earlier, thus the solution now :) ) only works for pictures.
The second command works for videos, and the third command works for audio files.
In case there are other issues... you have to check what default app Windows uses for that. Somehow find out the packagename for it, and replace it in the command (this is how I managed to make this work.)
No restart needed, after these commands, you can set the default apps for images, videos, and music files.
-Right click on file, Open With (other app), and select: always use this. OR:
-Right click on file, Properties, and modify there. OR:
-Settings menu, Default Apps, select the app. OR:
-Settings menu, Default Apps, enter the extension (like .mkv)
(remember, these only worked for me after entering the commands above)
Hopefully I could help all, who are in need :)