Whilst watching some Anime recently, I got a bit annoyed that the default language was always set to English, so I had to change the language, and fixed the subtitles every 20 minutes or so which got super annoying.
I eventually found a fix.
Right Click -> Options -> Playback -> Default track preference
Set the number to the "Subtitles" number to the order of the option you prefer at the bottom of the Right Click -> Subtitle Track list, and the "Audio" option to "jpn"