It may depend on what format your subtitles are in the MKV file. However I can give you an example and how to turn them off.
A lot of MKV files are produced from a DVD, perhaps using the MakeMKV tool although using MKV Merge results in the same 'problem'. DVDs use a bitmap format for subtitles - not a text format, for DVDs these subtitles are often referred to as vobsub format from the fact that DVDs use VOB files, and obviously we are talking about subtitles. These (bitmap remember) vobsub subtitles are stored in an MKV file as an additional video track as far as QuickTime is concerned and not a text subtitle track. Tracks of any type in QuickTime can be enabled or disabled, if the DVD used to make the MKV file has multiple language subtitles then one video track per language is created. Both MakeMKV and MKV Merge set the first of these vobsub tracks to be turned on, and any others to be turned off. As a result when you open it in QuickTime Player (thanks to Perian) and play it, QuickTime player sees the main movie video track is turned on (which we want), the main audio track is turned on (which we want), and the first video vobsub track is also turned on, which perhaps we don't want.
QuickTime Player X does not seem to have any way to let you manually enable or disable these tracks and because they are not normal QuickTime text based subtitle tracks, the normal subtitle controls don't apply. However QuickTime Player 7 if you have activated the QuickTime Pro key, does let you open a properties window via Command-J on the keyboard. In this window you will get a list of tracks, and you can enable or disable these extra video vobsub tracks or switch to a different language. If the subtitles are stored in a separate file to the MKV file, then you can in the Perian preferences, set Perian to not automatically load these 'external' subtitles.
My own experience is that VLC will also show the first video vobsub track as well. Just to make things more complicated, on DVDs there is a type of subtitle called a 'forced' subtitle. As far as I understand this is meant to always be shown as default, and a possible example of its use is a film that has actors speaking in two or more languages, e.g. a War movie, the main language spoken by the actors would typically be English and would not need subtitles, but when the enemy speak in a different language their speech would result in English subtitles being shown. Someone watching the DVD whose native language is the same as the enemy might in the DVD menu choose their own languages subtitles to be shown and then this would mean english speech by the actors would be subtitled but not when their own language spoken.
I have not tested what happens with forced subtitles on an MKV file but I know that MKV supports this capability.
PS. I have sent a request to the author of MakeMKV to allow setting a preferrence to still import subtitles but not to enable them.
