Hey,
You can indeed play MKV files in front row! For anyone reading this who does not have VLC, you will first need to download and install VLC Media player for OS X, get it at
http://www.videolan.org/vlc/download-macosx.html
This will install the proper codecs - the software necessary to play the video format. Now download and install sapphire. Get it at
http://appletv.nanopi.net/getFile.php?Sapphire_1.0b6.5.sh
You will notice you get tons of lines of text. Just click file - Save as at the top of your browser, and save it to Desktop. Now you will need to open the finder, go to Applications, Utilities, and open up the terminal. This seems a little daunting if you have never used the terminal before, but don't worry it's not too bad.
Now type "cd desktop" without the quotes and hit enter. Now type "sh Sapphire_1.0b6.5.sh" (no quotes) and hit enter. You may be asked to type you admin password, do so and hit enter. Then when it prompts you to hit Y or N, hit Y and hit enter. Now Sapphire will install. Once nothing else is scrolling across the terminal screen, you're done!
Now open up front row, and you will notice there is a new entry called sapphire. Click it, and you will be given some options. There is a bunch of directories, including /Users/You/Movies If you have the MKV files in there, then click play to navigate to that directory, and click play on the video you want to play. If you have them somewhere else, note you can also navigate to the / or the root directory. From there you can navigate to anywhere on your computer, and play the MKV files from there. If you have a folder with all the files in it, navigate to it from the / directory, and once you are at the proper folder, hit the fast forward button, and you will see an option to add this directory as a collection. Now this directory will be at the front page! No more navigating after that! Note that sapphire can play any kind of video, not just MKV. If you have the codec for it, it will play it.
Feel free to reply with any questions or concerns you may have. If you get confused, I will be more than happy to help!
Good luck,
Dylan Demora