That's because you haven't installed Boot Camp drivers.
By default, Windows can't read and write in HFS+, the filesystem that OS X uses for their volumes. The Boot Camp drivers include a HFS driver that allows you to read the OS X volume from Windows, so you should install them. Apart from that, it will install drivers for all the hardware you have, making Windows more stable and everything will work better.
The process to install the Boot Camp drivers depends on the OS X version you are using. If you are using Mac OS X Snow Leopard (Mac OS X 10.6), insert the Mac OS X DVD and install the drivers from there (you have to insert the Mac OS X DVD when you are in Windows).
If you are using OS X Lion or Mountain Lion, you have to download the drivers. First, start in OS X holding the X key while your computer is starting. Then, open Boot Camp Assistant, tick the option to download the Windows support software and follow the steps (you need a USB drive to burn the Boot Camp drivers in). Finally, start in Windows and install the drivers, so you will be able to read the OS X volume from your Windows Explorer