Java web stuff "Bad", as in it has been a vector for malware atacks via the web.
Java based applications using the Java VM are not the same thing. There are Java apps such as OpenOffice, CrashPlan, etc... which are not a risk.
However, if you do not have any Java based apps, then you can delete the Java components. Nothing in native Mac OS X depends on Java, only 3rd party products.
According to some Google searches, Apple stopped shipping Java with 10.7 (Lion).
Also current versions of Safari disable Java plug-ins.
If you have Java on your Mountain Lion system (10.8), then either you have been carrying it forward from a previous version, or you installed it (maybe indirectly when installing a 3rd party app).
I hate being pedantic, but saying that Java's not is installed is simply wrong. The only Java component missing in Lion and beyond is the Java Runtime Environment (JRE). Peruse those installations, looking in these directories, and be enlightened:
Java Web Start.app
to name a few, but probably not all the Java components installed with those OSs.