Theoretically, it should work, yes. Have you tried
resetting the PMU?
However, I strongly recommend against running with no battery for a couple of reasons.
First of all, even a mostly dead battery usually will hold enough of a charge to save your projects you're working on if the iBook is accidentally unplugged.
Next, if Apple had really intended using the iBook this way, they would have provided a battery hatch cover. As it is, using the iBook with no battery in place would subject the terminals to dust, lint, and debris. Needless to say, this is not good.
Apple has never really called its portables "laptops." They are not designed to run on your lap, but on a hard surface. I know most of us use them on our laps, but these days, I use one of
these, which my son bought me for Mother's Day. I use it folded in half, and use the foot to prop up the back so that my iBook is kept cooler than it would be otherwise. It works great.
If the heat bothers you, it probably bothers your iBook, too. You may want to look into a similar solution. One long-time Apple user has suggested an inexpensive acrylic cutting board as a lap desk, and that would work fine, too. I don't recommend running without the battery, though, even if you could manage to do it.