Welcome to the forums!
The way that the Scaling Action has been implemented in Keynote is pretty crude -- as best I can determine, the function seems to take the actual pixels that the starting point image takes up on the slide, and then scales those, rather than using the original image's detail. It seems like a hugely inelegant approach for what is otherwise a pretty polished app graphically-speaking, and I can't imagine that it won't get fixed in future versions.
That said, you can produce a very nice effect if, immediately after you scale the image, you put on top of it a copy of the image manually scaled to the final size, using a very fast Dissolve Build. The Scale Action looks fine while it is actually scaling, it is only the final product that is blurry -- this technique solves this problem by plopping a non-blurry version on top of the final Scale state.
It can be a bit tricky to get this just right, especially since the Scale Action use percentage, but "manually" scaling an image only provides the actual pixel dimensions. I find it easiest to use simple multiples for the scale, as that allows easy determination of the manual image resizing (for example, if you use 200%, you can simply abut two copies of the small image together to get the final width, and thus can proportionally drag a third copy to size it to that width and height).
Also, you may find it helpful to use a small "target" shape to position the move, as that will allow alignment of other objects to the final position.