If you want to mask with a custom shape, there are a few ways to do it in Keynote.
If the object is roughly a standard shape, you can start by masking the image with an existing shape. Do this by creating the shape on the canvas, then drag the image onto the shape from the Finer or Media Browser. Then once the shape is masked, you can edit the path of that shape to fine-tune it. Double-click to edit the mask, then go to Format -> Shape -> Make Editable. Now you can edit the bezier path by adding and removing points (hold down option and click on the path to add points), and adjusting the bezier handles.
Another way to do this is to start with a free form shape on top of your image. Drop you image onto the canvas, then use the Pen tool (last item in the Shapes menu on the toolbar) to start drawing your shape, on top of the image. Once you're done with your shape, select it and the image, and choose Format -> Mask with Selected Shape. Once you've done this, you can still edit the shape using the steps above.
Note that masking can be combined with Instant Alpha -- you can Instant Alpha around a shape, then mask it closely with a shape instead of having to individually remove parts of the image far away from the object of interest.