You may want to look how the files are on the SD card. THe camera connection kit will only read files in a folder named DCIM, and only if they have names of exactly 8 characters (DCS_2345 for example)
that's one reason your phone likely doesn't work. My phone at least gives my photos just numbers for names.
I have a canon camera and it reads my cards just fine, I've put in (thanks to a card reader that works) a 32 gig CF card from a nikon camera and gotten the card to read.
Now there are some camera connection kits out there that are bad, so if your camera puts the files on the card in a DCIM folder with a name of 8 characters and it still doesn't work, take it back and try another one because you may have a defective one.
If your camera doesn't follow that naming convention then you may need to look into a new camera.
Another way. You can use a USB flash drive & the camera connection kit.
Plug the USB flash drive into your computer & create a new folder titled DCIM. Then put your movie/photo files into the folder. The files must have a filename with exactly 8 characters long (no spaces) plus the file extension (i.e., my-movie.mov; DSCN0164.jpg).
Now plug the flash drive into the iPad using the camera connection kit. Open the Photos app, the movie/photo files should appear & you can import. (You can not export using the camera connection kit.)
Secrets of the iPad Camera Connection Kit
USB flash drives have about a 50/50 success rate. I tried 14 different ones (what can I say, i've collected them) and I got 7 of them to work. Everything from a 256 meg to a 4 gig,, but I had other 1 and 2 and 4 gigs that didn't work. So if you try to use USB flash drives you may need to experiment to find one that works.
I've found that ones with bundled software like the sandisk and kingston wont' work (might if you format them and remove that software). Ones with bright LED lights won't work. But I've also found that there's no corrolation between manufacturer because I had 2 lexar 4 gig drives, one worked, the other didn't.
So if you try it be prepared to experiment