It depends on which device. A non-phone relies of the router to have supplied its location to the internet Gods. If that device is not lost (you are just testing the location), then try your test in some public area. If it is lost, then it could be out of juice, turned off, or has had the service turned off - any number of ways for a thief to thwart Find My iPhone.
So, what are your circumstances?