If someone know my serial number?

Just to clarify, if someone knows your serial number, they can infer certain data about your machine - for example, they may be able to determine the specific model of iPad you're using, or even when it left the factory

It is possible they could try to use this information to trick you into thinking they have access to your system (e.g. "I know you're on a 4th generation iPad Air, and that you've had it about 6 months"), but that does not mean they have access to anything on the device at all.

Your serial number does nothing as far as allowing anyone else access to your device. About the only thing it is useful for is verifying support/warranty availability.