Tempo goes off beat gradually

I don't know if this is a bug with audio files, or with the entire project itself, but every time I make a song and add any kind of audio file to it, its always out of beat. These songs are in the same tempo as my project but the audio files will begin on beat, but slowly go off beat. Also, I never really touch the temp automation track so it's not that. If anyone could help me with this that would be great.