how to rt click copy

This has been a bit of an annoyance for me. In finder, if I right-click a file, select "copy", then go to my destination folder, I should be able to right-click, and select "paste".

However, if that folder is in "list display mode", and it has lots of files (that go past the bottom of the finder window), I can't right-click - as it selects the file that my pointer is hovering over. I then have to switch to "icon display mode" and then I can right-click and paste.

Is there a way around this annoying behaviour?