So, can I correctly assume that your photos are in iCloud Photos and you have optimized storage turned on? When optimized storage is turned on then photos that you haven't opened for awhile will be stored in the cloud, while photos that you have accessed recently will remain in the physical Photos library. That might explain why some of your photos are displayed and some are not.
If that's the case I'm not sure what would be the best way to download the photos in your situation. You could try gathering into an album the photos that are not displaying, and then select the photos as a batch and double click on one to open all of them. That should download them to your Photos library. You can just do the specific photos that are not downloading, or maybe just go to each album, or your Photos library, and open all the non-displaying photos to catch the one's not displaying. The above is not something that I have tried, so you might experiment with just one album to see if it works.
You could also turn off optimized storage and download all of your iCloud photos to your photos library, but that might consume more space than you have available, and also likely would take considerable time (sometimes days) if there are a lot of photos. Not sure whether your albums organization would carry over. Since I don't use iCloud Photos I don't have experience with this.
In your case it might be easiest to export to a desktop folder the photos in an album that you want to access in iMovie, and then drag the photos from your desktop folder into iMovie.
To rule out that the issue is not simply the result of a corrupt preference, before doing any of the above try deleting preferences. To delete preferences open iMovie while holding down the Option and Command keys and select to delete preferences in the box that appears. iMovie will open in a new library. Reopen your old library to get back to your projects. Deleting preferences is a safe procedure that will not cause data loss or disruption to your project. Now see if your albums will display in the iMovie Photos item in the media sidebar.
-- Rich