I think the reason may have been when I first turned on iCloud Photo Library and it uploaded my photos and removed them automatically from my device.
That just does not happen. There is no setting anywhere for automatic deletion from your device after uploading to iCloud.
When you turn the library off on your iPod, you will be advised the photos on your iPod will be removed. This does not affect your photos on iCloud.com itself. They stay until you remove them manually.
So after turning off the library on your iPod, all photos are still safely in iCloud and accessible on iCloud.com.
The only way photos on both your iPod and iCloud would be removed at the same time is if you removed them from either location, while the library was being synced.
That does not include when you turn off the library on the iPod. That does not trigger the syncing.
If the photos you are missing were from your Photo Stream, note they are not ever visible on iCloud.com, and are stored by Apple for 30 days or 1000 photos. It is temporary storage.