Yes, the library you are currently using seems to be highly optimized and only contain thumbnails of your photos. You have to download the photos from iCloud before you turn off iCloud Photos.
Try the following:
If you want to save all photos, but not the library organization,
- Open Photos on your Mac.
- Create a new folder on your external drive.
- Select all photos in Photos, then use the command "File > Export > Export unmodified Originals". Click Export IPTC as XPM. This will create sidecar files with the metadata (Titles, keywords, locations). Also enables "Subfolder format 'Moment Name'. Then the photos will behaved to folders named by the date and the location.
- Select the new folder on your external drive as the destination.
- If you want to save the edited versions as well, repeat the export with"File > Export > Export ... Photos" to save copies of the edited versions. Click the "v" to the right of "Photo Kind: JPEG" to select the original size and a high quality. Otherwise you will get only downsized versions.
Alternately, to save a complete photos library with all albums:
- Copy your Photos Library to the external drive. The optimized photos will be incomplete. So you will have to enable the copy of the library as your iCloud Photos Library and let it sync with iCloud, with the "Optimize Storage" option off.
- Double click the copy to open it in Photos. Enable "Photos > Preferences > General: Use as System Photo Library.
- Enable "Photos > Preferences > iCloud: iCloud Photos"; disable "Optimize Storage"
- Let it sync with iCloud and download all photos. It may take a week or so for a full sync. Once you have a full version of your library on your external drive, you can switch back to the version on the internal drive and use the library on your system drive again as your iCloud Photos Library. Or continue to work with Photos from the external drive.
This document explains how to prepare an external drive to be used with Photos for Mac. It needs to be locally mounted and formatted correctly:
See: Move your Photos library to save space on your Mac - Apple Support