From your post, "the flagged section of albums", it's hard to tell exactly what you are seeing/doing. You could have
(1 ) An album named Flagged
(2) A smart album named Flagged that collects photos with certain criteria, perhaps a keyword as LarryHN mentions.
It would be helpful to see a screenshot of what you are talking about.
Assuming (1), here goes:
If you have an album called Flagged, then you can safely delete the album or pictures within it. Albums contain references to pictures, not the pictures themselves. And deleting an album or the pictures in a album do not delete them from your library. In fact, for albums, the delete key maps to the menu item Image-->Remove 1 Photo from Album (with 1 replaced by a different number if you select more than one photo).
However beware a "gotcha". If you use COMMAND-DELETE rather than DELETE, you will in fact delete the picture from the Album AND from the Library. It will hang around for awhile in the Recently Deleted "view".
Beware yet another "gotcha". If the Year-Collection-Moment view, the DELETE key itself will delete the picture from the library. Apple had this behavior in Aperture and it can be dangerous. It's very easy to forget if you are in Year-Collection-Moment mode or Album mode.
One more interesting behavior. If you EDIT a picture in an Album, that picture will change everywhere, even in the Year-Collection-Moment view. I think that is a very useful workflow, but it's not consistent with the Delete behavior.
P