How can I set up subfolders for my photo albums?

There are no 'Smart Folders' in Photos. There are Smart Albums, and they are simple to set up - File -> New Smart Album - and select among the criteria offered for the Albums.

If you want to find images not already in Albums then

File -> New Smart Album

Album -> Is not -> any

Terms can be a problem. In Photos, albums show pictures, and folders show albums and folders. Folders cannot have pictures that are not in albums.

Yer-Man tells you about Smart Albums which can show pictures based on one or more criteria. You might have a Smart Album like this:

or more complicated like this:

Not that this searches for pictures that meet all the criteria.

Photos actually scans pictures for certain "categories," and this can be searched in a Smart Album using "text" like this:

where I searched for pictures of trains.

If you like, you can put several Smart Albums, like searches for certain years, into a folder representing decades, or however you want to organize your pictures.

I use folders like this, seen in my sidebar:

with folders representing Decades, years, sometimes months, and albums. But other parts are organized by families or activities, etc.

Keep in mind that a picture taken on a train in Colorado can be in an album of Trains and also in an album of Colorado, and also in an album for 2012, without taking up any extra storage.