Can't create new photo library

You don't get this window while holding down the Option key?

Be sure to hold down the key before double clicking on Photos and for a few seconds afterward.

Does the Option key work when selecting menus, like the  menu to get "System Information"?

Boot into Safe Mode (Use Safe Mode to isoloate issues with your Mac and Playing Safe: what does Safe Mode do?) and try launching Photos with the Option key held down from there. Any improvement? Reboot normally and try again.

NOTE: Safe Mode boot can take up to 10 minutes as it's doing some system cache cleaning, volume verifying and directory repairing. 

You can just double click on the library you want to open to change libraries. Photos has to be closed of course when you do that.

Log into another user account, Set up users, guests, and groups on Mac,  and see if the problem persists. (This tells us if the problem is limited to your user account or is system wide).

If it doesn't go back to your original account and download and run Etrecheck. Etrecheck is a diagnostic tool that was developed by one of the most respected users here in the ASC and recommended by Apple Support  to provide a snapshot of the system and help identify the more obvious culprits that can adversely affect a Mac's performance.

Copy the report

and use the Add Text button to include the report in your reply. How to use the Add Text Feature When Posting Large Amounts of Text, i.e. an Etrecheck Report

IMPORTANT:

Before running Etrecheck assign Full Disk Access to Etrecheck so that it can get additional information from the Console and log files for the report:

ALSO IMPORTANT:

Then we can examine the report and see if we can determine the root of the problem.

You do not need third-party security software on your Mac like Avast. They can interfere with the built-in security features and generally can be causing more problems than they will solve.