Hi davidblackburn,
welcome to the Apple (user-to-user) Community!
Re: "how increase magnification factor in picture-in-picture rectangle on new Mac mini Sonoma 14..2?"
You can zoom in and out on the screen using keyboard shortcuts, scrolling on your mouse or trackpad, or tapping on your trackpad. For how to turn these options on and off:
See Apple support's article :Zoom in and out on what’s onscreen on Mac - Apple Support
_________________________________________________
Seems there are a few different ways of controlling zoom;
Apple Support have another MacOS Sonoma article online that may be helpful:
Change Zoom advanced options for accessibility on Mac - Apple Support
Eg: says: "On your Mac, use the Advanced options of Zoom settings for accessibility to set appearance, control and focus options for the selected zoom style (Full Screen, Split Screen or Picture-in-Picture)".
To change these settings, choose Apple menu 
> System Settings, click Accessibility 
in the sidebar, click Zoom on the right, then click Advanced. (You may need to scroll down.)
The options you will find in settings are described in detail in the above linked article.
_____________________________
Other articles that may be helpful: in using more Zoom features.
With Hover Text on your Mac, you can move the pointer over text on the
screen to display a larger version of it, and display a larger version
of text as you type. (Once set up, pressing the command key displays larger text in it's own window) See how to: Use Hover Text on Mac - Apple Support
More info : Change Zoom settings for accessibility on Mac
How to: Make text and other items on the screen bigger on Mac
All the best :-)
