Help Me Understand How macOS Handles Resolution Vs. Windows

Macs with higher resolution screens as standard are deemed to have 'retina' displays. Macs with external displays even of an identical resolution do not necessarily behave in the same way although it is often possible to get them to do so.

Whilst Apple's own term for their own high resolution displays e.g. as built-in to the 5K iMac is a 'retina' display, the process of scaling the resolutions so that object and more importantly text is displayed larger i.e. large enough to read but with smooth edges is called 'HiDPI'.

First Apple's general document on this subject - Using a Retina display - Apple Support

As usual Apple do not bother detailing to ordinary customers how this works although admittedly for average users this will be beyond their understanding or need to know. However the following article is from a technical website and does give a more detailed explanation.

Basically the important difference is that scaling with a Retina display or similar high-res display enabled for HiDPI mode - to make it act like a Retina display does not do what older displays used to do which is simply enlarge an image and use more pixels so hypothetically scaling a Retina display from 4096x2304 to 2048x1152 does not mean the display is then halving the resolution an in effect using twice as many pixels for each scaled pixel. Instead Apple actually use a larger virtual resolution, so they take the desired 'scaled' resolution of 2048x1152 and use that to determine the 'size' of objects but render text and dialogs it at the higher real resolution and also if available use higher resolution icons. If an application is HiDPI capable then it can take advantage of this, so for example Apple's Preview displays photos at the full screen resolution no matter what the scaling is set to. In theory all Apple's applications including Final Cut Pro X and iMovie should be able to take advantage of this.

See this article for how to enable HiDPI on non-Apple screens How to Enable HiDPI Mode in Mac OS X - TekRevue

You can also create a customised monitor definition plist which enables scaled resolutions for your non-Apple high resolution screen using this website Scaled Resolutions for your MacBooks external Monitor | by comSysto despite what the title says it can do this for any model Mac.

You can also 'get info' on an Application and specifically configure whether it is to use HiDPI or not but this may mean the text in that Applications dialogs appears very small whilst the video etc. appears the desired size.

Or does macOS always use the native resolution of the display and just scale everything?

That one. The Mac OS has never supported resolution independence. If you use a lower screen resolution in the System Preferences, the OS just makes that "size" fit the native resolution, even though the actual display buffer size hasn't changed. That's why a 3840x2160 screen shot still gets you a 5120x2880 image.

And then there's the other side of that you've seen. Despite the faux lower screen size choice, some apps still recognize the screen's native resolution and displays 1:1, and not scaled up to match the lower display resolution.