Using Mac Portrait Mode for background replacement in OBS Studio
I wrote this as a user tip but it appears to be hidden away so I am reposting it here on the FCP forum.
Recently I experimented with FaceTime Portrait Mode to produce a video of myself in QT Player and PhotoBooth with the background replaced with an image of my choice.
The video was made by me standing back from my computer and using the webcam to produce myself and the superimposed background.
This clip was then put into FCP with a green screened image above it.
The background and green screen image were created by Gemini like this . . .
Here is the FCP timeline . . .
The result was good (not perfect) however those 2 apps have rather limited customisation regarding export codecs etc. so I decided to use OBS Studio which is far superior.
However, the OBS Studio background removal engine was appalling because on the wall behind me there is a high contrast archery target. This had not upset the Mac Portrait Mode but OBS was having nothing of it.
So I consulted Gemini AI which told me that the Mac Portrait Mode was superior because it utilises the Mac's Neural Engine Pipeline whereas the OBS filter simply used the CPU.
Furthermore, it told me that OBS can be configured to use the neural engine Portrait Mode directly which was great news and I did it.
Now every time I open OBS I can directly use the Mac Portrait Mode.
Here are the Gemini instructions:-
How to Use Apple's Superior Removal Directly in OBS
Because Apple already solved this at the operating system level, you don't actually need to use the buggy OBS background plugin at all. You can pipe Apple's native hardware-accelerated processing directly into OBS.
- Activate MacOS Portrait / Background Removal:
- ⚬ Open OBS with your webcam active.
- ⚬ Click the Control Center icon in your Mac's top menu bar (the two toggle switches near the clock) or click the Video Effects camera icon in the menu bar.
- ⚬ Turn on Portrait Mode (or Background / Studio Light depending on your macOS version). This applies Apple's M4 hardware neural filter directly to your raw camera feed at the system driver level.
- Send a Clean Feed to OBS:
- ⚬ In OBS, open your Webcam's properties.
- ⚬ Because macOS handles the segmentation, your camera feed inside OBS will already have a cleanly isolated subject or blurred background without requiring any OBS filters.
- ⚬ If you want to drop your bank vault behind you, select Background Replacement from the macOS Video Effects menu in the menu bar, choose your Bank Vault image, and let your M4 Mini do the heavy lifting natively without touching an OBS plugin!
This is the experimental video snippet I made originally using QT Player . . . no green screen, just the Portrait Mode background replacement filter. The "keying" is not perfect but as mentioned there were some high contrast objects in the room behind me.
EDIT: Not sure whether it's my internet but this YT clip of me appears slightly blurred . . . the original is crystal clear.
Mac mini