Soften the Rectangle of an Overlay

A picture is worth a thousand words: Background is blurred, overlay is feathered draw mask

Thanks for the idea. So you're saying that if the background clip (already blurred), the boundary between the background and overlay will have fuzzy background in one side and clear, focused image on the other (inside). I'll try that tomorrow. I've already had good success with a feathered shape mask. What I really want to do is have a blend between the two sides of the boundary.

I was thinking that it might be one of the compositing blend modes, but there's nothing in the help file to tell me what these modes are (the differences between them).

Thanks again for your suggestion.

Thanks, BenB. Yes, the fethered shape mask does what I want. I tried it immediately after my original post, but failed to cancel the question before you replied to it.

I'll try to describe it better. It's similar to what happens when you put a cell phone's vertical video on a movie's horizontal frame. Without filling the empty space on either side, they end up black. Here I want to put a picture in the video frame underneath the overlay's frame, and blur the underneath completely. The boundary is where the overlay is visible and where the underneath frame is visible (ie., out from under the overlay). That shared boundary is where the visibility of one starts and the other stops. And it's that hardness of the boundary that I want to soften or blur.

Would that gaussian blur affect just one layer or both (background and overlay). I'll try to get to an example this evening. (The actual video I'm working with is personal.)