Enable Flex Pitch WITHOUT conforming to tempo

I'm on Logic 10.4.4.

I'm working with an acappella vocal track that I have written music on top of. The vocal recording is free-time and shifts tempo. I created a tempo map for myself that follows the vocal to then start layering other instruments on top of.

I want to tune the vocal track using flex pitch, however, when I enable Flex Pitch on that track it also conforms that track to the shifting tempo map I made earlier. If I turn off "Flex & Follow" from the Region Parameter Box in the upper left corner of the screen, then I no longer have access to Flex Pitch on that track (i.e. when I open that track in the Track Editor, I no longer see note pitches that I can move around and alter).

I just want to be able to tune a vocal but not have it follow the project's tempo map.