Expression pedal keeps defaulting to all the way down

I've set my expression pedal to control different attributes in different patches. Mostly volume. And it seems like whenever I switch to a new patch, it very often defaults to sounding as if the pedal was all the way down, even though it is all the way up. For example, if it's mapped to the volume of a piano sound, so that the piano is all the way off when the pedal is up and the piano is at normal volume when the pedal is down, and then I switch to that patch while the pedal is all the way up, a lot of the time I can hear piano notes when I play, when according to how I set it up, the volume should be all the way down.

Any ideas how to fix this?

Thanks!