Hi MaitakeMe,
You can turn off the sound effect that occurs at the beginning and end of Dictation by enabling Silent Mode on your device. Please note that this will silence many other sound effects, including ringtones (Clock alarms and media apps will continue to play sounds). The following support guide can you enable Silent Mode: Silence iPhone - Apple Support
Alternatively, you can decrease the volume of the Dictation sound effect. To do so, start Dictation, then hold the Volume Down button on your iPhone until the volume slider is fully decreased, then turn off Dictation. This will not completely silence the effect, but it will make it quieter.
I hope this helps!