Office365 MFA app password not working with MacOS Apple Mail

I actually encountered this problem myself and found a way to solve it. However, the original thread has been closed without proper answer. Hence I want to document for people who has this same issue.

In order to use the App Password by Office 365 in MacOS Mail, do NOT pick "Sign In" to your Exchange account. Choose "Configure Manually". Then, use your App Password instead of the regular account password in the pop up. Then you should be able to sign in with no problem.