How can I insert mathematical operation signs - like divide -or insert equations and quadratic symbols?

Built into the Character Viewer you have some limited options.

control command spacebar opens the character viewer.

How to use emoji, accents, and symbols on your Mac - Apple ...

there may well be third party app aimed at mathematics/equations—

App Store

One way to access math symbols is to:

1: Open System Preferences

2: Select Keyboard

3: On the Keyboard tab, check "Show keyboard and emoji viewers in menu bar"

This will add a new menu extra to the top right of your menu bar. Do the following:

1: Click on the menu extra and choose "Show Emoji and Symbols"

2: This will open Character Viewer.

3: In the sidebar find Math Symbols

4: Open an application that can receive text input and have it in the foreground with an insert point active

5: Mouse over Character Viewer and double click the math symbol you want. This should insert it into your active document.

Please note, not all symbols are available in all fonts. Depending on your app, if you change the font you might drop your symbol.

Ah, but this will not get you quadratic notations. For that, I suggest the following:

1: Open Grapher (found in your Utilities folder)

2: Choose Default and stay on the 2D graph and hit OK

3: Type your equation in the equation bar. You will be able to use ( and ) as well as 5/6 to get fractional conversion of characters. For example:

You can then copy the equation and paste it into a supporting application.

Hope this is helpful.

Reid