If you go to the Apple site, Watch menu item and look at the comparison option you can see the differences between the two.
The Series 6 has more case options but if you are looking at the basic case options - Aluminum in Silver, Space Grey or Gold, the main difference seems to be that the Series 6 offers the extra heart monitoring facilities - i.e. ecg and irregular heart rhythm notification. It also has the Always On Display option, and a newer processor. If those options are worth the extra cost to you, the Series 6 might be worth it. Otherwise, the SE might be best for you. But as I say, check out the differences in the comparison option.