Apple advises that the issue of app icons not appearing in notifications on Apple Watch is addressed under iOS 11.1, released today.
watchOS 4.1 has also been released.
Follow the instructions here to update your iPhone and Apple Watch, starting with your iPhone and beginning by making a backup of your iPhone: