I found it, it was the messages mostly. Please ignore the post.
Also, the confusion came from the fact that there are 2 different places where you can see a breakdown of backup:
- If you have an old backup and you haven't deleted it: When you to your Apple ID settings and click on backups and then select the device (iPhone in my case), you'll see a list of installed apps and also the size of your next backup. Now this list might not add up to the number in "next backup size". This is because it doesn't show the messages and other things.
- Now to see the actual breakdown: Go to Apple ID settings and then iCloud and then click on "Manage Account Storage", here you can go in the first few options like Messages and see the actual size.
I found out that my messages backup is about 7 GB which is making my next backup around 9 GB. Apologies for creating a new post about this, I just checked there are existing posts but couldn't find this information, hope this helps someone. Thanks!