You can mix apple routers into the system.. just as WAP that is easy. (wireless access point).
You could setup one Airport as part of your main full access wifi. And setup another airport as guest. Assuming the new router has a fairly easy to configure guest configuration. Ie the setup is not on the airport but on the router.. which has one ethernet port dedicated to guest wireless WAP. Then your airport can easily work without a lot of messing around.
Trying to combine both guest and main wifi in one airport is done with vlan. But Apple provides NO ACCESS to vary settings for this. It is totally hidden from end user and is only because people love to hack things we know any details at all. Without vlan selection it is very difficult.
If your main router or managed switch is correctly setup with vlan (1001 from memory but I would need to check), it might work OK.. !!
Each pro router will have its own method of setting up guest wireless. (There is no standard set set by IEEE etc) And will be extremely unlikely to use identical vlan to the Apple router.. that means you are on your own configuring this. And while most pro routers have great interface.. and you can do amazing number of variations.. do you have the time to waste learning a new router OS.. or would it be easier to simply buy a package with all the bits integrated together.
Think cost in time and work out a monetary value to that. If your time costs 0 spending infinite time configuring it is fine.. if your time is worth more than 0.. it is important to understand what you are saving .. vs what % of your life is spent on unproductive.. unbillable hours.
Is it going to end up better than a properly managed pro system? The answer is clearly NO. And your saving in hardware cost are totally outweighed by your needing to configure the main router yourself from scratch without help.
(No pro hardware manufacturer will include setup using Apple routers as WAPs).
Then you have the problem if it fails.. Apple blames the router manufacturer and the router manufacturer blames Apple. It is impossible to get resolution when you give these companies an out. Mixing hardware is so easy for them to refuse support.
BTW go into any Apple store and ask what hardware they use for networking and wifi in the store.. hint.. it aint Apple.
It is most famous pro system.