The problem is that what Foursquare is telling to you is correct, the app is acting normal.
I was investigating the last couple of days and found that apps like Foursquare were intended to act like this on iOS 7, even if the user force closing the app if the app use location awareness then location services for that app will remain on.
For instance, even without Foursquare running in background if you are near a place with a promotion Foursquare via notifications will alert you, the very big problem is that iOS 7 had a bug that preventing apps doing this if the user force to close them.
Now seems that iOS 7.1 fixed that bug and apps like Foursquare are working as intended, leaveng location services always on even if the app is not running in the background. So yes, this behavior is a feature and not a bug.
Personally I don´t like this behavior, I´m not clear the impact on battery life (as Apple comments there is non) but besides this I really don´t like apps tracking me without my permision or without I letting run on background.
So in the case of Foursquare the easy solution as other post mention is to go to the in app settings and then account/privacy (allow to run background location and behaviorally targeted ads).