How long is long? I drain my battery once to twice a day (until I get the alert that I'm running low) and then recharge. A complete recharge usually takes 1 1/2 hours or so. If you're in the 3-4 hour range, I'd use Battery Health (available free at the App Store) and check your battery. Your 'battery health' should be at 99% or so (mine runs between 100.6-101 - I got a good one.
To check your battery, click the apple in the upper left corner of the menu bar, About This Mac, More Info, System Report, Hardware, Power and see what it says about battery Condition, Cycle count, Capacity, Voltage.
Long charging times could be a high use battery, or a charger that is starting to have problems. If you can borrow a similar charger you can check to see if it is your charger that is at fault.
You can also visit an Apple genius bar and have the techs run some diagnostics on your system, including the cahrger, to see if any faults show up.
Otherwise, independent sources like Battery University have shown that repeaded deep discharging of Lithium-Ion batteries significantly shortens the life of the battery. Optimum use/charge cycle seems to be around 50% discharge and then charge.