Can I just walk into an Apple store for battery replacement?

You should really make an appointment at the Genius Bar.

It's possible that there's an unreserved slot or a no-show, but I wouldn't count on it. I've walked into an Apple Store with something urgent and no appointment. Once I got serviced after waiting for an hour. Another time I was shown available appointment times (most were well into the next week) or was told to come back when they opened the next day to be placed on a walk-in waiting list.

If it's really urgent then there might be Apple Authorized Service Providers near you that have earlier appointment times. Have you checked for appointments? I'm looking up available appointments in my area for an iPhone battery replacement and see several Apple Stores and several AASPs with appointments today. I wouldn't let anyone at Best Buy touch it though.

But even if you can get them to accept your device for a battery replacement, it doesn't mean the service happens that day. My dad had a battery replacement appointment near the end of the day and it wasn't ready until the next day.