Apple remote control slow response on OS X Mavericks

Using my apple remote to control the volume while watching Netflix on my 2009 iMac takes minutes to respond. The battery is brand new. The OS is a fully updated El Capitan. What can I try to resolve this? This is a difficult problem to search the Internet for because the key words all point to problems not related to this. Thanks