Homepod mini cannot play apple music without iphone

I just bought my 2 homepod mini to make a stereo speaker for my apple tv. It worked well with my iphone, if I use my iphone to play apple music on homepod. However, if I leave home and back home at night, when I say hey siri play music, homepod mini response it cannot connect to apple music. After that, I have to use my iphone to control apple music on my homepod mini before I can use voice control over the homepod. It is quite annoying because I think I can stream apple music directly on homepod. Is there anyone having the same problem, or can anyone explain for me? Thank you