Full keyboard access and SwiftUI's ScrollView

Wa are trying to make our app better accessible with the hardware keyboard. According to Apple's videos we skip non-interactive elements and only focus on interactive elements when you navigate with the hardware keyboard.

We have a problem in SwiftUI with the ScrollView. When the scrollview contains text-only elements on screen and also buttons that are offscreen, we can't bring the focus with TAB to the scroll view. That means that if we use TAB-G to turn on Keyboard gestures, it can't perform a swipe up or down to scroll the scroll view as well.

If we scroll manually to the bottom of the scroll view where we have an interactive element like a button, then suddenly the scroll view becomes focusable and we can scroll and we can select the button.

You can test this with the following code in a sample app:

ScrollView {
	VStack {
		ForEach(0..<30) { number in
			Text("Text only")
				.font(.title)
		}

		Button {
	        ....
		} label: {
					Text("Interactive button")
		}
	}
}

How can we make sure a hardware keyboard user can reach off-screen elements that are interactive?