Can I transfer data from my old iPhone to the new without using iCloud?

You should be able to:

Use the Finder or iTunes to transfer data from your previous iOS device to your new iPhone, iPad, or iPod touch - Apple Support

Though, it's been a long time since I've had any reason to see what happens when trying to sync an iPhone 4 as it's about 12 years old now.

The support article you arrived here from provides you with three potential methods to transfer data between these devices. Most likely, the only one that will work for you is the last one, which is to use iTunes/Finder to make the data transfer.

Ref: Use the Finder or iTunes to transfer data from your previous iOS device to your new iPhone, iPad, or iPod touch - Apple Support

If you have a Mac, check and see whether the old iPhone can be set up to backup to the Mac. If it can, you can use the backup to restore back to the new phone during the setup of the new phone.

Back up iPhone - Apple Support

As I recall, the backup of an old iPhone 5 worked fine, but it's been some time since I did this. Don't know about an iPhone 4, but it will only take a few minutes to find out if it can backup to the Mac.

Sing out of apple ID

Victoriaw07 wrote:

Sing out of apple ID

How will that help transfer data?

iPhone unavailable discovered