I guess it is time to repost this as it is clear that many are not reading all previous posts in this thread before pontificating on the subject....
=======
It is not a bug in the iPhone, it is a bug in the viewing software and there have been several threads about this 'problem' in the last couple of years. The explanation is rather simple.
The EXIF metadata spec contains a field which defines how the image should be rotated for proper viewing, i.e. 'which side is up' 😀
Along about version 3.13 or so of iOS the software was changed to utilize that field of the EXIF spec--likely to facilitate faster photo saves from the camera. Prior to that time iOS devices with cameras physically rotated image data so that the rotation value field of the EXIF metadata could be set to zero. This allowed the images to display properly on those apps that were (and many still are) brain dead, i.e. they ignore the EXIF rotation field assuming it is zero, but that practice greatly increased the image to image time because rotating the image can be CPU intensive and is better left to systems with fast CPUs.
So, the problem does not lie within the iPhone, it lies within the other apps that have so far failed to recognize and/or support the EXIF standard properly. Don't complain just to Apple about the iOS device, also complain to those companies with brain dead software--and there are quite a few still in the dark ages of image presentation. As a previous poster stated even Apple's own Safari doesn't handle it correctly (or at least it didn't earlier this year...I haven't checked lately).
I have suggested to Apple that they provide a switch to enable the old practice of rotating the image prior to saving but that has never happened. If set to ON one could be notified that image to image time would be increased and the user could then decide to use it or not. I suspect it wasn't added because the explanation of the function it provides could be confusing to non-technical users but I have nothing to support that suspicion. If you agree with this sentiment you can so inform Apple at http://www.apple.com/feedback/iphone.html.
It should also be noted that virtually all high end cameras set the rotation field, they do not rotate the image prior to saving. The problem is that most users of a high end camera pass their images through software on a PC/Mac before posting. All decent software on the PC/Mac platform, such as Adobe's software, respects the orientation flag so the image is properly displayed. Photoshop goes so far as to rotate the image before saving which means the saved image has the orientation flag set to zero so simply loading it into Photoshop and saving it solves the 'problem'.