OpenGL advice requested

Question

Level 1

11 points

OpenGL advice requested

Very new to programming OpenGL on Mac, here. But here's my goal...

I want to display a Windows BMP-formatted image onto a full-screen second 1920x1080 display. Such images are in BGR-byte order (no alpha) and are likely not to be powers-of-two in size. This seems to be a problem for the Mac.

I've looked into glTexSubImage2D and glTexImage2D, but it's entirely unclear which (if any) current Mac video cards support GL_BGR (or GL_BGR_EXT) as a pixel source format. Some indications are that NVidia supports BGR (eg. Quadro 4000), whereas current Radeon-class cards do not.

I won't want to resort to twiddling the B-R bytes in RAM as that will likely be too slow, even if it were heavily threaded on a multi-core CPU. Swizzling on the GPU is a possibility, I suppose, if it were much faster -- particularly with the likelihood of upcoming K-4000 class Kepler cards destined for the Mac. (Side note: if conversion to RGBA is a required step in RAM, is that frame buffer then available to CoreAnimation or some visual effects?).

The reason I want the code to be so fast is the high potential exists to have the BGR-image delivered from an uncompressed AVI file instead, in which case such processing needs to occur at 30fps. So, I need a sustained data rate of approximately 190MBps. Add to that 16 to 20 rectangular regions will determine a final frame, possibly from different data sources. I suspect double-buffering in GL will be a given.

And, again, I confused if using PBO's - FBO's are overkill in this situation, or if they're even required. It would be great to have direct access to the screen buffer with some IOKit stuff like IOFBBlitSurfaceCopy but from what I've read some of those types of solutions are very much deprecated/unavailable in OS 10.8.

Another side note: if CPU processing weren't a constraint, I suppose any compressed media could be used (eg. compressed MOV) as a source, but that's pipe dream.

Ya… so…at the top end I'm asking for an OpenGL (or Quartz2D) solution to playing an uncompressed AVI @ 30fps. To this end, there's an impressive (but dated) tutorial by NeHe on "Playing AVI files in OpenGL", but he too resorted to twiddling B-R bytes -- I don't really want to have to go there. I attempted to rejig his tutorial to use GL_BGR_EXT source formats, but got nothing but white.

Can anyone offer any advice? A start with very quickly displaying BMP (BGR) images would be great.

-- bp

Posted on Mar 10, 2013 9:47 AM

Reply

Answer 1

etresoft

Level 9

56,656 points

Mar 10, 2013 2:04 PM in response to zzyzy

Could you explain it at an even higher level? OpenGL is for 3D graphics. The functions you references are specifically for adding textures to a model. That has nothing to do with video. Furthermore, AVI is just a wrapper format. What are the internal codecs involved? Do those exist for the Mac and do you have access to them?

BMP is a native file format on OS X. AVI is not. On a Mac, you don't worry about the details of video cards. Code to the interface which will handle those details.

Reply

Answer 2

zzyzy Author

Level 1

11 points

Mar 10, 2013 3:02 PM in response to etresoft

Thank you for your response.

Here's the higher-level question: What is the quickest way of getting pixel data onto screen? As in, from BGR-format (in RAM) to visibly on the screen. If I'm using OpenGL then, in it's most crude form, my "model" will be a 2D face that is 1920x1080, and I'll be texturing that.

OpenGL isn't "exclusively" 3D. There's tons of useful 2D manipulation routines in there, and its arguably one of the faster ways to gain access to the video card. If I'm to be proven wrong, then which way is better to use -- Quartz2D?

I'm aware that AVI format as of OSX 10.5.8 is no longer supported. Having said that, the data I'm accessing is uncompressed -- no codec required. And I'm resorting to doing low-level file calls (fread) to load the pixel data. It's worth mentioning that the format for a frame of compressed AVI "video" data (headers aside) is identical to that of the BMP internal file format. Both are in BGR row/column order.

In other words, I'll have the RAW pixel data sitting in RAM. Unfortunately, that data is in BGR (24-bit per pixel) format and assumably needs to be converted to BGRA-format before it OpenGL sees it. Where is that to be done? RAM w/CPU... or can OpenGL more efficiently get the BGR data onto the GPU for the hardware to effortlessly handle it? If so, which Mac video cards will handle that conversion automatically?

Reply

Answer 3

zzyzy Author

Level 1

11 points

Mar 10, 2013 3:09 PM in response to zzyzy

More to the point: I have a yellow pixel in reverse BGR-format (0x00ffff) in memory. How do I get that onto my second monitor (1920x1080) at the position (0,0) -- on my otherwise blank physical screen.

Rinse and Repeat. For 1920x1080 at 30fps. Using a blessed API.

Reply

Answer 4

etresoft

Level 9

56,656 points

Mar 10, 2013 4:42 PM in response to zzyzy

Wrong way, Corrigan. That is even more low level. What are you doing? And why?

Get your mind out of the video card. No good will come from that. Apple has gone to great lengths to give you a number of high-performance APIs for reading and displaying image data. You will not be able to improve on that.

OS X already supports BMP. Look at Core Image and Core Graphics for the functions you want to use.

fread is not a low-level function. It is a high-level, buffered function designed to comply with the POSIX API. If you do need to read low-level data, that is not the one you want.

Reply

Answer 5

zzyzy Author

Level 1

11 points

Mar 10, 2013 5:13 PM in response to etresoft

Imagine that your Mac's video card is connected to a hardware device that appears as a second 1920x1080 DVI monitor. That hardware device segregates the display into 4 separate 960x540 external displays. I need to run 4 separate uncompressed AVI's @ 30fps on those displays. Does that provide a better idea of what I'm doing?

Well, by low-level, I mean that I'm not going to go and do this with a unsupported AVI files because, well, I can't...

src = CGImageSourceCreateWithURL ((CFURLRef)url, NULL);

image = CGImageSourceCreateImageAtIndex(src,0,NULL);

context = CGBitmapContextCreate...

...

CGContextDrawImage...

Relative to these high-performance API's, I call "fread" low-level.

The fact that Apple supports BMP is not helpful to me in this situation unless I can open a large (>4gb) uncompressed AVI file and point to a section of it and say to the API... "look, heres some BMP data! Use it!".

Hopefully you can see my need for fast access to the GPU.

Reply

Answer 6

etresoft

Level 9

56,656 points

Mar 11, 2013 7:05 AM in response to zzyzy

zzyzy wrote:

Imagine that your Mac's video card is connected to a hardware device that appears as a second 1920x1080 DVI monitor. That hardware device segregates the display into 4 separate 960x540 external displays. I need to run 4 separate uncompressed AVI's @ 30fps on those displays. Does that provide a better idea of what I'm doing?

Not really. People have been doing that for the past 20 years. I don't see what the big deal is. VLC or MPlayerX can play AVIs.

Well, by low-level, I mean that I'm not going to go and do this with a unsupported AVI files because, well, I can't...

src = CGImageSourceCreateWithURL ((CFURLRef)url, NULL);

image = CGImageSourceCreateImageAtIndex(src,0,NULL);

context = CGBitmapContextCreate...

...

CGContextDrawImage...

Relative to these high-performance API's, I call "fread" low-level.

The fact that Apple supports BMP is not helpful to me in this situation unless I can open a large (>4gb) uncompressed AVI file and point to a section of it and say to the API... "look, heres some BMP data! Use it!".

Hopefully you can see my need for fast access to the GPU.

No. Get your mind out of the GPU.

Many Apple engineers have been working for many years to build a large, robust graphics API. No single developer is going to be able to copy and paste some snippets from Stack Overflow and do better. It simply isn't possible. Stop thinking about your hardware and start reading the documentation.

Reply

Answer 7

zzyzy Author

Level 1

11 points

Mar 11, 2013 7:21 AM in response to etresoft

I want the ability to take a large single (>4gb) AVI file and dice it up and place those dicings wherever I wish on that second display... all at 30fps.

Nothing in the Apple API does that. And VLC or MPlayerX most certainly don't do that.

I am reading the documentation -- something very high performance is needed that lets me push BGR-encoded pixels about. In other words, once I have a section of BGR-encoded data from the AVI in RAM, I want to get that to its rectangular destination on that GPU frame buffer ASAP.

Reply

Answer 8

etresoft

Level 9

56,656 points

Mar 11, 2013 7:47 AM in response to zzyzy

zzyzy wrote:

Nothing in the Apple API does that.

<sigh>

Programming 101

Step 1: Optimization

Reply

Answer 9

zzyzy Author

Level 1

11 points

Mar 11, 2013 8:24 AM in response to etresoft

Consider this. We have a large 40-foot long "strip" display that's 7680x270. Same number of pixels as 1920x1080.

Four rectangular dicings of 1920x270 would cover it, and our external hardware device will convert the 1920x1080 to 7680x270.

On the creative end, you would be asked to fabricate animations that were 7680x270. But imagine being in a place where you can't provide 7680x270 animations. (BTW, for the sake of the hardware involved, uncompressed AVI is the only format accepted). You must "optimize" (aka "predice") it to be 1920x1080. Not everyone has software tools that do that dicing and, pragmatiically, it's not going to happen either.

I will repost my query as something related to blitting.

Reply

Answer 10

etresoft

Level 9

56,656 points

Mar 11, 2013 8:55 AM in response to zzyzy

OS X is already designed to support displays 2.5 times bigger than that. Worry about parsing your AVI stream. Pick any one of the dozen file access methods. Once you have the data, then you can worry about which of the dozen display methods would be best. Here is a hint - start with the easiest ones. Don't automatically assume they can't support your data rate.

Reply

Answer 11

zzyzy Author

Level 1

11 points

Mar 11, 2013 9:09 AM in response to etresoft

Take this to an extreme to be most flexible -- if you attend a pro-sports event, the facility's "ribbon" boards are likely 32000x64 or some such ridiculous size.

OSX does NOT support that natively. Neither does WindowsXP.

Yet WinXP dominates this display market. With uncompressed AVI's ONLY, and uber-graphics cards.

Reply

Answer 12

etresoft

Level 9

56,656 points

Mar 11, 2013 11:29 AM in response to zzyzy

It seems like you are running yourself in circles, creating problems, without ever bothering to explore how you would go about doing. You are taking millions of lines of highly optimized graphics API and discarding them without even looking first.

Have you even bothered to display any BMP files yet? Call me crazy, but I think that would be your first step.

Reply