Ninjas are deadly. Chipmunk Ninjas are just weird.
About this blog
Marc Travels
Marc on Twitter
JustLooking on Twitter

Marc Wandschneider is a professional software developer with well over fifteen years of industry experience (yes, he really is that old). He travels the globe working on interesting projects and gives talks at conferences and trade shows whenever possible.

My Publications:

My book, "Core Web Application Programming with PHP and MySQL" is now available everywhere, including

My "PHP and MySQL LiveLessons" DVD Series has just been published by Prentice-Hall, and can be purchased on Amazon, through Informit, or Safari

Jun 29, 2007 | 20:50:34
Things I've learned about CoreImage (and Quartz, and OpenGL) in two weeks
By marcwan

I recently spent two weeks converting JustLooking, my Mac OS X Image Viewing program, from NSImage to CoreImage and friends. This experience was overall much easier than I expected, and I have learned a bunch of things, some of which might have been handy to have known in advance.

The good news is that it mostly lives up to the hype. The bad news is that it’s not without tricks and traps of its own. Here are some notes and comments.

  1. Once set up, properly configured, coded, and tuned, CoreImage rocks. Whereas before 10fps NSImage transitions between medium images dragged on an Intel Mac, 30fps transitions on 1GHz G4s are smooth like butter.
  2. Proper configuration is a must. I first just converted my various NSImage calls to their CoreImage counterparts (AppKit makes it easy to put CoreImage into your program), and saw maybe a 10-15% performance improvement.
  3. Using CoreImage pretty much requires you to start getting comfy with OpenGL and its usage in Mac Applications. Instead of NSView classes, you need to use NSOpenGLView. Going to do drawing in the view along with your CoreImage? You’re going to need to learn how to do this in OpenGL (or, if you’re like me, find some samples that do the same thing you’re doing and template heavily).
  4. This one took me the longest time to figure out: In Interface Builder, if your window has a subclassed NSView called MyNSView, simply changing MyNSView’s superclass to be NSOpenGLView instead of NSView and changing your code files likewise is not enough. My views were drawing all sorts of weird crap all over the place and it was driving me crazy. The part I was missing? You have to go back to IB
    • Delete your NSView from the window.
    • Drag on a new NSOpenGLView
    • And finally mark its subclas as being MyNSView (which now inherits from NSOpenGLView instead of NSView).
  5. NSOpenGLView is quirky. The weirdest of them all: It often doesn’t call your drawRect: when you first display it or tell it to change its contents. Thus, it seems as though a lot of people (myself included) have developed code that sets up a single-fire timer for some short period of time (0.1 seconds or so) to cause a redraw, which is respected.
  6. I still cannot get an NSOpenGLView to be semi-transparent. It’s trivial to do this with an NSView—just call [[NSColor colorWithDeviceRed: 0 green: 0 blue: 0 alpha: 0.7] set]. In some situations, overriding - (BOOL)isOpaque might be required so you can return NO. Doing this in the NSOpenGLView, however, is a monumental effort, and something I have to figure out. I found a few examples on the Internet, but was never able to make one work. (If you have done this before, please please please send me mail or add a comment).
  7. Any given CIImage instance you have is probably not an actual image. Instead, it’s a collection of instructions that the CoreImage code will later use to figure out the best way to render your final image to the screen, using all sorts of cool and clever optimisations. Applying resizing, scaling, and rotating transformations are all done at once, as none are actually computed until rendering time.
  8. The filters you can use with CoreImage images are just awesome. From the banal like regular NSAffineTransforms, to impressive scaling filters, to all sorts of colour and distorting filters, you might be forgiven for thinking you could write you own little Photoshop clone in a few hours. The CILanczosScaleTransform filter, in particular, provides excellent results (sadly, it isn’t quite fast enough to use in real time).
  9. The transitions available in CoreImage are all quite cool, and reasonably easy to use. Most simply require the setting of a few params and you’re good to go—one or two might require a mask image. The one thing the docs do not explain very well, however, is how to use the transitions. Oh sure, there’s a full page in the Programmer’s Guide, but it blithely leaves out a few of the key steps, leaving you scratching your head. Which brings me to my next point:
  10. If you’re going to use a CoreImage transition, download and learn to love the CITransitionSelectorSample2 sample. Until you start playing with CoreImage (or have read this article), you might not have noticed that it uses an NSOpenGLView, that it fires a timer to make sure that NSOpenGLView always paints correctly, and more. And It shows very clearly how to use all of pre-canned transitions in Tiger.
  11. The CoreImage classes are not without quirks, possibly bugs (although I’m such a newbie at this that I’m still willing to believe they were faults of my own). In my program I simultaneously do things like rotations, translations (so that images remain centred in the window), and scales to draw images on the screen. On a few occasions, I found that CoreImage would draw the images correctly for a few frames in a transition, and then suddenly start drawing them in unexpected places. Only by eliminating one of the transforms was I able to eliminate these problems.
  12. The CIImage method imageWithContentsOfURL: is extremely useful for quickly loading in images, but won’t let you get at things like EXIF or other image meta-data. You’re far better off using CGImageSourceRef APIs to load in CGImages, and then pass these to the CIImage method imageWithCGImage:. These will even let you load in all the frames in an animated GIF (something which NSImage simply does not).
  13. For thoses cases where you do want to use an NSView, such as printing, you’re still more than welcome to use CIImage objects. JustLooking does its printing in a regular NSView, and just uses AppKit extensions to the CIImage class to do the drawing. Extremely convenient!
  14. This isn’t so much a CoreImage observation as much as an artifact of design changes I made in order to best use CoreImage: Drawing text in an NSOpenGLView is a monstrous pain in the butt. There are various classes and tutorials on how to do this, which is quite disheartening. I avoided tackling this issue and got around this by just using other NSViews.

In summary, I’m extremely glad I took the CoreImage leap. The system is extremely cool, and has pushed NSImage to a minor utility role in my programming world. If you do decide to go for it, take heart—it’s really not so bad, and internet searches will give you all the help you could want.

Comments (1) Add Comment | Tags: mac osx cocoa coreimage programming quartz opengl nsimage
6. NSOpenGLView transparent
Posted By: cncool Jun 16, 2011 09:35:50
GLint zeroOpacity = 0;
[[self openGLContext] setValues:&zeroOpacity forParameter:NSOpenGLCPSurfaceOpacity];

Then call glClear(0,0,0,0);
Add a Comment





Copyright © 2005-2008 Marc Wandschneider All Rights Reserved.