EXIF Orientation Flags and the iPhone Camera

Alright, I have a rant. Bear with me…

Layers just came out this past Monday, and it has this great feature that allows you to add a layer to a drawing from your iPhone’s photo library. Simple enough – right? Apple provides the UIImagePicker API, we call a couple functions and get an image back.

For most purposes, that would work great! Write some code, test, commit, done. The problem is, the picker interface allows the user to adjust the scale/positioning of the image, and the cropped image is always returned at 320x320px (or less). 320px is really quite pathetic, and it means the images are smaller than the 512x512px drawing canvas in Layers. I could scale up each photo when it’s added to the drawing, but that’d be pretty lame.

The UIImagePicker API provides an editInfo dictionary containing the original image and the cropping rect information, so I decided to grab the original and re-perform the adjustments. Using the cropRect provided by the API, I could just re-crop the large, original image to 512×512… right?

Unfortunately, no. Photos taken with the iPhone’s camera use the industry-standard EXIF orientation flag to store rotation information. That means that the image data is always saved upright, and it’s the client application’s job to realize it should be rotated 90º, 180º or 270º because the user was holding the camera upside down or sideways.

Technically, this is great. The problem is, the editInfo dictionary contains (1) the original image and (2) the crop rect, defined in the coordinate space of the image after the EXIF orientation flag is taken into account. You can’t just jump in and crop the original image, because one has had transformations applied and the other hasn’t. So there are two options:

- Option 1: Rotate the original image you’re given based on the EXIF data, and then crop it using the cropRect. This is slow because you have to rotate the entire image and then you end up throwing most of it away. For extremely large images (which can be added into your photo library via Mail attachments), it fails entirely.
- Option 2: Adjust the cropping rect and undo the transformations that have been applied to it based on the EXIF data. This is better, but it requires writing some nasty CGRect transformations and lots of boxes drawn on paper.

I decided to go with option 2. I wrote a nice big switch statement to undo the transformations for each of the eight possible EXIF values. But then I discovered something else:

iPhone Photos App with Different EXIF Orientation FlagsSomebody was lazy. The iPhone’s Photos application only understands EXIF orientations 1, 3, 6, and 8. These correspond to the common orientations: UIImageOrientationUp, UIImageOrientationDown, UIImageOrientationLeft, and UIImageOrientationRight. Photos with the other four orientations (the “mirrored” ones) appear unrotated in the photo browser. (See screenshot at right. Numbers on the images correspond to their EXIF orientation values).

I want the user to get what they expect to get when they add a photo – even if it isn’t what they want. If the image is sideways while they’re cropping it, it should still be sideways when they press OK. I promptly deleted code for the other orientations so as to handle them as badly as the photo browser. Cool. moving on…

iPhone Photos App - Weird Vertical Pan BehaviorThe image picker allows the user to zoom in on the image of their choosing and pan around it, but the pan functionality is broken. You can pan beyond the edge of the image along the vertical axis, so that image is only partially visible within the gray cropping rectangle. This can lead to some strange results. A cropRect of (0,0,320,200) for an image of size (512,512), for example, indicates that a black gap is present at the top of the crop region. After playing around with this for a while, I was able to figure out how to differentiate between the cropRect values and properly draw the image to appear exactly as it did in the preview.

Fixing this problem took almost 5 hours – and it really shouldn’t have. Here’s the code that takes the original image and cropRect and re-performs the adjustments to yield a 512×512 cropped image that matches exactly what the user saw when they clicked “Choose” in the picker:

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingImage:(UIImage *)img editingInfo:(NSDictionary *)editInfo
{
    if ([picker sourceType] == UIImagePickerControllerSourceTypeCamera){
        // save the image to the photo library
        UIImageWriteToSavedPhotosAlbum(img, nil, nil, nil);
    }

	[self dismissModalViewControllerAnimated:YES];

    NSDictionary * assets = [NSDictionary dictionaryWithObjectsAndKeys:img, @"smallCroppedImage", editInfo, @"editInfo", nil];
    [self performSelector:@selector(imagePickerControllerDidFinishThreaded:) withObject:assets afterDelay:0.05];
}

- (void)imagePickerControllerDidFinishThreaded:(NSDictionary*)assets
{
    NSDictionary        * editInfo = [assets objectForKey: @"editInfo"];
    CGRect                editCropRect = [[editInfo valueForKey:UIImagePickerControllerCropRect] CGRectValue];  

    // 1. Determine original image orientation and size
    UIImage             * originalImage = [editInfo valueForKey: UIImagePickerControllerOriginalImage];
    UIImageOrientation    originalOrientation = originalImage.imageOrientation;
    CGSize                originalSize = originalImage.size;
    CGSize                desiredSize = CGSizeMake(512,512);

    // 2. Modify crop rect to reflect image orientation
    CGFloat oldY = editCropRect.origin.y;
    CGFloat oldOriginalW = originalSize.width;
    CGFloat tmp;

    switch (originalOrientation) {
        case UIImageOrientationUp:      //EXIF 1
            break;

        case UIImageOrientationDown:    //EXIF 3
            // X flipped horizontally
            // Y flipped vertically
            editCropRect.origin.x = originalSize.width - (editCropRect.size.width + editCropRect.origin.x);
            editCropRect.origin.y = originalSize.height - (editCropRect.size.height + editCropRect.origin.y);
            break;

        case UIImageOrientationLeft:    //EXIF 6
            // fix info for original image.
            originalSize.width = originalSize.height;
            originalSize.height = oldOriginalW;

            // fix crop rect
			tmp = editCropRect.size.height;
			editCropRect.size.height = editCropRect.size.width;
			editCropRect.size.width = tmp;

            // rotation to the left
            editCropRect.origin.y = originalSize.height - (editCropRect.origin.x + editCropRect.size.height);
            editCropRect.origin.x = oldY;
            break;

        case UIImageOrientationRight:   //EXIF 8
            // fix info for original image.
            originalSize.width = originalSize.height;
            originalSize.height = oldOriginalW;

            // fix crop rect
			tmp = editCropRect.size.height;
			editCropRect.size.height = editCropRect.size.width;
			editCropRect.size.width = tmp;

            // rotate to the right
            editCropRect.origin.y = editCropRect.origin.x;
            editCropRect.origin.x = originalSize.height - oldY;
            break;

        default:
            break;
    }

    // 2.5. make the damn thing square if it's ALMOST square
    if (fabs((editCropRect.size.height - editCropRect.size.width) / fminf(originalSize.height, originalSize.width)) < 0.0295){
        editCropRect.size.width = fminf(editCropRect.size.width, editCropRect.size.height);
        editCropRect.size.height = editCropRect.size.width;
    }

    // 3. Crop image using crop rect
    UIGraphicsBeginImageContext(desiredSize);
	CGContextRef context = UIGraphicsGetCurrentContext();
	CGImageRef image = CGImageCreateWithImageInRect([originalImage CGImage], editCropRect);
    CGRect imageRect = CGRectMake(0.0f, 0.0f, desiredSize.width, desiredSize.height);

    // Image width < Image height. Just center vertically
    if (editCropRect.size.width / editCropRect.size.height < 1){
        imageRect.origin.x = (desiredSize.width - editCropRect.size.width * desiredSize.height/editCropRect.size.height)/2;
        imageRect.size.width -= imageRect.origin.x * 2;

    // Image width > Image height
    } else if (editCropRect.size.width / editCropRect.size.height > 1){
        float extraHeight = desiredSize.height - editCropRect.size.height * (desiredSize.width / editCropRect.size.width);

        // If the crop rect's origin is at the top of the screen, some of it might be clear (IE, the user may
        // have dragged "too far" and have some white space at the top of the preview box
        if (editCropRect.origin.y == 0) {
            imageRect.size.height -= extraHeight;
            if (roundf(editCropRect.size.height) == roundf(originalSize.height))
                imageRect.origin.y = extraHeight / 2;
            else
                imageRect.origin.y = 0;

        // User dragged "too far" down, and white space is visible at the bottom of preview box
        } else if (fabs(editCropRect.origin.y - (originalSize.height - roundf(editCropRect.size.height))) <= 1.1) {
            imageRect.origin.y = extraHeight;
            imageRect.size.height -= extraHeight;

        }else {
            imageRect.origin.y = (desiredSize.height - editCropRect.size.height * desiredSize.width/editCropRect.size.width)/2;
            imageRect.size.height -= imageRect.origin.y * 2;
        }
    }

    CGContextClearRect(context, CGRectMake(0,0,desiredSize.width,desiredSize.height));
	CGContextDrawImage(context, imageRect, image);
	UIImage* croppedImage = UIGraphicsGetImageFromCurrentImageContext();
	UIGraphicsEndImageContext();
	CGImageRelease(image);

    // 4. Perform image rotation
    UIImage * finalImage = [self rotateImage: croppedImage byOrientationFlag: originalOrientation];

    // DO SOMETHING WITH finalImage!
}

#pragma mark Convenience Functions for Image Picking

- (UIImage*)rotateImage:(UIImage*)img byOrientationFlag:(UIImageOrientation)orient
{
	CGImageRef          imgRef = img.CGImage;
	CGFloat             width = CGImageGetWidth(imgRef);
	CGFloat             height = CGImageGetHeight(imgRef);
	CGAffineTransform   transform = CGAffineTransformIdentity;
	CGRect              bounds = CGRectMake(0, 0, width, height);
    CGSize              imageSize = bounds.size;
	CGFloat             boundHeight;

	switch(orient) {

		case UIImageOrientationUp: //EXIF = 1
			transform = CGAffineTransformIdentity;
			break;

		case UIImageOrientationDown: //EXIF = 3
			transform = CGAffineTransformMakeTranslation(imageSize.width, imageSize.height);
			transform = CGAffineTransformRotate(transform, M_PI);
			break;

		case UIImageOrientationLeft: //EXIF = 6
			boundHeight = bounds.size.height;
			bounds.size.height = bounds.size.width;
			bounds.size.width = boundHeight;
			transform = CGAffineTransformMakeTranslation(imageSize.height, imageSize.width);
			transform = CGAffineTransformScale(transform, -1.0, 1.0);
			transform = CGAffineTransformRotate(transform, 3.0 * M_PI / 2.0);
			break;

		case UIImageOrientationRight: //EXIF = 8
			boundHeight = bounds.size.height;
			bounds.size.height = bounds.size.width;
			bounds.size.width = boundHeight;
			transform = CGAffineTransformMakeTranslation(0.0, imageSize.width);
			transform = CGAffineTransformRotate(transform, 3.0 * M_PI / 2.0);
			break;

		default:
            // image is not auto-rotated by the photo picker, so whatever the user
            // sees is what they expect to get. No modification necessary
            transform = CGAffineTransformIdentity;
            break;

	}

	UIGraphicsBeginImageContext(bounds.size);
	CGContextRef context = UIGraphicsGetCurrentContext();

    if ((orient == UIImageOrientationDown) || (orient == UIImageOrientationRight) || (orient == UIImageOrientationUp)){
        // flip the coordinate space upside down
        CGContextScaleCTM(context, 1, -1);
        CGContextTranslateCTM(context, 0, -height);
    }

	CGContextConcatCTM(context, transform);
	CGContextDrawImage(UIGraphicsGetCurrentContext(), CGRectMake(0, 0, width, height), imgRef);
	UIImage *imageCopy = UIGraphicsGetImageFromCurrentImageContext();
	UIGraphicsEndImageContext();

	return imageCopy;
}

I hope that it saves you time - please leave a comment below if you find it useful! Also - I've attached a ZIP file with eight images that can be used to debug problems with EXIF orientation handling. Each image has a different EXIF orientation flag value, and a giant number in the center of the image lets you know what it is. On the Mac desktop, all eight will appear to be vertical because QuickLook properly adjusts them based on their EXIF values. Other apps, like Fireworks, will open them sideways, upside-down, backwards, etc... Enjoy!

Download EXIF Orientation Sample Images (44MB)

  1. Agha

    This is the best code ever seen. Only problem I had to copy the code. The line numbers come along. :-)
    I wish it was a whole project.
    Thank you for code.
    Agha

  2. Michael

    Hey,

    that’s really a great function, but I run into trouble somehow…

    I just need the rotateImage method. So what I do is calling this one:
    image = [self rotateImage:image byOrientationFlag:originalOrientation];

    to override the image with the right image. If I do this, my UIImageView which gets this image afterwards displays nothing. It’s just transparent. Where as if I return img; at the end of your function everything is displayed (just the wrong way of course), so it seems to be something wrong with the newly generated image. Do you have any ideas to that?

    Thanks, Michael

  3. Rich

    When I realised I had this problem, I didn’t have a spare 5 hours to come up with a solution myself so thank you so much for clearing the path for me. It’s a great piece of code and it does exactly what I needed it to.

    One problem I had was that the photos saved to the screen were always mirrored along the x or y axis (depending on the original orientation). I had to fix it by transforming the UIImageView I was displaying. Is the mirroring expected behaviour? Would there be a nicer way to correct it within the code above as opposed to my UIImageView hack?

  4. Ricardo Garriota

    You rock, dude. Always keep that in mind.

  5. Thanks for the information. I’m dealing with some orientation issues in the old Image Capture framework on the Mac. (It doesn’t automatically rotate the thumbnail for an item.) Have you ever seen an image in the wild that used a mirrored orientation?

  6. Gabriel

    Thanks a milion for the sample images!!

  7. Hey Ben, this is great. I recently found your site when I had to solve a subset of this problem. Because my solution is specific to the selection rect and it’s transformation, I wrote some code for transforming the rect using CGRectApplyAffineTransform. And I can fully sympathize with your rant!

    Here’s my post:
    http://niftybean.com/main/blog/16-selecting-regions-from-rotated-exif-images-on-iphone

    Also I found different correspondences between EXIF orientation codes and UIImageOrientationXXXX enum values. When I ran my tests, UIImageOrientationLeft = EXIF code #8, and UIImageOrientationRight = EXIF code #6.

  8. Thank you very much for the sample code. I struggled with a solution for this myself, but yours is cleaner.

  9. Rob Bruce

    I also had issues with the images getting released before I could use them. I wasn’t dealing with cropping, so I just kept the rotate code.
    My iPhone always rotates the picutres to the right, not sure why. In order to get them perfectly aligned I had to change the code like this or they were off the top by a bit.

    if ((orient == UIImageOrientationDown) || (orient == UIImageOrientationRight) || (orient == UIImageOrientationUp)){
    // flip the coordinate space upside down
    CGContextScaleCTM(context, 1, -1);
    CGContextTranslateCTM(context, 0, -width);
    }

    I changed -height to -width.
    Rob

  10. Hi Ben!

    Thank you for this great code. I enhanced the image orientation handling for all possible values of UIImageOrientation:

    Here’s my post:
    http://www.mrh-line.de/Micha/Web_Log/Entries/2010/4/14_Exif_orientation_in_iPhone_SDK.html

  11. Wei

    Only took 5 hours to figure it out…You are genius!!

  12. Fixed a bug:

    case UIImageOrientationRight: //EXIF = 8
    161. boundHeight = bounds.size.height;
    162. bounds.size.height = bounds.size.width;
    163. bounds.size.width = boundHeight;
    164. transform = CGAffineTransformMakeTranslation(0.0, imageSize.width);
    165. transform = CGAffineTransformRotate(transform, 3.0 * M_PI / 2.0);
    166. break;

    Should use imageSize.height in the translation part, not imageSize.width.

  13. Rafael

    Thankz man, the rotateImage method worked for me and helps me a lot.
    Regards

  14. Adam

    Great post, it’s helped me out a lot. I was wondering if you could give me a couple of pointers…I’ve spent the last day battling how to get cropping working. I have a camera overlay, and all I’m trying to do is take a small square crop from underneath the finger on camera press and save it. My overlay view is aligned perfectly with the camera view, but the cropped area never matches the finger press….I’m using your algorithm for UIViewOrientation calculation. I can’t for the life of me figure out where I’m going wrong.


    UIImage *image = [info objectForKey:UIImagePickerControllerOriginalImage];
    //figure out the scale between the image and the screen
    //is it possible that my problem comes from the fact that the
    //screen and the image are different width/height ratios?
    CGFloat scaleX = image.size.width / SCREEN_WIDTH;
    CGFloat scaleY = image.size.height / SCREEN_HEIGHT;

    //get the relative x/y locations on the image, back by the size I want to crop out
    CGFloat x = (overlayViewController.lastTouch.x * scaleX) - (CROP_WIDTH/2.0);
    CGFloat y = (overlayViewController.lastTouch.y * scaleY) - (CROP_WIDTH/2.0);
    CGRect cropArea = CGRectMake(x, y, CROP_WIDTH, CROP_WIDTH);

    //use cropArea in your algorithm to crop

    //save the images
    UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil);
    UIImageWriteToSavedPhotosAlbum(croppedImage, nil, nil, nil);

    I have a sneaking suspicion that the reason my crop area doesn’t match the image is because they’re different ratios of width/height (iphone 1.5, image 1.33)….but I’m not sure. Is there a better way to do this (basically crop a small rectangle around the touch area?)

    Cheers
    Adam

  15. Adam

    Followup – yeah, I’m pretty sure the issue is transposing the 3:2 coordinates system onto the 4:3 scaled up image. I could always transform/scale the image, but as your article points out, this is slow and clunky…pity there’s no library to transform a CGRect to a new aspect ratio, seems like something that many people would want to do with the increase in aug reality apps.

    Do you know any resources to elaborate on this, I’ve tried to come up with the math myself by I keep missing the mark (been a long time since I’ve done any math! Maybe just getting old).

    Cheers
    Adam

  16. bygreencn

    This is the best code.
    When i meet this problem few weeks ago, i had fixed it go with the option1 as you mention, but i found it will spent more time to rotate the large image(1600×1200), as my application requirement, i do not need so large image, so i scaled it and rotated, it works.
    Now i found your solution, i will check and change my code, since it is better choice to give larger image to user.
    Thanks!

  17. Tony

    Thanks a million! Saved me a lot of time. It’s ludicrous that we should have to do this ourselves. Whatever happened to WYSIWYG?

  18. fredwardo

    Searched all over the web trying to figure this out. Your code is fantastic! Thanks very much for posting this. It’s helped me out a ton!

  19. Josh

    Nice work, this did the trick for my app!

Reply to 'EXIF Orientation Flags and the iPhone Camera'