Skip to content

January 26, 2011

Surviving the Unit Testing Wasteland of Objective-C and OCUnit

Sooner or later you’re going to want to unit test your Cocoa apps. You’ll put it off as long as possible, but you’ll start to feel dirty without them. Your app will start to feel like peanut butter without jelly, or hot cocoa without marshmallows. You’ll put up with anything to get your code unit tested. No setup cost will be too great.

That may sound a little codependent, and unit testing in objective-c is a lot like an emotionally abusive relationship. There are a bunch of options out there, but OCUnit just feels comfortable and besides, it’s been with you since the beginning (because it came bundled with Xcode). OCUnit can’t stay terrible forever, it’s got to change right? right? What’s so bad about OCUnit anyway?

What’s Wrong with OCUnit?

Well for starters, all OCUnit tests happen at build time, not as separate run-time tasks. This small distinction makes it very difficult to debug unit tests. You can set things up so that when you choose Build and Debug in Xcode, it runs the tests once at build time and then a second time as your app starts running. On the second run-through, breakpoints will get hit, but the debugger doesn’t really work. It’s a mess that’s annoying to deal with, so you’re probably going to start falling back to NSLogging everything and just running the tests through Build.

Another problem is that there’s no separate GUI that shows all of your tests as green or red for pass or fail. If a test fails, the build fails. To find out which one failed, you have to drill down to the specific test in the build results window. It’s hard to do Red, Green, Refactor without red or green. There are third-party GUIs such as OCRunner that are doing their best to usher OCUnit into the mid-aughts by providing a nice GUI. I haven’t been able to get them working yet though (maybe a future post).

Finally, OCUnit requires specific naming conventions for test classes and methods. I admit that this is more of a style choice than a genuine shortcoming. In some other unit test packages such as NUnit for .NET, classes and methods are identified as test cases in metadata (NUnit uses attributes.) With OCUnit however, all test class names must end with “TestCase” and each test method must begin with “test” in order for them to be recognized by OCUnit. This may seem like just the choice of convention over configuration, but it’s really convention as configuration. Instead of making things easier and more intuitive, it hides information about the behavior of a class or method in its name.

What’s Right With OCUnit?

Ok, now you’re about to leave a comment “If you think OCUnit sucks so much, don’t use it!”, but that’s not true. I don’t think OCUnit is that bad. It gets the job done and could grow into the testing framework that we all want it to be. It does have some things going for it too.

When you’re choosing tools, a real concern is that development on a particular tool will stagnate and you’ll be left in the cold with outdated tools that you can’t migrate from. OCUnit is bundled with Xcode, so development is probably going to continue for the foreseeable future. Who knows, maybe Apple already has developers dedicated to improving it. Or writing a unit testing framework that’s compatible with it.

Also, despite my ranting, once you get OCUnit running and know its quirks, it’s pretty easy to use. Its shortcomings only really cost you time when setting it up and learning how to use it. Most of the problems I’ve mentioned above don’t have recurring costs.

Conveniently, you can minimize that initial cost of setting up and learning OCUnit by reading the rest of this post. It will not only help you survive the wasteland, but thrive with Unit Testing in the Objective-C world.

Add a new testing target

The first thing we need to do is add a new Unit Test Bundle target to an existing project that we want to unit test. Right click on the Targets section of the project file list and click Add>New Target.

Target Aqcuired.

Choose Unit Test Bundle and click Next.

Bundle Up!

Usually people name it HotPotatoTests where HotPotato is the name of their project. I don’t think the naming affects anything though, so feel free name it a cuss word if you like and click Finish.

I recommend naming it something safe for work if you're at work.

Add a new group for your test files

We’re probably going to end up writing at least one test class per functionality class, so the project’s going to start to get messy. Let’s just nip that clutter in the bud right now and create a group called Test into which we will put all of our test case files. Right click your project’s name in the Groups & Files pane, and go to Add>New Group. You should probably not name this a cuss word, you’re just going to confuse yourself.

Add a new Test Case file

Now it’s time to add a new testing class. Right click your newly created Test group and go to Add>New File… Choose Objective-C Test Case Class from the Cocoa Class section and hit Next.

This next part is VERY IMPORTANT. It is the first of two things that are absolutely vital to saving yourself frustration. Every test case class in OCUnit must end with the suffix TestCase. A nice convention is to name it HotPotatoTestCase where HotPotato is the name of the class under test. That way it’s stupid easy to find the tests for that class. Don’t forget to add TestCase to the end though. Promise me you won’t forget. I’m serious, pinky swear in the comments.

Once we have a test case class, the first thing we should do is write a failing test and run it to make sure it fails. Add the following code to the .m file of the test case class:

-(void) testThatTestingIsWorking
{
	STFail(@"YOU WILL ALWAYS BE A FAILURE");
}

This is the second VERY IMPORTANT part. It is again crucial to your sanity that you remember to do the following exactly: You must name all test methods so that they begin with the word test. Don’t forget! Oh please don’t forget.

Run your failing test

Now click that little unlabeled drop down in the upper left corner of the main Xcode window and switch the Active Target to your test target.

The next part is optional, but you probably want it. Expand the Targets section of your project. Now click and drag your executable target onto your test target. This will make your executable a dependency of your test target. Now, every time you attempt to build the test target, it will build the executable first. This ensures that the code under test always reflects the current code and not just whatever is lying around from the last time you built your executable.

Now comes the fun part, running the test! Just go to Build>Build and your test should run. The first time I did this, I was like “oh shit there are build errors, I’m doing this tutorial wrong.” Don’t worry, you’re not doing it wrong. Well you may be doing it wrong, but you should be expecting build errors here because a test failure counts as a build error. We need to dig a little to see exactly what the build error is though.

Finding the Failure

First bring up the Build Results window by doing command+shift+b. At this point, we could simply do a full expansion on the line where you see the errors by clicking the little button. But that gets you a big long log file that sucks to parse with your eyes. We can get a slightly more readable list of results by expanding Run custom shell script ‘Run Script’ > Run unit tests for architecture > Run test suite <octest path>. Here you will see a list of test case classes with an exclamation point if they’ve failed. You can further expand that to see the actual test methods that failed, and then drill down to the actual failure.

Ok, I guess it is red.

It’s annoying, but you’ll get used to it. Because you have to.

Test your actual code from your actual project

Most of the tutorials I’ve encountered stop here without telling you how to reference the code in your executable from your test case classes. It’s not intuitive but it’s just something you have to learn to do. I don’t know why they usually exclude this part, maybe the authors didn’t actually try to unit test anything significant. We live in the real world though, so we need to actually reference our actual code.

The first thing we’ll do is go to the Groups & Files pane and drag the code files (just the .m files) under test into the Compile Sources folder inside your test target. This is the magic that tells Xcode that you’re going to need access to the code from these files when you build the test target. So, if you’re testing a class called StringBasedTime, you grab StringBasedTime.m and drop them into the folder under Targets>HotPotatoTests>Compile Sources (where HotPotatoTests is the name of your test target.)

Cuss words everywhere!

Then in the test case file that you’re using to test this code (StringBasedTimeTest.m,) you simply add a directive to import the header file:

#import "StringBasedTime.h"

And that’s it, you can reference your executable class in your test code!

Example Unit Test

Here’s a simple unit test that I wrote in real life for a real reason. I’m using it to test the initializer of a class I’m building called StringBasedTime. StringBasedTime takes an argument in it’s initalizer that is a string representation of a time interval. The string representation is based on the output of QTStringFromTime, which is a method in QTKit, Cocoa’s QuickTime framework. It converts a QTime to an NSString in the format dd:hh:mm:ss.ff/ts. For example, @”0:00:00:00.00/00″ is the very beginning of a QuickTime movie. The test we’re looking at just makes sure that if someone initializes a new StringBasedTime with @”0:00:00:00.00/00″, each of the properties gets populated with zero. If one doesn’t, I want the failure output to tell the tester which property is non-zero. Here’s the code from the test case’s .m file:

#import "StringBasedTimeTestCase.h"
#import "StringBasedTime.h"

@implementation StringBasedTimeTestCase

-(void) testInitWithStringZeroTime
{
	NSString *ZeroTimeString = @"0:00:00:00.00/00";
	StringBasedTime *ZeroTime =
		[[StringBasedTime alloc] initWithString:ZeroTimeString];

	NSString* FailText = @"";

	if([ZeroTime days] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d days when 0 expected."
					, [ZeroTime days]];

	if([ZeroTime hours] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d hours when 0 expected."
					, [ZeroTime hours]];

	if([ZeroTime minutes] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d minutes when 0 expected."
					, [ZeroTime minutes]];

	if([ZeroTime seconds] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d seconds when 0 expected."
					, [ZeroTime seconds]];

	if([ZeroTime timescale] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d timescale when 0 expected."
					, [ZeroTime timescale]];

	STAssertEquals(@"", FailText, FailText);

}
@end

Debugging

Remember, NSLog is your friend when debugging. I know printf debugging sucks, but sometimes it’s your best option. It might be possible to get the debugger working with OCUnit, but I haven’t been able to crack that code. If you figure it out, please feel free to leave a comment!

Further Reading

I figured out how to do all of this from a few good tutorials on OCUnit (and a few bad ones!):

Leave a Reply

Your email address will not be published.