Skip to content

Archive for January, 2011

Surviving the Unit Testing Wasteland of Objective-C and OCUnit

Sooner or later you’re going to want to unit test your Cocoa apps. You’ll put it off as long as possible, but you’ll start to feel dirty without them. Your app will start to feel like peanut butter without jelly, or hot cocoa without marshmallows. You’ll put up with anything to get your code unit tested. No setup cost will be too great.

That may sound a little codependent, and unit testing in objective-c is a lot like an emotionally abusive relationship. There are a bunch of options out there, but OCUnit just feels comfortable and besides, it’s been with you since the beginning (because it came bundled with Xcode). OCUnit can’t stay terrible forever, it’s got to change right? right? What’s so bad about OCUnit anyway?

What’s Wrong with OCUnit?

Well for starters, all OCUnit tests happen at build time, not as separate run-time tasks. This small distinction makes it very difficult to debug unit tests. You can set things up so that when you choose Build and Debug in Xcode, it runs the tests once at build time and then a second time as your app starts running. On the second run-through, breakpoints will get hit, but the debugger doesn’t really work. It’s a mess that’s annoying to deal with, so you’re probably going to start falling back to NSLogging everything and just running the tests through Build.

Another problem is that there’s no separate GUI that shows all of your tests as green or red for pass or fail. If a test fails, the build fails. To find out which one failed, you have to drill down to the specific test in the build results window. It’s hard to do Red, Green, Refactor without red or green. There are third-party GUIs such as OCRunner that are doing their best to usher OCUnit into the mid-aughts by providing a nice GUI. I haven’t been able to get them working yet though (maybe a future post).

Finally, OCUnit requires specific naming conventions for test classes and methods. I admit that this is more of a style choice than a genuine shortcoming. In some other unit test packages such as NUnit for .NET, classes and methods are identified as test cases in metadata (NUnit uses attributes.) With OCUnit however, all test class names must end with “TestCase” and each test method must begin with “test” in order for them to be recognized by OCUnit. This may seem like just the choice of convention over configuration, but it’s really convention as configuration. Instead of making things easier and more intuitive, it hides information about the behavior of a class or method in its name.

What’s Right With OCUnit?

Ok, now you’re about to leave a comment “If you think OCUnit sucks so much, don’t use it!”, but that’s not true. I don’t think OCUnit is that bad. It gets the job done and could grow into the testing framework that we all want it to be. It does have some things going for it too.

When you’re choosing tools, a real concern is that development on a particular tool will stagnate and you’ll be left in the cold with outdated tools that you can’t migrate from. OCUnit is bundled with Xcode, so development is probably going to continue for the foreseeable future. Who knows, maybe Apple already has developers dedicated to improving it. Or writing a unit testing framework that’s compatible with it.

Also, despite my ranting, once you get OCUnit running and know its quirks, it’s pretty easy to use. Its shortcomings only really cost you time when setting it up and learning how to use it. Most of the problems I’ve mentioned above don’t have recurring costs.

Conveniently, you can minimize that initial cost of setting up and learning OCUnit by reading the rest of this post. It will not only help you survive the wasteland, but thrive with Unit Testing in the Objective-C world.

Add a new testing target

The first thing we need to do is add a new Unit Test Bundle target to an existing project that we want to unit test. Right click on the Targets section of the project file list and click Add>New Target.

Target Aqcuired.

Choose Unit Test Bundle and click Next.

Bundle Up!

Usually people name it HotPotatoTests where HotPotato is the name of their project. I don’t think the naming affects anything though, so feel free name it a cuss word if you like and click Finish.

I recommend naming it something safe for work if you're at work.

Add a new group for your test files

We’re probably going to end up writing at least one test class per functionality class, so the project’s going to start to get messy. Let’s just nip that clutter in the bud right now and create a group called Test into which we will put all of our test case files. Right click your project’s name in the Groups & Files pane, and go to Add>New Group. You should probably not name this a cuss word, you’re just going to confuse yourself.

Add a new Test Case file

Now it’s time to add a new testing class. Right click your newly created Test group and go to Add>New File… Choose Objective-C Test Case Class from the Cocoa Class section and hit Next.

This next part is VERY IMPORTANT. It is the first of two things that are absolutely vital to saving yourself frustration. Every test case class in OCUnit must end with the suffix TestCase. A nice convention is to name it HotPotatoTestCase where HotPotato is the name of the class under test. That way it’s stupid easy to find the tests for that class. Don’t forget to add TestCase to the end though. Promise me you won’t forget. I’m serious, pinky swear in the comments.

Once we have a test case class, the first thing we should do is write a failing test and run it to make sure it fails. Add the following code to the .m file of the test case class:

-(void) testThatTestingIsWorking

This is the second VERY IMPORTANT part. It is again crucial to your sanity that you remember to do the following exactly: You must name all test methods so that they begin with the word test. Don’t forget! Oh please don’t forget.

Run your failing test

Now click that little unlabeled drop down in the upper left corner of the main Xcode window and switch the Active Target to your test target.

The next part is optional, but you probably want it. Expand the Targets section of your project. Now click and drag your executable target onto your test target. This will make your executable a dependency of your test target. Now, every time you attempt to build the test target, it will build the executable first. This ensures that the code under test always reflects the current code and not just whatever is lying around from the last time you built your executable.

Now comes the fun part, running the test! Just go to Build>Build and your test should run. The first time I did this, I was like “oh shit there are build errors, I’m doing this tutorial wrong.” Don’t worry, you’re not doing it wrong. Well you may be doing it wrong, but you should be expecting build errors here because a test failure counts as a build error. We need to dig a little to see exactly what the build error is though.

Finding the Failure

First bring up the Build Results window by doing command+shift+b. At this point, we could simply do a full expansion on the line where you see the errors by clicking the little button. But that gets you a big long log file that sucks to parse with your eyes. We can get a slightly more readable list of results by expanding Run custom shell script ‘Run Script’ > Run unit tests for architecture > Run test suite <octest path>. Here you will see a list of test case classes with an exclamation point if they’ve failed. You can further expand that to see the actual test methods that failed, and then drill down to the actual failure.

Ok, I guess it is red.

It’s annoying, but you’ll get used to it. Because you have to.

Test your actual code from your actual project

Most of the tutorials I’ve encountered stop here without telling you how to reference the code in your executable from your test case classes. It’s not intuitive but it’s just something you have to learn to do. I don’t know why they usually exclude this part, maybe the authors didn’t actually try to unit test anything significant. We live in the real world though, so we need to actually reference our actual code.

The first thing we’ll do is go to the Groups & Files pane and drag the code files (just the .m files) under test into the Compile Sources folder inside your test target. This is the magic that tells Xcode that you’re going to need access to the code from these files when you build the test target. So, if you’re testing a class called StringBasedTime, you grab StringBasedTime.m and drop them into the folder under Targets>HotPotatoTests>Compile Sources (where HotPotatoTests is the name of your test target.)

Cuss words everywhere!

Then in the test case file that you’re using to test this code (StringBasedTimeTest.m,) you simply add a directive to import the header file:

#import "StringBasedTime.h"

And that’s it, you can reference your executable class in your test code!

Example Unit Test

Here’s a simple unit test that I wrote in real life for a real reason. I’m using it to test the initializer of a class I’m building called StringBasedTime. StringBasedTime takes an argument in it’s initalizer that is a string representation of a time interval. The string representation is based on the output of QTStringFromTime, which is a method in QTKit, Cocoa’s QuickTime framework. It converts a QTime to an NSString in the format dd:hh:mm:ss.ff/ts. For example, @”0:00:00:00.00/00″ is the very beginning of a QuickTime movie. The test we’re looking at just makes sure that if someone initializes a new StringBasedTime with @”0:00:00:00.00/00″, each of the properties gets populated with zero. If one doesn’t, I want the failure output to tell the tester which property is non-zero. Here’s the code from the test case’s .m file:

#import "StringBasedTimeTestCase.h"
#import "StringBasedTime.h"

@implementation StringBasedTimeTestCase

-(void) testInitWithStringZeroTime
	NSString *ZeroTimeString = @"0:00:00:00.00/00";
	StringBasedTime *ZeroTime =
		[[StringBasedTime alloc] initWithString:ZeroTimeString];

	NSString* FailText = @"";

	if([ZeroTime days] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d days when 0 expected."
					, [ZeroTime days]];

	if([ZeroTime hours] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d hours when 0 expected."
					, [ZeroTime hours]];

	if([ZeroTime minutes] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d minutes when 0 expected."
					, [ZeroTime minutes]];

	if([ZeroTime seconds] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d seconds when 0 expected."
					, [ZeroTime seconds]];

	if([ZeroTime timescale] != 0)
		FailText = [FailText stringByAppendingFormat:
					@"%d timescale when 0 expected."
					, [ZeroTime timescale]];

	STAssertEquals(@"", FailText, FailText);



Remember, NSLog is your friend when debugging. I know printf debugging sucks, but sometimes it’s your best option. It might be possible to get the debugger working with OCUnit, but I haven’t been able to crack that code. If you figure it out, please feel free to leave a comment!

Further Reading

I figured out how to do all of this from a few good tutorials on OCUnit (and a few bad ones!):

Ridin’ Dirty With your MySQL Database Backups

I switched credit cards a while back and forgot to change the autopay settings for my Dreamhost account so they turned off my hosting. During the recovery process, I lost a few (terrible and uninteresting) blog entries. Even though it was my fault, it’s left me twice shy about leaving my data integrity in someone else’s hands. For most of my data, I’m running Mac OS X on my laptop and use Time Machine to back up to an external drive. I needed to find a way to get my WordPress database onto the Time Machine train. I know this blog is starting out a little Dreamhost heavy, but this should apply to anyone wanting to backup a remote MySql database.


MySQL comes bundled with an awesome utility called MySQLDataDump. You throw it a database, user name and password, and it outputs a SQL script that creates all the tables and inserts all the data. First, you have to install MySQL locally though. Finding the right download for MySQL is a little like finding the installer for Java was back in the day. They try to hide it to make you accidentally pay money for it or something. I’ve gone ahead and sussed out the link for you though. After you install MySQL locally, it’s really easy to get a dump of your database from Terminal (This should all be on one line, but I didn’t want to make you have to scroll to the right forever so I broke it up a little):

	--password=AReallyStrongPassword WordPressDatabaseName

Once the terminal window stops scrolling, we can talk about piping this output to a nice text file and zipping it up, but first a Dreamhost digression: In order to access a MySQL database remotely, you need to add the host to the Allowable Hosts list on their web panel. Don’t believe me? Check out the wiki. I’m not sure what I’m going to do when my IP address changes, but for now, I just went to Goodies>MySQL Databases on the web panel, then clicked on the user that I wanted to have remote access, then added my current IP address to the list of Allowable Hosts.

Put The Output of MySQLDataDump Into a File

I knew I would probably want to schedule this backup with a cron job later, so I wanted to put a date stamp in the filename that we pipe the output of MySQLDataDump to. Fortunately, Unix’s date does exactly what we need it to (again, this was put onto multiple lines for readability, the real command should be just a single line).

	--password=AReallyStrongPassword WordPressDatabaseName
	~/Documents/Projects/ Database Backup/
	$(date +%m%d%Y)

That’s nice… real nice. All this crap is going to start filling up the drive though, so let’s zip it up

	--password=AReallyStrongPassword WordPressDatabaseName
	| gzip >
	~/Documents/Projects/ Database Backup/
	$(date +%m%d%Y)

Make sure you name the output file so that when you unzip, it has the .sql extension. Who knows, we might run this script in Windows some day.

Schedule it wih Crontab

I had never used crontab before this, though I knew it existed. It’s a little idiosyncratic, but easy to pick up. You can always use a third-party task scheduler like Lingon, but I find cron jobs to be way simpler to use.

In order to schedule a task, you need to add a line to crontab’s configuration text file. The line format is split into two sections, the first is five numbers separated by spaces indicating the minute, hour, day of the month, month, and day of the week that you want the job to run. You can substitute an asterisk (*) for any of these numbers to tell cron to execute the job every hour, minute, etc. The second section (after another space) is the actual command you want to execute. Here’s a nice little diagram that I stole from this site:

*     *     *   *    *        command to be executed
-     -     -   -    -
|     |     |   |    |
|     |     |   |    +----- day of week (0 - 6) (Sunday=0)
|     |     |   +------- month (1 - 12)
|     |     +--------- day of        month (1 - 31)
|     +----------- hour (0 - 23)
+------------- min (0 - 59)

How do you get to this config file though? just type

crontab -e

and you get a nice VI window with the file opened. If you’re confused or just don’t want to think right now, here is the line I inserted to tell crontab to execute the data dump every day at noon:

0 12 * * * /usr/local/mysql/bin/mysqldump
	| gzip >
	~/Documents/Projects/ Database Backup/
	$(date +%m%d%Y)

This should all be on one line though, here it is for copying:

0 12 * * * /usr/local/mysql/bin/mysqldump -h --user=TheBestHuman --password=AReallyStrongPassword WordPressDatabaseName | gzip > ~/Documents/Projects/ Database Backup/ $(date +%m%d%Y)

Just paste that line into the crontab config file and your MySQL(YourSQL?) database is on the Time Machine Train!

"Your data's safe with me, Marty!"