yield thought

it's not as hard as you think

Make Exceptions

with 10 comments

I used to think removing special cases – the extra if statement that handles the unusual file format that doesn’t quite fit – was the best way to improve my programming. Every time I had to add a special case to a function it meant I wasn’t solving the problem generally enough.

In a way, which I’ll come to in a moment, this was ok. In a much more important way, it was really-and-I-mean-shoot-yourself-in-the-foot-and-throw-the-remains-into-a-vat-of-industrial-strength-paint-stripper stupidly wrong.

What made it right was my innocence – back when I knew nothing about programming (i.e. when I thought I knew everything) longer-lived programs started accruing odd if-statements and shoe-horned additional functionality all over the place until they were an ugly, barnacled mess of additions and changes that I didn’t understand and couldn’t work with any more. Every time I promised myself that next time I’d resist the temptation to add just one little exception and instead refactor to a more general level right away and everything would be clean and smooth and regular and easy to work with forever.

I guess lots of people have reached the same conclusion, because I notice examples of the unsaid assumption that Special Cases Are Harmful a lot. Today, I’m taking a stand: adding special cases – making exceptions – is the single most important thing we can do.

It’s All Fun And Games Until Someone Loses An AbstractEyeMonsterFactoryFactory Class

I wrote lots of little games when I was learning to program and I’m guessing you did, too. I made them because I wanted to play them, and that meant I tried to generate levels and enemies randomly in the hope that my own game could surprise me. I didn’t want to add special code for boss monsters or for each level – that would defeat the point. Every level had to run the same code, just with different data. No special cases, remember?

Well, all of my games sucked. Sometimes the mechanics were fun enough, but after a couple of levels the game just got boring. Algorithmically-generated enemies and levels simply weren’t interesting, because after a while the patterns became obvious and then there was no reason to keep exploring deeper into the game; one level was much the same as any other.

I’m sure this is stupidly clear to anyone who actually designs games for a living or works in the industry, but it was a revelation to me that the most interesting and valuable parts of a game are the parts that show a human touch, where I could feel the presence of a real person who put some thought into the experience. While ‘special cases’ looked like cruft at the level of an individual algorithm, they turn out to be the essential, core content and personality at the user-facing level. I’d been systematically stripping my work of any personal, human touch whatsoever.

But, seriously

In real applications, whether games or business sites, special cases add a massively disproportionate amount of value.

Posterous know this – they’re laboriously and painstakingly adding beautiful import functionality from each of the other major blogging platforms. Each one is a special case, they didn’t restrict themselves to one common ‘import blog’ feature that scans a page and does its best to rip the text, formatting and images – no, they give users from each of the major platforms special, individual treatment.

Google were the first to do this in a big way in search: how cool is it when Google converts a currency right there are the top of the page? Or shows the weather forecast? Or the cinema listings? Each of these are laboriously hand-coded special cases. I once met a guy at Google whose full-time job was working on showing sports results at the top of mobile search queries. That’s dedication to the special case.

There’s a couple of reasons why looking at special cases are often better than adhering doggedly to an increasingly complex general case:

  1. Special cases are solvable. We can be very, very clever when we’re only handling a small, discrete part of a problem. The general ‘show the most relevant information to a search query in an immediately readable and usable way’ is so hard it’s still unsolved despite decades of attempts. Just look at Wolfram Alpha. In comparison, pulling up the current forecast for queries containing a local word for ‘weather’ is almost trivial.
  2. The results are better. The general ‘import blog’ problem is hard to get exactly right, but we can write code to import just wordpress blogs perfectly because the problem domain is both smaller and better defined.
  3. Whatever we think about solving the general case is almost certainly wrong until we’ve solved a few special cases first. The real world is not a well-defined problem; special cases help us explore the problem space.

I’m tired of hearing that ‘if you solve a problem right, you only have to solve it once’. Yes, this is fine in theory, but is completely bonkers when applied to the changable, unpredictable real world. If we try then most of the time we:

  1. Never find a perfect ‘general’ solution for all cases, or
  2. Find one but spend so much time and effort on it that we neglect a million other important things, or
  3. Get ‘close enough’ and just stop, which means a solution that’s imperfect in most cases, and by ‘imperfect in most cases’ I mean ‘that makes most people slightly unhappy’.

Look After The Pennies And The Pounds Will Look After Themselves

You know what? When we try so hard to make each special case just right, general cases will start to fall out naturally. This is the right way around. The other way, hypothesizing a general case and enforcing it onto everyone, that is the wrong way.

Look after the special cases and the general case will look after itself (well, unless you’re unusually incompetent)
— Mark, Make Exceptions

Adding super-slick handling for a few common cases is such ridiculously low-hanging fruit that I can’t believe so many companies miss the opportunity. I think it’s this pattern of reasoning we learn when we begin programming – this desire to avoid messy details and refactor to a more general level that handles all those cases implicitly. It turns out that this simply doesn’t apply well to product design and, if we want anyone to use the programs we’re writing, we should always be thinking about product design.

So yes: we can make exceptions for people and go out of our way to make them smile. People are trying to accomplish real tasks with our applications, our websites, our businesses. We have to keep looking out for next special case we can handle that makes just one of those tasks improbably and awesomely simple. Software’s not about ticking the most feature boxes with the fewest function calls, it’s about making someone’s day.

Note: Yield Thought has moved to http://yieldthought.com – check there for the latest posts!

Written by coderoom

July 29, 2010 at 2:36 pm

Posted in Uncategorized

10 Responses

Subscribe to comments with RSS.

  1. nice post 🙂

    I think you are right that niche treatment can be a very good thing but we must not swing between extremes – there is no binary choice between having special cases and not having special cases!

    I think the point is that many but not all special cases are warts – they really should be eliminated and a clean high level design introduced. Most often you then collect the code warts as a data file that drives your general model – so you have nice uniform code and all of the messy specifics in a data file. Overall the complexity of understanding your system has just dropped immensely 🙂

    Andrew

    July 29, 2010 at 6:28 pm

  2. This is amazingly good advice. It’s sort of a generalization of my personal distaste for the modern “figure out what patterns seem to match, and beat the problem so that it fits” method of developing software. I’ve long held that patterns are a result rather than a target or a goal; they emerge from your solutions but you can’t any more define what patterns you’ll use in your software up front than you can predict the weather. Well, unless you want software that’s hard to use, change, and extend.

    Everything is a special case until it isn’t. Proper development shows us that file handling and XML processing can be generalized. In the case of Posterus, it is quite likely that they have common code between all of their importers, but if they “did it right” they only discovered the common bits after they’d implemented a small number of importers. Even so, not all of the importers necessarily need all of the common behavior.

    I really enjoy reading your posts. Thanks!

    Tom Plunket

    July 29, 2010 at 11:02 pm

  3. I think you still have it the wrong end of the stick.

    The fundamental problem is failing to think, and to keep thinking, for the entire time you’re developing something.

    Overgeneralization and early generalization are indeed significant problems, and they are often a result of someone trying to write perfect, elegant, generic code from the get-go. You don’t know enough about the problem space to find the correct generalization except by chance, and where you *do* get it right you won’t know, because you don’t know what the tradeoffs are that make that the *right* generalization.

    You end up with complex code that nobody really understands — your clever “AbstractEyeMonsterFactoryFactory” reference hits this dead on — nor would any sane person want to use.

    So I agree with that much of your thesis.

    The other extreme — to give up on generalization and to embrace lots of little special cases everywhere — is just as bad. That road leads to the same place: impenetrable, opaque, fragile code that’s difficult to comprehend and harder to maintain.

    You touched on the solution: *refactor*.

    Solve the problem in front of you. Keep an eye on the big problem, and try to avoid doing anything that will shoot you in the foot later. When you’ve got some code *working* (and tested, mind you), _refactor_ that code into something better. Trying something out? Go ahead and put in that if-statement. If that’s the last thing you do in that piece of the code, fine, let it be. If you keep revisiting the code to add more if-statements, then you’re working with some code that could stand some generalization — so put down the keyboard, let go of the mouse, and sit back and *think* for a bit.

    Then grab your engineering notebook (you keep a notebook, right?) and *write* *down* the problem, and at least two approaches to solving (generalizing) it, and the tradeoffs for each approach.

    Refactoring lets you keep your *working* *behavior* in place, while you fix the organizational problems in the code. Throwing away working behavior (or never going after the low-hanging fruit) is bad, but so is turning the code into the sort of mess where if you get hit by a bus it would be easier to start over. Both extremes are failures.

    SJS

    July 29, 2010 at 11:20 pm

    • This is all true and would’ve made a great blog post in its own right; I was actually talking about something more subtle though – the impact that programming habits have on our design decisions. If Google implemented weather snippets using if cases or Strategy patterns or a sorting monkey doesn’t matter: they implemented it as a specific case rather than trying to solve the general ‘show a useful snippet for any search query’ problem directly.

      coderoom

      July 30, 2010 at 10:52 am

  4. This reminds me of YAGNI (You Ain’t Gonna Need It).

    You probably don’t need that AbstractInformerStrategy class, but just a switch statement. If the need arises later, you can always extract it AFTER you know a few specific cases.

    Nice Post!

    Pablo

    July 30, 2010 at 2:15 am

  5. “””
    While ‘special cases’ looked like cruft at the level of an individual algorithm, they turn out to be the essential, core content and personality at the user-facing level.
    “””

    I think this reflects the fact that humans naturally tend toward complexity; It requires discipline to prevent this. Compare the city planning of Levittown with that of Boston, MA: One seems devoid of humanity, the other seems vibrant and lively.

    But it is a lot less scary and stressful to drive through Levittown. Simplicity, even austere simplicity, still makes thought easier.

    Andrew Farrell`

    July 30, 2010 at 12:22 pm

  6. Nice article.

    Agree with Pablo, this is YAGNI and Pragmatism by a more controversial title.

    It’s easier to produce great solutions to specific problems by coding in the small (by making an exception). In my experience, the trick is to do so cleanly and with test coverage, so that when more general problems become apparent the code is easier to refactor to incorporate those general solutions.

    Elwyn

    July 30, 2010 at 12:31 pm

  7. True, but not a simple guideline.

    Terry Davis

    July 30, 2010 at 11:29 pm

  8. […] Make Exceptions I used to think removing special cases – the extra if statement that handles the unusual file format that […] […]

  9. “we should always be thinking about product design”

    Bingo – This should be the guiding light, special cases or not.

    James

    August 1, 2010 at 6:24 pm


Leave a reply to Andrew Cancel reply