yield thought

it's not as hard as you think

Code for Flexibility: A Manifesto

with 12 comments

Different environments give rise to different programming styles. Over the last few years there’s been a massive trend for software to move from:

Work for a whole year then release a fixed binary, then write for another year.

to the more dynamic, web 2.0 ideal of:

Whip up a web application and upload each feature as soon as it’s done.

This is a much more efficient way to do business, but it’s a very different kind of programming environment and benefits from a very different programming style – not the one we’ve all picked up by copying API conventions and old-fashioned corporate practices.

If you’re able to make your software available to customers instantly, there is one optimal strategy: code for flexibility.

The flexibility of your code is defined by the ease with which you can modify it to fulfill some purpose you hadn’t envisaged at the time you wrote it.

This is important, because when you give your customers a rapidly-iterating product you’re going to find your initial guess at what they wanted was completely wrong. In fact, all your guesses are probably wrong. You’re going to spend years learning from your customers and adapting your software in all sorts of unexpected ways to fit their needs as precisely as possible.

For a long time I confused generality with flexibility; I was always worrying about how I might want to use a class in the future and trying to abstract out all the common use cases right form the start. As it turns out, this is the opposite of flexibility; this builds rigidity into the project. The moment you realise that actually it makes much more sense to have flip your design so that users vote for *each other’s* questions a little part of you dies inside, because you now have five levels of abstraction and four different database tables to rewrite.

In fact, flexible code is specific code. We want to write code that expresses, as succinctly as possible, a solution to a problem. It’s quicker to read and understand code with fewer syntactic elements, such as variables, functions and classes. It’s also quicker to repurpose.

We also build flexibility – or rigidity – into the large-scale structure of our programs. I’ve often fallen in love with the idea of encapsulation for its own sake; I thought that each part of my program should be as well-hidden from the rest as possible, with only one or two well-specified interfaces connecting things. This is a very tempting dream that results in a kind of tree-like structure of classes. This is extremely rigid, because by design it’s difficult to get access to classes in another branch of the tree.

The best way to build for flexibility is drawn from the Lisp world and championed by Paul Graham: when working on a problem, write code that builds up into a language for describing and reasoning about the problem succinctly.

You can magnify the effect of a powerful language by using a style called bottom-up programming, where you write programs in multiple layers, the lower ones acting as programming languages for those above.

This way the solution ends up being a clear, readable algorithm written in this language – the language of the problem domain. In non-lisp languages you end up with a set of functions and classes that make it easy to reason about the problem, a little like the standard string and mail classes make it easy to work with strings and email.

Building up code for reasoning about the problem domain is vital for flexibility, because although our solution to the problem might change drastically as we get extra insights from our customers, the domain of the problem will probably only change incrementally as we expand into new areas. Adopting this flatter, domain-language appropach to program design almost always increases flexibility.

Another aspect of flexibility is robustness. A function or class that makes assumptions about the circumstances it is called in is going to break as soon as we repurpose it in some other part of the code; sooner or later I always forget that the caller needs to check the file exists, or that the user isn’t null and already has a validated address. There are two schools of thought on how to deal with this – contract programming (essentially visible, machine-verified preconditions) and defensive programming (make no assumptions, handle all the exceptions you can locally and try hard not to destroy any persistent state if things go wrong). It doesn’t really matter which you use, but use one of them.

Many of the extreme programming principles help us write flexible code – the c2 team recognized that:

Change is the only constant, so optimize for change.

They looked at this from a project management perspective, but code guidelines like Don’t Repeat Yourself and You Ain’t Gonna Need It are well-recognized as being fundamental to producing flexible code. In fact, these two oft-quoted principles are worth looking at more closely in terms of how they create flexibility.

Don’t Repeat Yourself: When modifying a bit of code it doesn’t help if there are three or four undocumented places it also needs to be changed; I never remember them all. A perfectly good way to deal with this is to add a comment in each block mentioning the link rather than to refactor right away. Over-zealous refactoring tends to merge lots of bits of code together that look kind of similar but ultimately end up going in different directions. At this point very few people take the logical step of splitting a class or subsystem back into two more specialized units; instead it’s over-generalized again and again until it’s completely unmaintainable

Refactoring should always be Just In Time, because the later you leave it the better you understand the problem; refactor too soon and you’ll combine things in a way that restricts you later, building rigidity into the system.

You Ain’t Gonna Need It: This doesn’t just apply to extra functionality. It applies to all elements of your code, from accessors (a waste of time in most cases) to overall class structure. You’re not going to need to handle N kinds of book loan, you’re going to handle at most two, so don’t write some complicated code to handle the general case. You’re not going to need three classes, one interface and two event types for your “please wait” animation because there’s only ever going to be one and it’s always going to be while printing. Just write a function with a callback and release – the simpler it is, the quicker we can write it today and the more easily we can modify it tomorrow.

There is a tension here. On the one hand, we want to write our software by constructing recombinable blocks that represent the problem domain, yet on the other we want to write the simplest thing possible – and writing reusable blocks of code isn’t usually the simplest thing possible.

This tension is resolved when we stop looking at a program as a snapshot of its state at any one time and instead look at it as an evolving codebase with many possible futures. So we write the simplest thing possible, yet while doing so we keep an eye on its potential for being refactored into nice, reusable blocks and avoid doing making poor choices that will make that process difficult. The potential for refactoring is just as valuable as the refactoring itself, but by deferring the refactoring we keep our options open.

The ability to see the code you’re writing today and its possible evolution through time is something you can focus on learning, but it also grows with experience. It’s one of the things that makes a great hacker’s code subtly better than a newbie’s code – both start writing simple code that addresses the problem directly, but the newbie makes all sorts of small structural mistakes that get in the way of the code’s growth, whereas the hacker knows when it’s time to turn this set of functions into a class, or to abstract out this behaviour, and has already prepared the way for doing so. The effect is to make programming look effortless again, naturally flowing from the simple into the complex.

I’m not a great hacker who does this as naturally as breathing. I have tasted it, have seen glimpses of it in my own work and in other people’s, but I still make mistakes; I mistime my refactoring – sometimes too early, sometimes too late. I still bind objects to each other too tightly here, or too loosely there. Despite my failings, just aiming at this goal makes me far more productive than I’ve ever been before. It’s not an all-or-nothing premise. The closer you come, the smoother and lower-friction your development will be.

We can gather all these heuristics together into a concise manifesto:

A Manifesto for Flexibility

  1. Write new code quickly, as if you’re holding your breath. Cut corners and get it in front of users as quickly as possible.
  2. Keep your code specific, with a clear purpose. Don’t over-generalize and don’t refactor too early – it should be as simple as possible, but not simpler.
  3. Stay aware that all code is thrown away or changed dramatically. Hold the image of the refactoring you’d like to do in your mind and avoid doing things that will make it more difficult – this is slightly better than doing the refactoring now.
  4. Recognize the point at which simple code becomes messy code and refactor just enough to keep the message clear. Just In Time refactoring is not the same as no refactoring.
  5. Build up a language for reasoning about your domain and express your application in that language. Avoid building up a large, rigid hierarchy of over-encapsulated classes that embody your particular solution to the problem.

Postscript: You should do this in a startup and you should do it in personal project, but you shouldn’t do it everywhere. If, say, you’re writing the Java or .NET API then none this applies to you because:

  1. Millions of people will abuse your API in every way possible the very second it’s released.
  2. Every time you change something a howling mob will descend upon your office, making it difficult to get a good parking spot.
Advertisements

Written by coderoom

April 16, 2010 at 12:37 pm

12 Responses

Subscribe to comments with RSS.

  1. […] haven’t become great yet. What might the next step be? I don’t know. Reading more code? Writing less code? Perhaps. But at the moment I have a feeling that to grow in another dimension – creating […]

  2. Great article, definitely worth reading =)

    Koen Metsu

    April 23, 2010 at 11:59 am

  3. You write about duplicate code: “A perfectly good way to deal with this is to add a comment in each block mentioning the link rather than to refactor right away.” I’m sure this is definitely a worse way to deal with this problem. If you follow this approach, you’ll have to add links for each duplicated piece of code for each other instance of it, so information will be multiplied much more times. And what’s more, comments are not in the model, so these links will be even more difficult to maintain. So I think you should reconsider this method…

    thSoft

    April 25, 2010 at 8:51 pm

    • If you have to duplicate a fragment of code once, then sometimes that’s too early to refactor – perhaps you won’t keep it or one copy will need to change independently soon enough anyway. In these cases a little comment in both snippets is perfect. If you copy it again then it really does need to be refactored; there’s no argument to support 3-way or N-way comment links!

      coderoom

      April 26, 2010 at 6:12 am

      • Personally, I tell people I’m working with:

        1st time, write it.
        2nd time, grimace and copy+paste.
        3rd time, refactor.

        I probably stole this from someone else, but I have no idea who.

        Xiong Chiamiov

        August 2, 2010 at 10:51 pm

  4. […] – coderoom […]

  5. […] Lean startups and the Minimum Viable Product are all about starting in the middle. Paul Graham’s advice for startups can be summed up as ‘first solve the interesting part of the problem, then build the business around it’, but the process is also fractal – starting in the middle applies right down to the level of writing a new class, or a single function. First write some code that solves the problem even if it’s imperfect or partial, then expand it out with your favourite blend of accessors, inheritance and polymorphism (or don’t). […]

  6. You have opened my miiind!

    These principles seem like a really good basis for writing code for research too.
    Too often research code just ends up being poor quality code for the very reasons outlined here (at least, mine does).
    Rigidity leads inevitably to hideous workarounds when you have to “modify it to fulfill some purpose you hadn’t envisaged at the time you wrote it”.

    Thanks, I can’t wait to apply this.

    Chase Gunman

    May 19, 2010 at 9:36 pm

  7. “Build up a language for reasoning about your domain and express your application in that language.”

    Would love to see this concept expanded on more thoroughly.

    Justen Robertson

    June 23, 2010 at 8:33 pm

    • This is explained somewhat in Uncle Bobs book Clean Code. It suggests that you should write at the correct level of abstraction for each level of the program. So you would use technical terms in your high level general methods, they would be tucked away inside a lower level. That way you can write code that reads quite like a natural language and hide away the actual technical process of doing it further down inside the tree. I think that’s what is being hinted at here…?

      rtpHarry

      June 24, 2010 at 11:35 pm

  8. […] O conselho de Paul Graham para iniciar projetos pode ser resumido como “primeiro resolva a parte interessante de um problema, então contrua o negócio em volta dele”, mas o processo é também fracionado – iniciando no meio, você desce ao nível de escrever uma nova class, ou uma função. Primeiro escreva códigos que resolvam o problema, mesmo que imperfeito ou parcialmente, então continue com seus acessórios, herarquia e tudo mais. (Nota: nem pense em se preocupar a não ser que você se odeie). […]


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: