Screening Programmers Via Email

There is lots of good articles out there on the web about how to interview programmers. Here’s one. Most of this understandably focuses on the interview process whether its a phone screen or and in person all day event. At some places the process essentially stretches out for a 90 day trial period.

I don’t have much new to say about any of that. I will just say that when you have a bad feeling during the interview, you should probably pay close attention to it. If you aren’t sure you can always interview them again and see if it goes away.

The part of the process I want to write about today is the part where you get (hopefully) lots of resumes and you have to decide which ones are worth a phone call. You are going to need the phone call or in person interview at some point but these take time. How can you best decide which ones to interview from the resumes you get? If you only get a few this is not a problem, but how do you sort through dozens?

Sure enough, a resume contains a lot of information and there are good and bad ways to read it. Again I won’t go into details other than to say that I like this comic.  There is however a problem. The resume leaves out a lot of things which may help you decide. This is especially true if the candidate is non traditional in any way at all (age, work experience, education etc). I’ve had bad hires with degrees from MIT and great hires who were still in High school. There has to be a way to look a little deeper before deciding on who makes it to the next round since in the end:

You are looking for someone who can write good code not a good resume.

At a previous job we started using a series of questions to get a more complete picture of the candidate before talking to them. The trick is these questions aren’t about programming knowledge per se since someone could just Google the answers.

What they do is provide a way to encourage the candidate to tell us more about them so we can make a better decision. They can also function as a kind of programmer personality test. Over time, the questions have changed but the basic concept remains the same. You send out a list of these questions to any of the candidates you think might remotely fit and then scan the responses to decide who to interview.

Here’s an example:

What programming language do you wish you knew better and why?

How someone answers this usually can tell you alot about how they view their skillset. People love to list on their resume a dozen languages that they have written 5 lines of code in, but what they want to learn tells you alot about how they think about themselves and the future. Like all of these questions there is no one right answer you are just hoping they have some kind of positive interesting answer. If they answer something like: “I already know all the important languages” or “I have been thinking of learning X cause, there are like lots of jobs and stuff for that” then you may have a problem.

Another one:

Have you had problems working with others or a boss before? How did you deal with it?

Personality matters for a lot. This question is pretty good at weeding out those people who just seem so unlucky that every place they worked was toxic. These people never suspect that they are the poison but  a question like this will usually give you hints.

There are more but that’s the basic idea. These kinds of question may not pass muster with your HR department but they will help you get better insight into the candidate before the phone interview and give you some additional things to talk about besides “Tell me about your job @ X”.

Its much more common for these questions to cause someone to stand out from the crowd in a positive way than for them to get rejected based on the results. A resume doesn’t really let people explain what their passions are and when you try to do it a cover letter it can come off as kind of icky. In this kind of informal survey of questions a candidate can feel free to give a long answer if its needed.

One of my best hires every was a fresh from school programmer with no real work experience, nothing impressive on his resume or cover letter, etc. Probably not worth the phone call but sending out the list of questions via email is easy so– In one of his answers to the email survey he explained about his experience with a product he bought and how he didn’t like the software that came with it. So he wrote his own and then released it as open source and was managing a small community of people.

Ding!

Another Story About How Fred Brooks is Smarter than Me

Maybe I could do a whole book, but I guess he already has.

Back when I was in gradual school Fred Brooks was the adviser for something called the building walk-through project. The idea was pretty crazy. You would model a building in a computer before it was built and “walk” around in it using 3d graphics. The system had a map view which was 2d overview and a first person 3d view which was generated by Pixel Planes 4. At some point there was a treadmill hooked up where you could really walk around and turn using  a bicycle handlebar setup. Oh yeah. And there was a full axis HMD. I’m bad with dates but I think it was around ’88-’89.

Some interesting stuff came out of this project. One that I recall was the relationship between navigation mode and frame rate. If the frame rate dropped below around 10fps the user would stop looking at the 3d display and revert to the 2d overview for navigation.

Anyways, Dr. Brooks was always preaching the idea that building walls were mostly flat and vertical and the floors were mostly flat and horizontal. Somehow there must be a way to take advantage of this when you render to get better performance.

I think I speak for me and a few other cocky grad students in saying our response to this was something like:

Look old man, back in your day you might have been hot stuff with your 360 machine, but nowadays its floating point matrices all the way down. How can what you say  matter? Besides what if the user tilts her head? Sheesh.

As far as I know none of us thought much about this problem.

A few years later when I was out in the “real world” someone brought in this game called “DOOM”. We were all blown away how this little 486 was doing perspective corrected 3d textured graphics so fast, it was almost comparable to the SGI’s we had. We thought and thought about how it could be doing this. We had no idea for quite some time. Then after looking at some other demos etc it clicked,

…perspective transform, constant , verticals, horizontals…

FRED!!!!!!!!!!!

Gradual School

A previous post got this comment:

You were in a gradual school?

I am guessing the commenter was “helping” me with my spelling which I do need in  general but in this case I was referring to:

Walt: Daddy what’s gradual school?
T. S. Garp: What?
Walt: Gradual school. Mommy say’s she teaches at gradual school.
T. S. Garp: Oh Gradual school is where you go to school and you gradually find out you don’t want to go to school anymore.

I learned many things at gradual school some of them are useful and some of them were learned in my CompSci classes.

Actually there were some good  things I learned in class, but none are more applicable to the job of being a programmer than this:

Never subtract two almost equal numbers and multiply the result by a really big number

Thanks Dr. Pizer.

This is without question the single most valuable programming thing I learned in class during the undisclosed amount of time I spent at UNC. If you remember this and can juggle clubs in pairs you got much of the value of a MS in CS.

Some random thoughts on programming

Since this post actually got read by people I made a follow up to clarify some of what I was saying. Thanks internet for helping me realize how bad I am at making my point.

As I near the end of my third decade programming, I keep finding myself coming back to pondering the same kind of questions.

  • Why are some people good at this and some are not?
  • Can the latter be trained to be the former?
  • What is the role of language, development environment, methodology, etc. in improving productivity?

I’ve always been partial to the “hacker”. The person who can just flat out code. For some people this seems to come easy. I have been lucky enough to work with a few people like this and it has been awesome. As you talk with them about a problem they are already breaking it down and have an idea of how the final system will work. I don’t mean that other aspects of professionalism, experience, etc. aren’t important its just that I believe in the end there is a raw kind of talent that you can’t teach. My wife can sing, I can’t, and not for lack of practice.  She practices, but she can also belt out a perfect tune without hardly trying.

What are the core talents that makes someone a good hacker? I think there are two:

  1. The ability to break down an idea/system into an abstract set of “concepts and actions”
  2. The ability to understand how these “concepts and actions” execute in a particular environment.

Note that I didn’t say anything about coding. Coding is actually the least interesting part. If you can do the others well, coding is just another kind of typing. Coding is the thing your fingers do while your mind is flipping between 1) and 2). Learning a new language is less about learning the syntax than it is about learning the execution model.

Writing good, correct code involves not just conceiving of how something should be 1) but also understanding all that can happen 2).

I honestly don’t know if these things can be taught.

In 30 years I have only seen a few examples where a mediocre programmer became good or a good one became great and zero examples of a mediocre programmer becoming great.

What does this mean for people who want to build large software systems? The easy answer is to only hire great people. Which is fine when you can do it but not everyone can be or hire the best so what is left for the majority?

What should be done to make things easier for people aren’t in the top 5%?

There are lots of nice things that have been invented like source code control, higher level languages, garbage collection, etc. which reduce or abstract away some of the problems. And while sometimes these abstractions are leaky, they are really one of the few weapons we have against the beast called Complexity. Complexity is the key problem we face, especially if we are not wizards because:

A persons talent with regard to 1)  and 2) above determine how much complexity they can deal with.

When I was in gradual school at UNC-CH Fred Brooks (the mythical man month guy) used to talk about Complexity as being a major thing which makes programming hard. Its not something easily cast aside:
Quoting Dr Brooks:

The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence.

I wish I had appreciated everything Fred said when I was a cocky grad student as much as I appreciate it today… sigh.

Some things are hard because they are hard and not much can be done about them. Still our goal needs to be to not try to make it worse than it is.

To this end, some people say:

Do the simplest thing that will work.

I agree but prefer this formulation:

Strive for the simplest thing that can’t possibly fail.

If you think about what can go wrong you are more likely to find the problems than if you think about what will go right.

I wish I could say we were making progress in reducing complexity but we seem to be adding it into our programming environments rather than removing it. One of the things I have thought about recently is the complexity of these abstractions themselves. Programming languages of today are so much more abstracted and complex than what I learned on (assembler, basic, pascal, C).

For example the world (or at least the web) is currently engaged in writing or rewriting everything in Javascript. Not because JS is so awesome, but because 15 years ago it got stuck into browsers, then we all got stuck with browsers and thus we are all stuck with JS. Sure there’s lots of server stuff done in various other languages but really how long can that last? How long can one justify having to write 2 parts of an application in 2 different languages?

The future looks like node.js.

I am not a JS hater. Its just that I used to think Javascript was a simple language. As someone who wrote 3D Game engine code in C++, my thought was “if these webkids can do JS then it can’t be that complicated”. After building a webapp using Javascript/Dojo/svg to create a realtime browser for genetic information, I realize how wrong I was.

Sure its all nice and simple when you are doing this:

$(“#imhungry”).click(function() { alert(“food”);});

However a lot of complexity lurks just under hood. And its the way it all interacts that concerns me. Crockford called it  Lisp in C’s clothing. This should excite a few people and scare the crap out of the rest.

JS has lots of opportunity for expressing yourself in interesting ways. Expressiveness is good, but too much of it can create more problems than it solves. Back when C++ was a bunch of source code you downloaded from at&t and you had to compile the complier yourself before you could use it, we grad students stayed up late eating bad fried chicken and debating the merits of O-O programming. It was at this point I began to wonder if the productivity gains of O-O were more the offset by the time spent debating about what good O-O practice is. This tape has played in my head more than a few times in my career.

Its not all bad of course: On balance, garbage collection is a win and I see lots of benefits to using closures in JS, but the underlying complexities of things escape most people and their interactions are understood by even fewer. If you have to slog through this (and you should) to understand what happens when you call a function, you are cutting out alot of people from fully understanding what is going on when thier code executes.

I hope that some of this will be address by standards of practice. Things like Google’s guidelines seem to be a good start.  As with C++, I think that the best hope for improving productivity will actually be to agree on limits on how we use these tools rather than seeing how far we can push the envelope.

Well except for wizards and we know who we are 🙂

YAMSR

I recently switch to using a Mac for my main computer. I decided I didn’t want to move to vista and while I love ubuntu I don’t really want to use an os which still requires me to edit files in /etc to hook to a secure wifi access point. I do have a gutsy box I play around with and enjoy and there are several windows XP machines still in the house but we are definitely moving in the direction of Cupertino here at home.

I got a mac book pro and I have to say its the nicest piece of PC hardware I have ever owned. Its quiet, well built, and has lots of nice little touches that make working on it very satisfying. Most of the differences were expected. Its not like I have never used a mac before but some things suprised me both good and bad.

Unix like os underneath – By far the biggest positive surprise is how much this has helped my productivity. The shell in windows has always sucked and having the option of doing things with a nice gui or nice shell has really made things easy. Having all the linux tools like rsync etc available can be done on the pc via cygwin etc but it always seemed like a kludge whereas this is much nicer

Parallels – This suprised me in some good and some bad ways. First It seems to work very well even in Coherence mode where its liking having a PC and a Mac on the same desktop. I wanted parallels for testing with things like IE which don’t really exist on the mac. I figured I could just run the server for the Web app I’m making on the mac and then use IE/Paralles to talk to it as localhost. I haven’t been able to get that to actually work but it hasn’t really been a problem. Most of the time I just remote desktop (using a MS provided native Mac app) to a pc and do all the work there. I find I use Parallels alot less than I thought I would.

Firefox – sucks really bad on Mac. This one really surprised me. The Windows version of FF running under Parallels totally rules compared to the Mac version running native on the Mac. Camino is maybe a little better and you get shiny buttons but its still slow.

Safari 3.0 (beta)- Is really fast. I like it and if I ever get a mac with 10GB of ram I will like it even more.

Preview.app – starts up so much faster than Acrobat that somedays its enough reason to buy a mac all by itself.

Adium – I figured I would be using iChat on the Mac but some friends turned me onto Adium and I have never looked back. Maybe I’ll check out the leopard version of iChat but Adium is really pretty sweet. I don’t miss trillian at all.

Expose/Dashboard – I use and like these more than I thought I would.

Tethering with a Windows Mobile phone – Works sweet.

Mouse Buttons – While I still think its idiotic that macs ship PRO laptops with 1 button, it hasn’t been as much of a pain as I thought. The two fingers touching the trackpad while you click thing is pretty easy to get used to and works fine. I’m already finding it bugs me when PC laptops don’t work this way.

Control Keys – The placement of Ctrl-Option-Apple etc are pretty different and I’m still in transition here. Overall thought I think I’m pretty used to it. The first week or so was pretty rough. I have never been an expert key user and use the mouse for lots of things that most experts would do with key commands.

I’ll probably try out leopard this weekend but from the little time I spent with it at the store so far I’m underwhelmed. The big selling point for me is the development enviroment changes which proport to include better python support (yay!).

Testing Faith

At 40+ I consider myself an “old” programmer although it feels better to just think of myself as “old-school”. Its an interesting ride because I go back and forth between feeling like I have seen all the things come around 100 times and thinking there is so much new stuff I will never be able to keep up. For example, I have seen people make the mistake of thinking portability has something to do with programming languages instead of realizing its about platforms probably 100 times, but spending too much working on videogames made me miss the whole foundation of web programming and I still feel behind the curve.

One thing that seems to be new in recent years is the religious fervor about testing. Before I say more, I should point out that I think testing is good. More testing is better etc. etc. Testing isn’t new but this view of unit test suites as a panacea for all ills does seem to be new and in my view potentially dangerous.

I first starting thinking about this when I started reading all the discussion about how Unit Testing is better than strong typing. You know how it goes. Strong typing is for small minded cretins who can’t even trust themselves, and besides what matters is having good unit tests. Type checking is nothing more than a pathetically weak test suite. Oh well um whatever. The correct answer about typing is to make it optional. Python is moving this direction. Its a shame that C++ choose to instead inflict templates on us all but none of this has anything to do with unit testing. We need to test programs because people make mistakes even if they are using a language that has closures, lambdas, and currying.

The problem with obsessing about these unit test scripts isn’t that they are bad, its that they are not good enough. No matter how good your suite is there is are things it won’t do for you and you better make sure you realize that or you are never going to write code that is as good as it should be. Testing is not a substitute for some other technique. Its not an alternative to Typing. Its not a new way to do design. It’s not a definition of overall quality. Most of all its not a substitute for THINKING. Its just a way to make sure that none of the things you might have expected to go wrong did. You should make your tests as good as you can, but if you start believing that its good enough, then you are doomed.

Its worth pointing out of course that the value of testing is greater in some projects than others. If you are working on a library with a pretty clean API then a good suite of unit tests can give you a pretty high level of confidence whether or not your code will produce correct results. As things scale up to larger projects with complex, stateful interfaces, this level of confidence will get more elusive since the test harness can become as complex as the program its testing. I’m not sure how to write a test that verifies that my algorithm to find the shortest path always finds this path. I don’t know how many cases is enough and just having 100% coverage isn’t going to help me.

Another good application for testing is when a software project must under go some major changes. Having a good suite of regression tests is a nice way to make sure you haven’t gone too far off the map but its not a real test of equivalence.

Thinking
Let’s say you just finished making some non-trivial changes in your local copy of the software you are working on. There’s alot of things you could do before committing these changes. You can:

  • Sit and think hard about what you did, trying to imagine in your mind what the impact of your changes will be on the rest of the system
  • Do a little local testing by hand to see if things are going like you thought
  • Look over every line of diffs to make sure the code you think you are going to commit is what you are going to commit
  • Try to think objectively about whether or not this is the best way to do what you are doing
  • Run a unit test suite.

The correct answer is all of the above. If you think that your test suite lets you off the hook from any of the other steps you are wrong. Btw the steps which are just thinking should be done both BEFORE and AFTER you write your code. Even though this process of thinking is imperfect (if we were perfect then we wouldn’t need tests) its still important because it helps you become smarter about the code and helps insure that the code itself doesn’t become too crappy.

TDD
I have tried hard to understand this concept of “write the tests first” as a way to do design. I have followed a few examples on the web of people demonstrating this. All of them suffer from what I call the “factorial effect”. Often when someone is showing off a new programming language they will show the cool way you can write some simple program like computing a factorial in this language. The problem of course is that its dead simple to write factorial in any language so all we have proved is that the new language makes easy things easy. Woot! For a complex system writing the tests seems to me about as hard as writing the system so how about we just do the latter?

I know that most people write these tests as they go and build up the library of tests little by little. My guess is that 90% of the time they are sort of cheating on the idea a bit, maybe even alot as their deadlines approach. But what really worries me is that I’m dead sure there is some large software project being developed right now where a team of programmers is spending months writing tests for a system they haven’t started developing. While such a project would likely end up as a huge mass of Fail, it would however succeed in taking an idea which started as “agile” and turning it into a pathetic kind of Waterfall model. The Test becomes the Spec but any Spec is almost always doomed to become less valuable than the thing its trying to specify. What the program does is reality. What the spec says or what you think the tests validate is a dream.

My other problem with this idea is that coding to the tests seems about as useful as teaching to the test does in the school system. Right now someone is probably putting some ugly if statement in their code “so the damn tests will pass” and that just makes me cry.

Quality
As I said above I don’t think tests are bad its just that they aren’t every going to be a perfect measure of correctness and so we better not make the mistake of thinking they are. However even a Perfect Test Suite (PTS) could not guarantee anything which would be worthy of the word Quality. Even if it would find and report every possible false output it still doesn’t say that much about how good the code is. I don’t mean whether the resulting application is good in the sense that iPhoto is “better” than Picasa or whatever, thats a whole other question. I’m just speaking about software quality. Passing the tests is only step 1 on the way to having good software quality. I’m actually scared there are kids out there picking up some idea that all programs capable of passing PTS are equally good. Um, no.

This would only be true if you work in the sales or marketing dept. since to these people the software is a black box, they only care that it ships on time and it works. That sounds good except that the program is not a black box. If you are programmer its more like an organism. It lives and changes and interacts with a changing environment. If you are the person in charge of maintaining it, its an organism you have to live with, so it better be good and easy to get along with on the inside as well as looking good on the outside.

Overall I just worry that focusing on the test suite is actually setting the bar far too low. Thinking your code is good because it passes the tests is like thinking your life is good because you have HDTV.

Croquet

Lots of new announcments from the Croquet People. A Company a consortium and the 1.0 SDK release.

Its nice to see they are still alive and kicking. It would be even nicer if they hired some decent artists.  There are some really smart and talented people behind Croquet but so far thier screenshots actually make second hype look good.

The Tasks of Software Development

I recently started a new programming project. For me its new in a few different ways. Its the first job I have had with or for a local company in about 3 years, Its the first project I have worked on (for money) that was small enough for me to complete by myself really in maybe 10 years, and its the first serious Web development project I have done ever. I’m using Django and so far I’m mostly loving it. I would say 95% of my troubles have more to do with either me not undertanding some web technology, or me just not being able to believe that some web technology sucks as bad as it does.

All of this has gotten me to think more about what this thing we call programming actually involves. What does it take to go from the client’s needs to a working product? In short, what do they pay me for? I think there are 3 main areas of work that go on in my head or get commited to my subversion repository.

  • Knowing what you want to make
  • Construction of an abstract Model
  • Translating this Model into code

These three tasks are the major parts of any software project and are the tasks we sign up for as programmers. Some things to point out about the list. While it does seem like the tasks are in order that you need to accomplish them the truth of course is that often you iterate through these steps a few times. Its not something to fear but something to prepare for. Deciding everything up front is overrated and unless you have a boss/client who can never make up thier mind you will converge on something better if you start out thinking flexibly. If you do have a boss/client who can never make up your mind a detailed spec won’t help you in the end anyway.

Also its good to realize that moving down the list moves you into lower and lower value territory. If you are only good at “writing code” then you have basically limited your ability to make money. People who can work higher up in the list are less likely to have thier jobs sent to India and more likely to be able work as independents or entreprenuers.

Knowing what you want to make is maybe the most squishy to define of the three. Depending on the type of project you are doing it could mean different things and its very hard to know if you have done it correctly which is a big reason why you have to be open to iteration of the whole process. If your project is to write a program to convert jpg files into png files then you have a pretty good chance of nailing this part right off. You might even be able to write the tests first etc. If instead your project is to build a social network platform which is tailored to astronomers who want to share imagery and other observational data, you might have to spend some time on this before you even know where to start. You might think of this part as Design and you might think that this work is to be done by Designers and its the programmers job to wait till they are done and pick it up from there. It might be like that where you work but if it is then I’m glad that I don’t work with you.

Construction of an Abstract Model involves working out (in your head, on paper, on a whiteboard, or anyway you can) the basic concepts or building blocks your application will contain and how they will work together. How it is going to solve the problem. At this point you should be thinking about which things are going to be easy and which ones are going to be hard and what you are going to do if you are wrong about either of those assumptions. This too is essentially a design task but one that requires a more logical and frankly conservative approach. Over design in this stage can often be worse that over-design knowing what you want to make stage because sometimes its easy to just cut a few features than it is to rework your whole platform.

Translating this Model into code should be the easiest part but there are often difficulties in translating from the Abstract Model based on an inability to adequately describe this model to the people doing the actual coding. This is the key reason why small teams are so effective because usually the people who design the Model also code it up so nothing is lost. Even if you were diligent about specifying the model, its VERY hard to capture intention behind the model so you are stuck with either something under specified or over rigid. Your best hope is lots of clear open communication, preferably with food.

We are almost ready to talk about actually writing code but first I’m going to divide this task into its own little hierarchy of skills etc which are needed to do this well. Starting with the hardest and working our way down the list to the easiest.

  • Coding so it must work
  • Learning the platform
  • Mindless Typing

Coding so it must work means being able to determine the simplest most direct way in code to express what you need to do. Good programming means thinking about how your program might fail and not just how it will work. If there is an Art to coding it lies here in the ability of some people to produce clean, beautiful code while others seem to be throwing statements into the editor like some C++ version of magnetic poetry. Think hard about how your code can fail make sure you know what will happen in every case. No matter how smart you are, this is hard so your best option is try to keep the number of code paths (including exceptions) down. Less is almost always more. Simple pure expression of an idea can still make me smile as I read through code.

Learning the platform might not be considered programming per se but its what programmers spend a fair amount of time doing these days. It used to be that all you needed to know was libc, but for my current project I need to understand HTTP, django, DOM, reportLab, CSS, python, MySql and Javascript. The growth of the importance of platform over the last 20 years has led to these long lists of skills on resumes and job postings. I don’t put much stock in those lists. I have a few predjudices against people who have never used c, c++ or a simlar bare metal kind of language but all in all I think most people who can work well with one platform can learn another. They will be slower at first but they will eventually get back to thier normal speed. Almost everyday I learn some new thing about django or Javascript that makes me more productive and opens up new possibilities.

Mindless Typing is the lowest level of programming and one which has been the most helped with tools like macro processors, IDE, etc. What I mean by this is that some parts of programming are just basically brain dead entering characters into the keyboard for something you have probably done 100 times before. It could also just be something repetitive you need to do because your environment requires you to repeat things or you are refactoring some simple operation but you need to change it 10 places and you couldn’t make a good macro for it. The DRY principle and modern dynamic languages have reduced alot of the need for this and more and more tools have become available. I’m more productive in python than C++ even though I know C++ way better. I know that some of it is due to garbage collection and a better built in library but alot if just comes down to the fact that I don’t have to do so much typing.

The lowest form of Mindless Typing is trial and error. This is what you are down to when you are just typing in different numbers in your CSS file hoping something will make the damn image line up the same in both FireFox and Internet Explorer. Not that I am ever be reduced to doing something like that…