Category Archives: programming

Django CRUD and Angular.js

In the post on Simple Django Class based CRUD views the focus was on setting up simple views with ModelForms to build all the CRUD views for a Model. I have been playing around a bit with Angular.js and wanted to see if I could link the two up.

A word of caution. I really don’t have any idea what I am doing in Angular.js land so take this all with a grain of salt.

Angular seems like a kind of retro-future solution to the javascript application problem. Backbone.js seems to want to get away from the DOM as anything other than presentation. In Backbone the models exist in a pure JS land and the views render them out into the DOM as needed when things change by subscribing to events on the model. Angular in contrast, embraces the DOM more, turning <input> elements into models and providing updates via some dependency injection magic.

Angular also lets you essentially extend html by letting you add your own element and attribute types using directives. But we are getting ahead of ourselves.

Every Angular tutorial shows some example like this:

 <input type="text" ng-model="MyModel"/>
{{ MyModel }}

In this case angular will dynamically update the page by reprocessing the stuff inside {{}} each time the user types something new into the model. Neat.

Another thing Angular can do is provide validation of fields. This works well with html5 <input> types like number and email. If the input element has an ng-model attribute, angular will add some css classes ng-valid or ng-invalid in response to user input.

Which got me thinking. Can I link up Django ModelForms on the server side with all this Angular stuff on the client? The idea is adding some functionality into the Django forms so that they render in way that angular will take advantage of.

If you build the forms by hand in Django you can basically set the widget attributes to whatever you like, but if you want this to happen more automatically when using ModelForms you are probably going to want to check out formfield_callback. In each model form you can add a new formfield_callback function that will determine how django converts the fields in the model to fields in the form. Unfortunately, it doesn’t seem like you can inherit this function from your own ModelForm base class. You have to add it into each class definition. Cry.

class MyForm(forms.ModelForm):
    formfield_callback = angular_formfield_callback
    class Meta:
        model = MyModel

Where angular_formfield_callback is the function we are going to write.

Lets think first about the <input> types. By default django doesn’t seem to do much here. Even if your model has a field of type PositiveIntegerField, the element django gives by default is just type=’text’. So lets change this by defining our own widget for positive integers.
Here we set the input type and also force the min attribute to be zero since this is a PositiveNumber field:

class PositiveIntWidget(TextInput):
    input_type = 'number'
    def __init__(self, *args, **kwargs):
        if not 'attrs' in kwargs:
            kwargs['attrs'] = dict(min=0)
        else:
            kwargs['attrs'].setdefault('min', 0)
        super(PositiveIntWidget, self).__init__(*args, **kwargs)

Then we would define our form field callback something like:
def angular_formfield_callback(f, **kwargs):
    if isinstance(f, PositiveIntegerField):
        field = f.formfield(widget = PositiveIntWidget(),**kwargs)
    else:
        # We don't have a special one just let django pick for us
        field = f.formfield(**kwargs)
    if field: # guard against the case where there is no field...
        if 'ng-model' not in field.widget.attrs:
            # set up a model for this so ng can do stuff with it
            # the dj_ prefix helps avoid name collision
            field.widget.attrs['ng-model'] = 'dj_%s' % f.name

Now when we render the form from django in our django template using {{form}} it will be decorated with the ng-model tag with its value set to the field name of the django model field.

But there is still a problem. Angular really wants the initial values of the fields to be stored in the $scope object. When you use the technique above by itself and you are trying to have it work for updating existing data, the values which django sets as the value attritbute in the <input> element don’t seem to get copied into Angular’s scope.

I tried a few ways around this but ended up writing my own directive for this. The directive just copies the value attribute into the scope using the element’s model name.

var app = angular.module('dj_app', []);
app.directive('djInit', function() {
            return function(scope, element) {
                initial_value = element.val();
                if(element[0].type == 'number')
                {
                    initial_value = parseInt(initial_value);
                }
                scope[element[0].attributes['ng-model'].value] = 
                      initial_value;
            }

That junk with type==’number’ makes it so the scope value is actually a numeric instead of string value.

Now we just need to add the dj-init attribute to all the field objects in the form field callback at the same spot where we set the model. For fun we can also copy over some info from Validators on the original model field and we get:

def angular_formfield_callback(f, **kwargs):
    """
    Applies our own assignment of widgets when using model forms
    """
    if isinstance(f, PositiveIntegerField):
        field = f.formfield(widget = PositiveIntWidget(),**kwargs)
    else:
        # We don't have a special one just let django pick for us
        field = f.formfield(**kwargs)

    if field:  # guard against missing field
        for v in f.validators: # Copy any model validation into the widget
            if isinstance(v, MaxValueValidator):
                field.widget.attrs['max'] = v.limit_value
            elif isinstance(v, MinValueValidator):
                field.widget.attrs['min'] = v.limit_value

        if 'ng-model' not in field.widget.attrs:
            # set up a model for this so ng can do stuff with it
            field.widget.attrs['ng-model'] = 'dj_%s' % f.name
            # set the model name as an attribute of the 
            # init directive in case we want to know it later
            field.widget.attrs['dj_init'] = 
                           field.__class__.__name__.lower()

    return field

Its easy enough to play the same trick with other data types like email or url.

Things to watch out for

Both django and Angular.js use {{ for macro expansion. {% verbatim %} {% endverbatim %} is your friend.

An Angular directive called fooBar is referenced in the markup as foo-bar.

Simple Django Class based CRUD views

Class based CRUD views in django have been around for a while, but I haven’t been using them because every time I tried I ended up with more code that seemed no better than the old way. I just started a small project from scratch and decided to give it a go again.

The basic idea is pretty simple. There are default views like DetailView, ListView, UpdateView etc. you can build your views from. Then in urls.py you just say MyDerivedView.as_view() for the view function.

So before it might have been:

url (
    regex = '^detail/(?P<pk>\d+)/$',
    view = 'scrud.view.detail_view',
    name = 'scrud_detail'
),

Now its

url (
    regex = '^detail/(?P<pk>\d+)/$',
    view = ScrudDetailView.as_view(),
    name = 'scrud_detail'
),

The trick comes in how we define these views. The defaults are fine if you aren’t doing anything fancy. There are lots of nice parameters to the base classes you can modify as shown in the docs.

urlpatterns = patterns('',
    (r'^publishers/$', ListView.as_view(
        model=Publisher,
        context_object_name="publisher_list",
    )),
)

Personally, I don’t like gumming up my urls.py with that stuff so I go ahead and make a views.py and define new classes there.

Either way lets try to see how this works for a simple model.

class Scrud(models.Model):
    foo = models.IntegerField(default=0)
    bar = models.CharField(max_length=20)
    owned_by = models.ForeignKey(User, blank=True)
    last_change = models.DateTimeField(auto_now=True)
 
    def get_absolute_url(self):
        return reverse('scrud_detail', args=[self.pk])
 
    def __unicode__(self):
        return u"Foo:%d Bar:%s  Owner:%s Date:%s" %
        (self.foo, self.bar, self.owned_by, self.last_change)

The get_absoulte_url() method is going to be what some of the views use by default.

To make things easier lets just create a Mixin class we can use for all the views we want.

class ScrudMixin(object):
    model = Scrud
    def get_success_url(self):
        return reverse('scrud_list')
    def get_queryset(self):
        return Scrud.objects.filter(owned_by=self.request.user)

This is nice because it will limit the queryset to objects owned by the user as well as set up the defaults needed by each of the generic CRUD (and List) views.

We can force login by changing ScrudMixin(object) to ScrudMixin(LoginRequiredMixin) and add something like:

from django.contrib.auth.decorators import login_required
from django.utils.decorators import method_decorator
 
class LoginRequiredMixin(object):
    @method_decorator(login_required)
    def dispatch(self, *args, **kwargs):
        return super(LoginRequiredMixin, self).dispatch(*args, **kwargs)

Now we can define views like:

class ScrudListView(ScrudMixin, ListView):
    pass
class ScrudDetailView(ScrudMixin, DetailView):
    pass
class ScrudCreateView(ScrudMixin, CreateView):
    pass
class ScrudDeleteView(ScrudMixin, DeleteView):
    pass
class ScrudUpdateView(ScrudMixin, UpdateView):
    pass

and use them in urls.py like:

urlpatterns = patterns('',
       url (
           regex = '^list/$',
           view =  ScrudListView.as_view(),
           name = 'scrud_list'
       ),
 
       url (
           regex = '^detail/(?P\d+)/$',
           view =  ScrudDetailView.as_view(),
           name = 'scrud_detail'
       ),
       url (
           regex = '^create/$',
           view =  ScrudCreateView.as_view(),
           name = 'scrud_create'
       ),
 
       url (
           regex = '^delete/(?P\d+)/$',
           view =  ScrudDeleteView.as_view(),
           name = 'scrud_delete'
       ),
       url (
           regex = '^update/(?P\d+)/$',
           view =  ScrudUpdateView.as_view(),
           name = 'scrud_update'
       ),
)

Oh yeah. We are going to need some templates. You can override the names if you like but you’ll need something like:
scrud_list.html

{% for object in object_list %}
  <a href="{{ object.get_absolute_url }}">{{ object }}</a>
{% endfor %}

scrud_detail.html

{{ object }}

scrud_form.html

<form action="." method="POST">{{ form }}{% csrf_token %}
   <input type="submit" value="Go" />
</form>

scrud_confirm_delete.html

<h1>Delete This?</h1>
{{ object }}
<form action="." method="post">{% csrf_token %}
    <input type="submit" value="Delete" />
</form>

Now there should be a fully working crud system for your object. Read some hackernews to celebrate.

When you come back you will realize that you probably want something a little different. Like for example. Maybe you want the owned_by field to be set automatically based on the user. You don’t want it in the default form. No problem you can override the form with your own custom model form.

class ScrudForm(forms.ModelForm):
    class Meta:
        model = Scrud
        exclude = ('owned_by')

The top of your ScrudMixin is now looking like:

class ScrudMixin(LoginRequiredMixin):
    model = Scrud
    form_class = ScrudForm

Which takes care of not letting the user change the owner but doesn’t actually set it. For that you need to add one last thing your ScrudMixin class, a form_valid override. This changes what happens when the form is valid so that the field can be set correctly.

from django.views.generic.edit import ModelFormMixin
def form_valid(self, form):
    # save but don't commit the model form
    self.object = form.save(commit=False)
    # set the owner to be the current user
    self.object.owned_by = self.request.user
    #
    # Here you can make any other adjustments to the model
    #
    self.object.save()
    # ok now call the base class and we are done.
    return super(ModelFormMixin, self).form_valid(form)

I hope this at least helps some people get started using class based CRUD views. Better yet, I hope someone will tell me an even better way.

Flattening the Backbone.js Learning Curve

I just spent some time getting into Backbone.js finally.  The learning curve was a little steeper than I expected, mostly because I think I expected Backbone.js to be something that it is not.

From the docs:

Backbone.js gives structure to web applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions,views with declarative event handling, and connects it all to your existing API over a RESTful JSON interface.

Well not exactly. Backbone.js doesn’t give structure. It gives you the tools to create structure.  A minor nitpick maybe, but it makes a difference when you are trying to understand how something works and you can’t find the parts you expect to be there.

The docs are well written, the examples numerous, and there are also several blog posts which try to help you get started. Oh, and there is the totally awesome annotated source.

Still there were a few things I wish I had understood before I dove into a tutorial.  This post is my attempt to try to help boost someone up the learning curve a bit. The idea is not to get into many details or app design philosophy.  Not being an MVC ideologue myself, I care little about how things ought to be.  I am just trying to get to the essence of what the parts actually are. I’m also skipping some things on purpose, as I said the docs are well written.

Models/Collections

Key Idea 1: Wrapper for a hash of properties and that can trigger events when the properties change

Key Idea 2: Includes code to load and store from a Rest API

Models and Collections are the data management objects in Backbone.js.  The Model is really a wrapper for state for an object. Each model object has a field “.attributes” which is a map of the data managed by this model.  Since you change the data using .set methods of the model, this allows the model to trigger events when the data changes:

A Collection is a collection/list of models. Duh.

Both Collections and Models are able to fetch themselves from a server if you have a nice API.  Just set .url for the object foo and call foo.fetch().  An ajax call will be made to the server and the data will be loaded or added (you can chose with an option). You can also save a model to the server. And you can do lots more, including overriding virtually all of this.

Views

Key Idea: Wrapper for a DOM object which renders it based on events and other state

Views are what tie into the DOM. Each view attaches to a DOM object. You can set this to one in your document or the View will create a new one ( a <div> by default) when you create the View object.  It’s your job to make sure the element is in the page in the right place in the hierarchy. The View always keeps a reference to the DOM element in .el or .$el if you prefer the JQuery wrapped version.

A view generally has an associated model and listens for changes in that model,  re-rendering the DOM objects when a change occurs. The view subscribes to events on the model and calls the views render method to update the DOM objects based on the new data. Generally this is done via a client side template mapping the Model’s attributes into HTML.

Routes

Key Idea: Custom events or state which work with browser history

Routes allow you to define new URLs within a single page application that can trigger events in your app. You create a router object which includes a bunch of defined url routes which are pretty similar to the urls.py in django actually. The routes/urls have parameters which are passed to the callback function when that state is reached.

Calling router.navigate() will set a new state which allow the user to move around using forward and back in the browser. This is done via #tags which you give as names to the routes.

Don’t forget to call Backbone.history.start() before using Routes.

 


 


Coder’s Block

I have tried before to write about what makes programming hard. I’m not really happy with that post or the followup. I still believe that Complexity in all forms is the biggest problem thought its not the only one. Sometimes designing an algorithm can be really challenging even for something which feels pretty simple. Same goes for things like performance optimization. It can be pretty difficult to properly analyze a system to determine what parts are slowing it down even if the system (or at least the part of if you have any control over) is pretty small.

But even beyond all that, much of the time it seems that the really hard thing about building Software isn’t writing the code, it’s deciding what code to write.  Its one reason we see so many projects which amount to “X now written in Y”. The objective and scope are clear and easily understood. Lets fire up vim and have at it!

Currently I’m building a large software product, hoping to turn it into money in the future. I don’t have specific tasks or deadlines besides the ones I set for myself and I sometimes fall victim to Coder’s block. Writer’s block isn’t an inability to type – it is a failure to be able to decide what to type next. Coder’s Block is similar.  When I am stuck (aka. hitting Command-R on news.ycombinator.com) it is rarely because I am faced with something I don’t know how to do. It is way more likely that I just can’t decide which of three different methods I could use to do it.

One way I deal with this is I try to keep two different Todo lists. One is the hard decisions I still have to make about how the software should work (Ex. what is the best representation for a huge list of genomic variants). The other is just a list of things I can knock out whenever I am stuck on the first list (Ex. Fix the report CSS styles so they don’t look like crap).  When I was a manager, I tried to encourage this habit among the coders who worked for me, but It mostly came out like I was trying to make them double up on work. Um sorry.

Another technique is to just code your way out of it. This works pretty well for small to medium size blockages. The other day I was stuck trying to decide what kind of reporting system I wanted to build. I wasn’t getting anywhere in my head so I just starting using python to think with.  I think I have written it about 3 different ways in the past week but now I am pretty happy with the result.  It certainly is easier to play around with options and keep transforming your code using new technologies than It was trying to do this with a C++ app using Boost and other crap.

Sometimes it is hard to get up the desire to use either approach. That’s when it’s time to step away from the keyboard and go for a bike ride.  Sure you could sit there and refresh hacker news, but it will be there when you get back.

Accessing the Child Class In Django Multi-Table Inheritance

Django has 3 different ways to deal with model inheritance:

The differences are described pretty well in the linked documentation so I am not going to recapitulate them here. I just want to talk about a specific pattern I have been playing with for Multi-table inheritance.

In Multi-table inheritance the base class has its own separate table. This makes it possible to query all the objects from all the subclasses at once.

So if we have:

class Animal(models.Model):
    def speak(self):
        print "Generic Animal Sound"
 
class Dog(Animal):
    def speak(self):
        print "Woof!"
 
class Cat(Animal):
    def speak(self):
        print "Meow!"

The Animals.objects.all() will find all the Cats and Dogs in the DB. The trouble is they will all be of type Animal and so if you run:

Dog.objects.create()
Dog.objects.create()
Cat.objects.create()
 
for animal in Animal.objects.all():
    animal.speak()

you get:

Generic Animal Sound
Generic Animal Sound
Generic Animal Sound

There is a way to access the child class specific information. animal.dog.speak() will produce Woof! if animal is actually a Dog. It will however toss an exception if it is a Cat. Worse is that testing this out generates some useless Db queries which becomes even more impractical as the number of subclasses increases. One approach is to store the name of the child class with the base class so you only have the one extra query needed to grab the data for the child object. If we create an Abstract base class like:

class KnowsChild(models.Model):
    # Make a place to store the class name of the child
    _my_subclass = models.CharField(max_length=200) 
 
    class Meta:
        abstract = True
 
    def as_child(self):
        return getattr(self, self._my_subclass)
 
    def save(self, *args, **kwargs):
        # save what kind we are.
        self._my_subclass = self.__class__.__name__.lower() 
        super(KnowsChild, self).save(*args, **kwargs)

And derive Animal from it.

class Animal(KnowsChild):
    def speak(self):
        print "Generic Animal Sound"

Then we can run:
Dog.objects.create()
Dog.objects.create()
Cat.objects.create()
for animal in Animal.objects.all():
    animal.as_child().speak()

and get:
Woof!
Woof!
Meow!

I am still honestly not sure if I like this or not but its working out well so far for a reporting system I am making where ReportItems can be one of several different types but sometimes I just want to work with the elements in the base class. A lot depends on how often you need to know the specifics of the child class.

No Virgina, there is no such thing as Write Once Run Anywhere

I think the first time I heard “Write Once Run Anywhere” was the uscd-p system pascal.

From Wikipedia:

UCSD p-System achieved machine independence by defining a virtual machine,

UCSD p-System began around 1974 as the idea of UCSD’s Kenneth Bowles[1], who believed that the number of new computing platforms coming out at the time would make it difficult for new programming languages to gain acceptance.

Ok I was 10 in 1974 and probably wasn’t paying attention right at the start but its safe to say the idea has been around a long time. But read the article and weep about the fact that 37 years later we are still chasing (or hiding from) the same paper tiger.

Being cross platform is easy until you try to actually touch something about the platform. JNI anyone?

Its not just about companies trying to embrace and extend. Its just a hard problem and its everywhere.
Here’s a fun little PySide snippet adapted from some code which bit me yesterday.

from PySide.QtCore import QSettings
settings = QSettings('foo','bar')
settings.setValue('key',False)
a=settings.value('key')
if a:
    print "Windows"
else:
    print "Mac"

So this will print Windows on Win7 and Mac on Snow Leopard. The reason is apparent if you add this line:
print a, type(a)

On Win7 you get:
false <type 'unicode'>
On the Mac:
False <type 'bool'>

Ok so maybe you think the Mac is correct and the Win7 version of PySide has a bug. Doesn’t matter. The worst part is you can “fix” this by changing:
settings.setValue('key',False)
to
settings.setValue('key',int(False))  # or a var which is <type 'bool'>; with a value of False

And it all works. The windows registry (in PySide’s world at least) seems to understand Ints but converts bools to strings. Yay!

Of course we can all think of other examples. The point is just to remember that there is no cross platform development environment and there is unlikely ever to be one.

For a while people wanted to think that the Web was the answer. You know make some cool web app and if you obey the standards it will work for everyone without any special hacks. Ha. Good one.

With current trends in Virtual Machines, I think Write Once Run Anywhere might happen because we will just ship the App as a complete Virtual Machine and the OS will just be something like VMWare with a switcher. Then the switcher will get more and more complicated and there will be competing companies making them. Each will add their own features to differentiate themselves until…

Django Logging Setup

Post 1.3, Django has some nice integration with the python logging module. It can dictConf to specify the formatters, handlers, etc. This is all pretty well documented here.

What wasn’t immediately obvious (to me anyway) was that you can use any of the handlers in django logging and pass the __init__ params to them in this config. So for example if you want to have a log which rotates everyday and keep a week of these files, then you can simply define:

'handlers': {
        # Other handlers here
        'rotating_file':
        {
            'level' : 'DEBUG',
            'formatter' : 'verbose', # from the django doc example
            'class' : 'logging.handlers.TimedRotatingFileHandler',
            'filename' :   MY_LOG_FILENAME, # full path works
            'when' : 'midnight',
            'interval' : 1,
            'backupCount' : 7,
        },


and then use this handler for a logger:

'loggers': {
         # other loggers
         'my_logger': {
            'handlers': ['rotating_file'],
            'level': 'DEBUG',
        }
    }

You can use this logger in your app by grabbing the logger:

logger = logging.getLogger('my_logger')

And log away!

logger.warn("Here be Dragons!")

PySide + PyInstaller = Joy

I have a small project to make a pretty simple windows GUI executable. In the past I have always used wxpython + py2exe for this kind of thing but decided to try something new.

After some poking around I settled on PySide which is the bindings for QT from Nokia. Its nice. Seems a little cleaner but actually pretty similar to wxPython in many ways.

At first I tried to use py2exe to make a single exe binary but I could never get this to work properly. The exe would always die somewhere during load and after spending too much time trying to sort that out I decided to use cx_freeze to much the same result. I also tried going back to Python 2.5 since that seems to have fewer MS DLL issues and PySide is available for that as well. Still no love.

Finally I tried Py-installer and it all pretty much worked right out of the box. Just go through the install step by step and you should be fine. Py-installer integrates with Upx and my final exe came in at around 6MB.

If you are stuck with having to deploy an app on windows, you don’t need anything more that python, pyside and pyinstaller. As a bonus the app will also run on your real computer.

Email only Signup

One of the things that bugs me about the web is having to create logins everywhere. There are lots of attempts to deal with this problem (OpenID, Facebook Connect, 1Password) etc. but they all seem to have problem or require me to do things like:

  • Use only sites that implement them
  • Copy and paste stuff on my phone
  • Spend Money
  • Trust Mark Zuckerberg

Many of these websites are just some new thing I want to try out.  I’m not talking about sites which require high security such as bank accounts or email (if you use the same password for you email account and other random web accounts, stop reading and change it right now!).

I am talking about this weeks random new social/mashup/mobile/fu thing your sister just txt’ed you and said “Try this out, Its Awesome!”. These sites are all trying to get as many people to sign up as possible so the make the barrier to entry a low as possible and signup typically requires only a email and a password. I’m starting to think that is one thing to many. I don’t think most of these sites need to ask me for a password, they just need my email address.

How do I know this?
Since almost of these sites let me reset my password via email then my email address (and the ability to read it) is evidentially good enough authentication.

Ok yeah, I know what you are thinking, great more spam in my email, more steps to login etc. But the thing is that most of these sites also leave you logged in for a long time (forever?) and often I never go back to them after a week anyway.

In this scheme login (and in fact signup) looks like this:

  • Enter your email address.
  • Click login.
  • Receive an email with a magic one time use key.
  • Click the key and you are logged in.

Note: that anyone who tries to log in as you will cause you to get an email notifying you of the attempt…

You can also add a captcha etc in the email for the first time this login is used.

Sites can always still allow the user to update the account with passwords/2-factor/security questions etc later, but why bother unless it becomes necessary?

Sure its not the best solution for every case but nothing is.

I have played around with this idea on a demo site for myself and it seems to be workable. What am I missing?

A simple implementation using django is on github here

Please Stop Giving Me The Business

As a programmer there is good and bad about working inside a company who’s core business is not directly the software you are writing.

The Good:

  • No one knows what you do
  • You can get to learn about other businesses/fields

The bad:

  • No one knows what you do
  • You can end up being treated like the other

When  working inside of a company that doesn’t make software as it’s Core Business,  I want nothing more than to support the people who do other stuff in the business. I get that we need the sales, mktg, and manufacturing people to sell, market, and make the widgets since that’s where the money comes from. When you are developing in house software, its rare that the end consumer is out there saying:  “I am going to buy my widgets from ABC Corp since I hear their internal billing system is written in Node.js.”

However, I am getting really tired of being told “The business wants” or “This is a business requirement” or anything when the speaker is saying

YOU are just a programmer, the people in the offices with the windows THEY are the BUSINESS, just do what they say.

Its like they are saying that the developers can’t possibly understand what they do, so just let them write some stories about what processes need to be implemented and then write those up. Oh and when you are done we will have THEM do acceptance testing to see whether you suck or not.

Um Really?

Where is the WE? Where is the love?

The phrase “the business” is a kind of trump card.

Yeah, Yeah I hear you but the BUSINESS wants…

Here’s what’s funny, I have been at this a long time and sometimes maybe most of the time the users don’t really know what they want in house software to do. They only know what they do now and they have some vague ideas about what they would like to be different. They generally have a lot of good domain specific knowledge but they can be blinded by being so close to it they can’t really see what is going on. It is also the case that while they may be super smart in their area, they don’t really know much about process design issues.

The frustrating thing is that they have people in the company who are really good at thinking about logic and processes. Who can spot the weak points in a checklist, or notice what steps should be logged etc without even thinking hard about it. You think they would use this resource. And sometimes this happens. I remember a biz guy from Microsoft telling me that in his department anyway he always had the programmers review the contracts since they were the ones who were best at Math etc and could tell him whether there were any problems with his deal. So it does happen, but It seems to be rarer than it should be.

Its more likely that if the company perceives and issue with its processes, it will contract some expensive Six Sigma (or other religious) consultants to spend a week capturing ideas and process capabilities or whatever. Why would they invite the “computer” people?

So why would someone even think of working in a place like this? Often you have to fight against the grain of whatever the culture is to make something good happen. There are times when Agile isn’t nearly Agile enough etc. but If you persist and you are willing to take some risks you can make some good things happen and get some other benefits as well.

As I mentioned at the top, when you work somewhere that doesn’t do software as its core business you can learn a great deal about another subject. During my last contract job, I received a free education about clinical applications of human genetics from some super smart people. I was able to use and learn a variety of tools I selected to create the tool.

Nice work if you can get it, despite the fact that every now and then, someone is giving you the business.