an extremely hard day – I don’t usually bitch…
This has not been a good day. A long, tough business meeting let me to a surprising
and inescapable conclusion… the project we have been working so hard on is
more than we can handle with our current resources. This came as a complete
and total surprise, and very unwelcome ๐
It’s not that the architecture is too hard, we have that nailed now… it’s
not that the code is too hard, I have that nailed and I am genius anyway ๐
… it’s simply much more work than our preliminary understanding/design had
indicated and we need help.
You might be wondering how the amount of work involved in a project I designed
and was coding could sneak up on me? Ah, a good question and one I will take
a moment to delve into and takes us on a long rant about how I do business…
sit back and relax… this is gonna ramble and I am tired.
Software is an iceberg. For everything you see, there is much
much more foundation underneath. When you design a system – especially a "clean
slate" system with no preexisting code there is a unknown nightmare of
details and gremlins ahead of you – but managing them is part of the territory.
The issue is this … you have to design your system from someplace. You can
start bottom up or top down but you need to begin someplace, and often on a
clean slate system that is top down. Decide what it needs to do and then work
backwards from there to find out what the architecture has to look lick – the
features will tell you.
Take a simple enough task on a website – allowing users to communicate with
each other. E-mail, in a way. Now in something your just knocking together for
a personal site that is simple – send and read a message. But in a production
system, when that system is to be used by thousands of people and user seriously
it is much more complex. Especially when that e-mail system will be used to
conduct business transactions:
- [F]orward
- [R]eply
- Address book?
- E-mail saving?
- CC and BCC?
- Allow file attachments?
- Can someone forward that e-mail to their "real" e-mail account?
- What happens when an e-mail account fills up?
- Can the user download mail?
- Privacy/Security?
- Can the internal e-mail accounts get e-mail from "outside"? (
[email protected] )
The lists go on and on and on. Now, that is obviously not the surprising part
– that is my job and I have been doing it well for more than 15 years. The thing
is, until you start pinning down each feature in detail you can’t really know
all the gotchas.
OK then, why not just pin down all the features on paper in detail before you
start writing code? Because you often don’t know if something is possible –
or practical, or what the side effects are until you work with it.
Take user security for example… it turns out that IIS wasn’t going to provide
the security system I needed for this project, so I looked into coding it myself
and that turned out to be the "wrong thing[tm]". I looked around and
found a great
module for doing user authentication, it solved the problems – and in return
it created opportunities that had far reaching implications. In fact massive
chunks of the design for user subscriptions later on changed to accommodate
it and I had to put advanced security in much earlier than I planned.
All in all, a "win" for the project. The design was cleaner, simpler,
faster and more flexible… it simply took a lot of time to get to that point.
There was simply no way to know that in advance without a research and testing
process that would have been much longer than the "live" process took.
This happens often when you are working with "new" systems, developing
custom solutions to bring someone’s vision to reality. It would not happen if
you were rewriting that same system or one very similar to it for another client
– that’s called experience, and the more you branch off into the unknown the
more valuable your past experience gets.
I have lots of experience, and every time I bring new software to life that
experience helps me and my clients avoid the traps and pitfalls – but when I
do something basically new then I am learning right along for the first time.
Of course, because of my skills and experience I learn much faster
than most people do when the subject is programming and that is part of why
we are worth the money.
In any case, a large project opens like a flower… or a snarl of tangled wires.
Each feature puts down roots and begins to show you more and more of how the
underlying structure has to be and those features intertwine with
each other and it all takes shape and if your lucky it doesn’t tangle and you
can prune it and shape and viola! Just what you wanted.
Sometimes things go wrong. When it is a large project that is doing new and
innovative things then sometimes bad things happen. Surprises happen. Stuff
like this sometimes happens.
In my entire professional career, this is the first time this has happened
to a project for a client that I was in charge of.
I simply cannot, it turns out, deliver it on time with the resources I have
on hand. It turns out that it is more work than I can do all by myself (I am
our only developer) at this point… and more than that, it demands a high quality
of the end product because someone’s entire dream is riding on it – their future,
their livelihood, their desire to get something that matters to them online.
So… how does it feel to tell someone that you were wrong? That you can’t
do what you thought you could do, what you were sure you could
do? How does it feel to have to say "I was wrong, and I can’t do it in
the time I said, not even close".
Not even close. That’s the part that really makes you feel stupid.
Thinking back on it, it didn’t sneak up on us totally. It just slapped me around
a lot. See, we have been discussing with our client that in order to meet our
timelines we wanted to look into spinning off some of the work. We just didn’t
realize those pieces we needed to spin off were the tip of a vast expanse of
work that was about to come crashing down on us.
So in a way, we have sort of been preparing for this deep down maybe – we just
didn’t have any idea it was this big. Maybe subconsciously we knew something
– all of us, because our client asked us a few times if we thought we had the
resources.
I said yes. Because I honestly thought I did. Everything we knew at that moment
said we did. It wasn’t until we sat down with all the little changes and implications
and the stuff I learned over the last few days and so one that we realized we
were wrong.
How do I feel? I feel like an asshole.
But I’ll tell you what would make me feel worse. To know that there was a serious
problem like this and NOT tell my client the instant I realized it. It is simply
unthinkable. I don’t work with million dollar corporations… I work with people
who are using some of their limited resources to buy my best efforts in their
fight to realize a dream – whatever that is. It is morally imperative for me
to tell them as soon as I see a problem because I will not let
their money go down the drain while I try and save my ego.
Even when that problem makes me feel this stupid.
I have been told in the past by clients that I am a "loose" contractor.
In other words, I am very honest with them if it looks like they need to buy
something from another vendor… computers, software, consultations or what
have you… and I always, always make sure that they know that I can help them
find someone else if I am not doing the job.
In other words, I don’t lock anyone into me.
Why is that? Because I am a computer consultant. Just after dentists and lawyers
I think people hate us the most. How many times have you heard a consultant
tell someone you know "ah, well, we have to rip all that out of there and
start over" before they rape them for their last penny – only to disappear
and leave them for the next consultant to "start over" and so on.
Not my clients… I am honest about my abilities. I use industry standard hardware,
I use the standard OS’s. I comment my source code. I don’t do bizarre and funky
things. I don’t disappear. And I am more than happy to help explain it all to
someone else if you replace me.
Because of all that, I have had consistent clients that are with me after 14
years.
I am amazingly proud of my team here, and when I am not feeling stupid about
this I realize how good we really are – the reality is that if we do get a chance
to finish this the design/architecture we have in place will be capable of competing
with a set of websites that cost millions of dollars to
produce… and I promise you we don’t have a fraction of that sort of budget.
We couldn’t have predicted this outcome, it all looked really good till very
recently (this morning). Yes, there were delays… but that does happen all
the time in software. I feel bad, even though we did our best and I can’t see
how we could have done it much different walking through it. Sure, hindsight
is 20/20 and all… but that isn’t a way to judge your past.
::Sighs:: Well, in a nutshell.. That is what happened today. It turns out we
were wrong.
We spent most of the day figuring that out, then the rest of the day coming
up with a plan to deal with it because I also don’t leave clients hanging…
and I don’t just walk away from them. We apologized, we did what we could to
try and help defray some of the expense for out client and we have a tentative
plan in place… one we will firm it up over the next few days and we will move
on and get this stuff done.
Still. Not a good day.