[UPHPU] Software Development Scheduling advice

[Jonathan Duncan]

On Tue, 30 Jan 2007, Mac Newbold wrote:

> Today at 1:43pm,  Victor Villa said:
> 
>>> Estimates for any project that is even moderately long are just
>>> fantasies.
>> 
>> These are might true words.  Not sure how people like Mac Newbold set
>> accurate timelines and achieve them, though that is prolly a good uphpu meet
>> topic.
> 
> Well, Victor, thanks for asking...
> 
> My philosophies on this are all based on being up front with the client. If 
> it may take you 10 hours, or may end up taking 20, tell them it will take 
> 10-20 hours, and for price tell them 10*hourlyrate to 20*hourlyrate. If there 
> are certain aspects of uncertainty you can pin down, like the first 3 hours 
> is research to find out what problems you might run into, then you can say 
> that after the first 3-5 hours you'll be able to give a better estimate. 
> There's always the chance that you'll run into something completely 
> unforeseeable, and when that happens, tell them what's up, and give them 
> enough info to make an informed decision about how to continue (like slogging 
> forward anyway, changing course, or aborting that part of the mission as "not 
> worth it").
> 
> All of the proposals/bids/estimates we do here at work are price ranges (and 
> time ranges) and are only estimates. If they make changes or there are 
> unforeseen circumstances, it's all subject to adjustment, and any additional 
> work is billed at an hourly rate. This eliminates all the hassle of change 
> orders and figuring out how much extra it will cost to add in a given change. 
> Estimating takes time too, so the less you have to do it the more time you'll 
> be able to spend actually programming. It allows for much more flexibility 
> for the client too. It is rare that a project will finish with the exact same 
> specs and scope as it started with, because you'll have better ideas and 
> information once you get into the project. It is more cost effective to do 
> iterations like John suggested, since you don't spend time working on stuff 
> that won't be necessary in the end. Think about taking a trip. If you're 
> trying to chase down a car, and they're at location X, do you plan a path and 
> drive to location X? Or do you check periodically on the way toward location 
> X to see if they have moved? If they have moved, or a road is blocked off, or 
> whatever, you may have to take some detours or change course or destination. 
> The more often you check your progress (as long as you can do it quickly) the 
> less likely you will be to waste effort in unneeded tasks.
> 
>> Me, I feel like Johnny Carson when somebody asks me for a deadline.
>> (putting the envelope against my forehead and coming up with the wildest
>> latest date possible that is somewhere between reality and the day before
>> they go nuts if it isn't delivered.  Though, mainly for me the problem comes
>> from projects expected of me that have components from other people in the
>> company.
>> 
>> Who knows?
> 
> These are all good points. Conservative is good, but they tend to be happier 
> if you give them a reasonable "best case" and a conservative "worst case" 
> estimate, so they know that there are things that can't be known from the 
> beginning, but it will likely end somewhere between those two. It also works 
> really well if you can back up your numbers. When you've got to estimate on a 
> big project, come up with a rough outline of a plan. Break it down into its 
> major components or steps, and estimate all of those, then add them all up. 
> Estimate times and costs, and you can use your times to determine a schedule, 
> based on how much time will actually go toward this project in any given 
> week/month.
> 
> One that really can get you into trouble is not factoring things you'll need 
> that are beyond your control. If you will have to wait for someone to test, 
> approve, decide, or gather information for you, put that in the schedule. 
> Make a reasonable guess of how long it will take them to get back to you with 
> answers you can use (sometimes a day, sometimes a week, sometimes more). 
> Don't forget about the graphic design, excel files, PDFs, images, or other 
> stuff that you need in order to build the site but aren't your 
> responsability. They generally will end up late. The other advantage is it 
> lets your boss put pressure on the right people. If you can't move forward 
> because you're waiting for a decision or action from the CTO or the graphic 
> designer, or the sysadmin, then you can point him at the right place to apply 
> pressure instead of it all coming down on you. Trust me, his pressure on them 
> will do a lot more than his pressure on your and your pressure on them will 
> do.
> 
> The smaller you can break up a task, the easier it is to estimate, but if you 
> get too small (less than a couple of hours) the estimating error can really 
> add up. (You might be able to do 4 tasks in a hour or two, but if you 
> estimate them each at an hour, then it will be a high estimate.) Be 
> especially careful about things that multiply, like putting in bodies for 100 
> hand-coded html/css pages. You could say an hour each, but some might be 10 
> minutes, and others might be 5 hours. Guess on a good average, and remember 
> to give a range that covers you.
> 
> If you constantly come in on the high end of your ranges, you may need to 
> back it up with what happened and why it wasn't best case. It depends on the 
> client/boss on how likely they are to get picky about the details. This is 
> especially true when it throws you off the timeline and the budget. But if 
> you can back up what you've done, most rational people will be understanding 
> about it and will realize you're doing your best and doing good or great work 
> for them in tough situations.
> 
> Don't forget to include things like quality assurance. Testing and debugging 
> take extra time on top of what it takes you to build it in the first place. 
> Debugging can be 1-2% of the total time, if you're lucky or really good, or 
> it could be 10-20% on a particularly troublesome piece of work. Testing will 
> take some time no matter what you do or how good you are. I recommend testing 
> a lot as you go (unit testing) so that you can test the small pieces before 
> you build on them to make big pieces. If you know the small parts work right, 
> it is much easier to debug the big parts, since it narrows down where you're 
> looking for the problem. Then you should always make sure to do some whole 
> system testing whenever you finish anything big or anything that works 
> together with other parts. Then, it really helps if you can have someone 
> other than the main developer on the project try it out, especially if they 
> understand well what the desired result should be. Keep in mind the Principle 
> of Least Surprise: if your tester is surprised by a behavior, it might not be 
> the right behavior.
> 
> What percentages you need to estimate for QA, testing, and debugging will 
> depend on you and on the project probably, and on how picky the client is. 
> Many clients underspecify, and expect you to just choose something, but then 
> always want the thing you didn't choose, and often want you to change it for 
> free. Don't let them yank you around, but make sure if that might be the 
> case, that you factor that into your estimate. You may want to plan on a 
> margin of about 10% of the total time for testing and debugging everything, 
> and all the final fixes that the client will ask for.
> 
> Well, those are some of the ways I usually do it. The more you do the more 
> you'll learn, especially if you really track the actual results, and adjust 
> accordingly in the future. New clients are harder to judge because you don't 
> know them yet, so if you want to be safe, use a larger range than usual, and 
> make sure that there's an hourly rate in there for extra work they request, 
> and that you document everything they ask for that falls outside the original 
> plan. Then you can back up why it went over budget, because they changed a 
> million things.
> 
> Anyway, there's a lot of buzz going on in #uphpu on IRC too, so you might 
> want to check http://ibot.rikers.org/%23uphpu/ tomorrow if you're not in 
> there. Or of course, come join us and you can ask questions too.
> 
>

Excellent and very helpful insight.  Thank you.  This is a good topic.  I have 
recently beening required to do the same thing.  It truly is something that 
comes with experience.

Jonathan