software requirements again, take 483
in [Perl]
|
From: J�rgen Exner on 3 Apr 2010 18:24 ccc31807 <cartercc(a)gmail.com> wrote: Same old story again. >Charlton, yesterday (Friday) when I came to work, I found a new >project on my desk. It was a log file from our corporate call center. >The manager wanted to know the locations of the callers. He had >attempted to do some of it by hand, but that is an arduous task, and >he soon gave up. > >It was pretty simple to do, and I did it, from scratch, in less than >four hours, including finding and parsing the telephone exchange data >(which was a little difficult). Let me rephrase that for you: I jumped right into coding without checking if the spec was accurate, the business guy and I understood it the same way, or even if it was complete. >When I finished, I called the manager >in charge and asked what format he wanted the report in. The answer? >"I don't know." > >Think about this -- a manager assigns a project without thinking it >through to the extent that he knew the format of the output. And that is exactly what is biting you over and over again. The spec, no matter if a 2 paragraph email or a 2000 page document, was incomplete and you didn't even check before starting coding. It is not the job of the manager but YOUR's to make sure that you have all the necessary information to finish the task. That is a very basic software engineering requirement. >Here I >was, with nice work and a ton of data, and no way to return it. >This is a communication problem. It's not a failure to communicate, >but a failure to have something to communicate. It's difficult to say >something if you don't have anything to say. No, it is not a communication problem. It's a problem of poor processes. You keep jumping to step 5 before completing or at the very least thouroughly checking steps 2-4. >We'll get this sorted out on Monday so I'm not worried about it. The >point is that the technical person can't work without knowing what the >requirements are, and at least in my situation, I am often assigned >work with unclear, ambiguous, or absent requirements. Then why on earth don't you ask for clarification? It is absolutely essential and in the very own interest of a programmer to know exactly what he is supposed to program before starting to program. And that's what (a large part of) software engineering is all about: making sure you know what you are supposed to do. You cannot expect a business person or any user for that matter to provide you with the perfect or even a meaningful specification. Therefore it is YOUR(!) job to work with them, to discuss, to point out open questions, and yes, even to help them to make up their mind how certain things are supposed to look like or the output is supposed to be formatted. And this doesn't have to be a 300 page spec written in blood and notarized. A short email may be sufficient, but it absolutely needs to be addressed and absolutely has to be in writing such that everyone is working off the same expectations. And that comes before(!!!) you write a single line of code. >Finally, maybe I should add that the whole process is iterative, and >that the client doesn't know how he wants the data until he sees the >data. In this case, the client and the developer work together through >a number of iterations until the client sees how he uses the data, and >the developer helps him my writing output in a number of formats. Nothing wrong with that. If that is the preferred way, then it should be stated as such in the spec. Then it becomes part of the process, is expected, and can be scheduled and credited accordingly. Without it being documented in the spec the conclusion will be: "Oh, John just doesn't know what he is doing. I had to come back to him 5 times until finally he got the output right. And because of that we were 2 weeks late, too, and missed the deadline for the appliation." >(Quick story -- We had a big push for graphical formatting and >visualization, the bosses wanted to 'see' the data. I spent several >days and developed an application that took data and spat out gifs, >literally hundreds of gifs, bar charts, line graphs, etc. The final >result was a 40+ page PDF with eight charts on each page. Guess what? >After I submitted this, I never heard from it again. The theory about >visualizing data is good, but it has it's drawbacks as well, and you >might not realize that you really don't want to 'see' the data until >after you see it.) The secret keyword for that is "prototyping". If he doesn't know exactly what he is looking for (and that is not uncommon) then do a few different prototypes for him to choose from. Then do a few more variations with simple, limited data sets for those he favours. And only after he confirms "This here, this is what I want", then you go full steam ahead and implement the full solution. Is it stands now you probably wasted several days of work while with using prototypes either he would have gotten what he was looking for or if charts were a bad idea to begin with he would have found out much earlier after spending maybe just a few hours on the task. jue
From: ccc31807 on 3 Apr 2010 22:38 On Apr 3, 5:24 pm, J rgen Exner <jurge...(a)hotmail.com> wrote: > Let me rephrase that for you: I jumped right into coding without > checking if the spec was accurate, the business guy and I understood it > the same way, or even if it was complete. The business guy was out of the office at a conference. All I had was the log file and that he wanted it by Monday. Initially, I didn't even know what it was that he wanted, but it wasn't difficult to guess. I didn't want to wait until I knew the requirements in toto when I knew I could do 90% with the information I had. > And that is exactly what is biting you over and over again. The spec, no > matter if a 2 paragraph email or a 2000 page document, was incomplete > and you didn't even check before starting coding. It is not the job of > the manager but YOUR's to make sure that you have all the necessary > information to finish the task. That is a very basic software > engineering requirement. The 'spec' in this case is an email with no message, just an attachment. After I figured out what he wanted, I had enough information to do the heavy lifting. The format is immaterial to me as I can stuff the data into any format he is likely to want (probably an Excel file). I had all the NECESSARY information to basically complete the job, but not enough to produce the final product. > No, it is not a communication problem. It's a problem of poor processes. I totally agree. > Then why on earth don't you ask for clarification? Because I couldn't. The clarifier was not available. > It is absolutely > essential and in the very own interest of a programmer to know exactly > what he is supposed to program before starting to program. Disagree strongly. It's true that you need to know precisely what you are doing before writing code, but particularly with agile processes, you don't need the specs nailed 100% before starting to write some code. > >Finally, maybe I should add that the whole process is iterative, and > >that the client doesn't know how he wants the data until he sees the > >data. In this case, the client and the developer work together through > >a number of iterations until the client sees how he uses the data, and > >the developer helps him my writing output in a number of formats. > > Nothing wrong with that. If that is the preferred way, then it should be > stated as such in the spec. Then it becomes part of the process, is > expected, and can be scheduled and credited accordingly. Which is pretty much the way I work. People who work with me are accustomed to have some back and forth before they get the final product. > Without it being documented in the spec the conclusion will be: "Oh, > John just doesn't know what he is doing. I had to come back to him 5 > times until finally he got the output right. And because of that we were > 2 weeks late, too, and missed the deadline for the appliation." John: "I'll take the blame for assuming you were not an idiot, and for assuming that the previous requirements you gave me were accurate. I should have known better." I have said this directly to some people, and those people, while they don't care for me, do take pains not to give me any excuse to say that their instructions were imprecise or unclear. > The secret keyword for that is "prototyping". If he doesn't know exactly > what he is looking for (and that is not uncommon) then do a few > different prototypes for him to choose from. Then do a few more > variations with simple, limited data sets for those he favours. And only > after he confirms "This here, this is what I want", then you go full > steam ahead and implement the full solution. That's essentially what I did with the project mentioned before. The HR staff did not think through the process, and I had to go back to them several times with a 'What now?' question. This was the reason that the software was a week late -- their tardiness in answering my questions. If these issues had been resolved in real time, I would have finished on time. Also, it's stupid to blame the programmer for lack of domain knowledge when the SMEs didn't have the appropriate domain knowledge. > Is it stands now you probably wasted several days of work Not true at all. I had been playing with GD for several years, wanting to use it in code. This gave me the excuse to learn GD. I don't consider that a waste, and in fact was able to leverage that experience to complete a side job. In effect, I got paid twice for the same work. Ain't life grand. CC.
From: Peter J. Holzer on 4 Apr 2010 04:09 [the project which started this thread took several weeks] On 2010-04-03 10:37, ccc31807 <cartercc(a)gmail.com> wrote: > Charlton, yesterday (Friday) when I came to work, I found a new > project on my desk. It was a log file from our corporate call center. > The manager wanted to know the locations of the callers. He had > attempted to do some of it by hand, but that is an arduous task, and > he soon gave up. > > It was pretty simple to do, and I did it, from scratch, in less than > four hours, including finding and parsing the telephone exchange data > (which was a little difficult). [...] > (Quick story -- We had a big push for graphical formatting and > visualization, the bosses wanted to 'see' the data. I spent several > days and developed an application that took data and spat out gifs, > literally hundreds of gifs, bar charts, line graphs, etc. The final > result was a 40+ page PDF with eight charts on each page. Guess what? Carter, please do yourself and us a favour and try to distinguish between projects which take a few hours and those which take days or weeks (or months or years, but you don't seem to have those). For something that takes a few hours I agree with you that it often makes sense to knock up a prototype *before* asking the user for clarification: It gives you a chance to look at the problem so that you can ask the right questions, and it gives you something tangible to talk about (some users have absolutely zero imagination). And if you know your users there's a good chance that your prototype will already be good enough. However, for longer projects trying to solve 90% before asking the user what they really want is just stupid. It's still a good idea to have a look at the problem before asking for clarification so that you have an idea what they are talking about and what's possible and that you can ask the right questions. But before putting significant work into the project you have to make that you do what the user wants (now - he will probably change his mind later, but that's his problem). I think J�rgen and you make the same mistake, but you come from different directions: You usually do small jobs (I actually hesitate to call them "projects") where almost any formalization would cost time, not save it. And then you try the same approach with larger projects (still rather small, but somewhere beyound the size where a bit of planning pays) and are frustrated that it doesn't work as well as for your small jobs. J�rgen obviously comes from the opposite direction: Projects which have a defined scope (probably in a business environment where the customer and the contractor agree on a price beforehand and sign a contract). And he doesn't see that there are jobs where working like this adds too much overhead. You can't have a one hour meeting for a five minute job. hp
From: Mart van de Wege on 4 Apr 2010 05:10 Charlton Wilbur <cwilbur(a)chromatico.net> writes: > > So if you write the requirements, the business process owner approves of > them, and the software matches the requirements, the bug is not your > fault, and you should push back HARD when the business process owner > tries to make it your fault. If you do this well, you can start > teaching the business process owner how to communicate better with you, > and get across to him the importance of accurately communicating > requirements. > Agreed, with one caveat: it is not necessarily the business process owner who has a responsibility to communicate. As a software engineer (or any service job implementing a solution to a business problem) you are also required to ask questions to get the spec right. Merely depending on what the customer tells you, and then turning around and blaming the customer when it doesn't work, is not the right way to go about it. And unfortunately, I see the latter happen all too much. Mart -- "We will need a longer wall when the revolution comes." --- AJS, quoting an uncertain source.
From: ccc31807 on 4 Apr 2010 09:33
I agree with almost everything below. The only thing I would add is that planning is necessary for small scripting tasks as well as the major projects, and that experimentation and an ad hoc approach can yield insight into specific areas in large projects as well as the smaller ones. In a former life, I did a lot of extemporaneous speaking. 'Extemporaneous' doesn't means having no knowledge about a subject, it actually requires a much deeper understanding of the subject that a prepared speech. When called on to say say something (which likely might be an extended analysis of a complex problem) it took be about half a minute to develop an outline of the speech in my head while make some introductory remarks (e.g., telling a joke or complimenting the previous speaker). Doing a small job is exactly the same. You can develop the pseudo code in your head while writing: #! usr/bin/perl #informational header, input and output use strict; use warnings; print "Running ad_hoc_job.plx\n" exit(0); The point is that some kinds of tasks become so familiar that you can outline the script in your sleep. The problems tend to come in the little regular expression that you need to break apart the input line -- and THAT can take more time to write the the entire rest of the job. Thanks for your insightful comments, CC. On Apr 4, 4:09 am, "Peter J. Holzer" <hjp-usen...(a)hjp.at> wrote: > Carter, please do yourself and us a favour and try to distinguish > between projects which take a few hours and those which take days or > weeks (or months or years, but you don't seem to have those). > > For something that takes a few hours I agree with you that it often > makes sense to knock up a prototype *before* asking the user for > clarification: It gives you a chance to look at the problem so that you > can ask the right questions, and it gives you something tangible to talk > about (some users have absolutely zero imagination). And if you know > your users there's a good chance that your prototype will already be > good enough. > > However, for longer projects trying to solve 90% before asking the user > what they really want is just stupid. It's still a good idea to have a > look at the problem before asking for clarification so that you have an > idea what they are talking about and what's possible and that you can > ask the right questions. But before putting significant work into the > project you have to make that you do what the user wants (now - he will > probably change his mind later, but that's his problem). > > I think Jürgen and you make the same mistake, but you come from > different directions: > > You usually do small jobs (I actually hesitate to call them "projects") > where almost any formalization would cost time, not save it. And then > you try the same approach with larger projects (still rather small, but > somewhere beyound the size where a bit of planning pays) and are > frustrated that it doesn't work as well as for your small jobs. > > Jürgen obviously comes from the opposite direction: Projects which have > a defined scope (probably in a business environment where the customer > and the contractor agree on a price beforehand and sign a contract). And > he doesn't see that there are jobs where working like this adds too > much overhead. You can't have a one hour meeting for a five minute job. > > hp |