iAnalysis or rake analyze

Why isn't there something like Intellisense for analysis yet?  Surely we've mastered the patterns involved in analysis so far :)  Why can't I type a user story and hit ctrl-spacebar and have a series of analysis methods pop down for me to choose or even override? hmmmmmm.

What are we building - documents or software?

One of the people I like to read and get insight from is Scott Ambler.  One of the things he and several other thought leaders in the Agile movement point out is that the end product we are building is software, not a document repository (unless you happen to be building some software that is actually a document repository.. but I digress).  I think this thought is generally accepted as part of the agile movement (we value working software over comprehensive documentation).  That has been a value of mine now for several months and has been shaping the way I approach business analysis.  The important thing to remember here is that we aren't tossing documentation out the window, but we are getting rid of the wasteful documentation that doesn't help get the software developed.

So what do I do with all this free time?  If you read up on some of the things Scott mentions, I've picked up some old skills of mine when I was first doing development and help my team out writing unit tests.  We aren't doing TDD yet, which is a specific pain point for me because I know how much it cleans up your code and reduces complexity, but these unit tests are giving us some assurance that we aren't breaking code.  Another thing I've been doing a lot is reading, which has proved fantastically rewarding.  I would recommend a recent book by Andy Hunt that I just read, "Pragmatic Thinking and Learning: Refactor your Wetware".  This book is a really good primer on thinking in a different way.  If you happen to major in philosophy like I did, you'll see a lot of things that you probably read from Phenomenology, Husserl, Heidegger, and some of the ancient philosophers.  

Working for work's sake

The idea of producing less documentation still bothers me from time to time though, not because I think this approach is wrong, but I still have that Industrial age mentality that imposes the mindset that I must always be producing widgets all the time.  So I still have to stay on my toes to continually help the software come into being, while making sure I don't fall into the old trap of writing documents now on something that I won't work on for months.

Whats wrong with "getting ahead"?

There are two major pitfalls with creating large amounts of documentation about something that won't be coded/implemented for months.  

1. What you document today will ultimately change (emphasis on will) by the time you implement it for the users to benefit from it.
   
2. What you document today most likely won't contribute to what is being built right now.  Yes this is tied directly with point 1 as well.

So why is this?  For both points, we all know that requirements change, and as an agilist you welcome those changing requirements.  However, if you spend time and effort writing documentation now, then that is ultimately wasted when it comes time to implement it because the requirements have to be "re-written" or updated because they aren't what is needed now.  Also, that time you spent writing the documentation then prevented you from helping your team on the current software iteration/increment now.

Balance this with Domain Knowledge

There is the idea of domain knowledge that is sometimes regarded as beneficial documentation.  I would agree on this with a caveat.  That documentation needs to be somewhere that everyone sees readily and quickly - like a Wiki or on whiteboards/Post-It note boards in the room.  It should not be secured by the analysts only to be brought out when the sprint is going to implement it, the team should be learning the domain knowledge along with the BAs, though the BAs probably take on more of a coaching job of the domain knowledge to the rest of the team.

Software development is about developing software ultimately, don't waste your time or your client's time by creating documentation that no one will read and that won't help your team.

Should Agile be hard?

Agile software is supposed to help us deliver quality software that the customer wants as quickly as possible.  There are series of twelve principles that have been put in place to help form the idea of agile software development.  Shouldn't we be able to utilize this information easily to create software?  It depends.......

I find that the principles that make up agile software development are helpful in generalizing what we need to aspire to.  They don't give you any specifics really, just beneficial guidelines that should help you get there, something like a small flashlight to help guide you down the path just far enough.

The problem is that if the people involved in an agile project don't understand the meanings behind these principles, they aren't of much use.  I think this points towards what you'll find in the Dreyfus Model of Skills Acquisition regarding people's skill in either software development or whatever trade they have that will be combined on an agile software project.  The following may or may not make sense:

• Agile projects that combine people with skillsets of varying degree might be successful as long as the variance is within the skillset, and that skillset has some people at the proficient level (or competent level at the very least but this introduces a lot of risk).
  
• Agile projects that combine a group of people with a certain skillset where no one has the skillset above Advanced Beginner are in danger of failing.

I say this because I'm starting to believe that agile software is extremely dependent on skilled "artisans" that understand what those 12 principles mean.  Those 12 principles are understood within context far better than they are if just considered by themselves.  Someone in the Beginner or Advanced Beginner skillset may look at those and take them as hard and fast rules that are applicable everywhere (Best Practices anyone?).  Even those in the Competent skillset are in danger of doing this.  However, when you crossover into the the Proficient and of course Expert skillset, you don't understand these as hard rules that must followed rigidly in all areas, you understand that things exist in context of something, and thatthey need to viewed in that light to be understood.

So expand this past just the development team, I think we understand what happens when you have a group of junior developers and you want them to try agile with no one versed in the skills that a proficient or expert developer can explain.  Apply this thinking to the management team- what happens when you have a group of people in the management role that are fairly unskilled in it.  I think this lends to creating environments that are unsafe for agile to be practiced and crafted.  To me, people with management styles that are still rooted in the industrial revolution mentality (people are widgets or resources and can be replaced easily) are bound to doom an agile project because they aren't prepared to treat the team as a group of skilled craftsman, but instead as a group of people that replaceable.  If you have a management team that contains those at the Proficient skillset, I believe they tend to lean towards management practices that let a team strive to meet a goal, and accept failures while expecting success.  Proficient and Expert managers tend to understand that it is much more valuable to have people that are motivated to learn and to do something than it is to have people who only do their job out of fear of what you may do to them.  Where a beginning manager may see people as subject to rules that they have encountered in management seminars, proficient managers see people as unique individuals living within unique contexts that don't respond the same way to one stimulus or teaching.

Anyways, I've rambled on long enough...  My point to all of this is that if you have the right team setup that is geared towards Proficient skillsets, then agile will probably be easy because chances are you'll probably already be doing it.  If you don't have that mix, then expect a bloody struggle and possibly a remake of the Myth of Sisyphus.

Long Delay

Sorry for the delay, but after attending an agile conference in Orlando back in November, I really had my assumptions and understanding of agile shaken to the core. I find it hard to call what I was doing before as agile, but more like FrAgile. I heard of agile, knew itwas probably good and was striving towards it. My only problem was that I was really focused on the process of Scrum to be my "agile" instead of trying to understand what it means to "be agile".

Of course, I now welcome failures like this. And I don't think of failures as really all that bad (unless you happen to be performing surgery on me at the moment). Falling flat on my face has always been the most rewarding experience for me because I never stay down. I always get up, turn around to see what tripped me, and keep going again and avoid what tripped me.

Anyways, what really kicked me over from the conference was my realiziation that I was focusing too much on doing scrum as a process, that I neglected what should have been my goal - delivering quality software to my clients. I just assumed that if we followed what the scrum books said then it would naturally follow.

So when I got back I began to focus on the principles from the Agile Manifesto (http://agilemanifesto.org/principles.html). From there I've tried reading more theory about agile and its principles and less about how a process can save the day. I'm still battling with how to be a BA, but I'm less focused on the role of BA now. I'm debating the use of roles on an agile team (in some contexts they are useful, in others maybe not so much). I'm really getting interested though in how I can help my team complete their tasks by helping them get the information they need as quickly as possible.

More to come... I want to write something else now.

Requirements: Just in Time - Part Three

Iterative Requirements.....
How does one accomplish iterative requirements? I almost wonder how one cannot accomplish this, as the process for gathering requirements is in itself iterative. Version 1.6 of the BABOK hasn't covered this yet, though maybe future versions have. As the process of business analysis continues, the analyst is continually refining (refactoring) his/her requirements document each time they meet with stakeholders for a project. The big jump though from the waterfall process to the agile process is that we don't spend code-less iterations doing this prior to code actually being written. We must perform smaller iterations of requirement elicitation concurrently with development.

Here's the big stepping point.... instead of the normal effort spent way up front attempting to discern with infinite precision what the software will do, we are now doing it at the same time the code for it is being written. This gives us immediate feedback about the requirement/product and quickly tells us if we are failing or succeeding.

I'll admit this is a bit of a roller coaster ride as you know very quickly if the requirement you covered was done successfully, or if a developer/qa person uncovers something you didn't think of. -- As an aside, this is the kind of thinking I have to move away from, you'll never catch everything in a requirement. -- As opposed to the waterfall process where you spend a great deal of time basking in the thought that your requirements perfectly explain how a system will function, until you see months later that instead of a perfect billing application, you have a login screen, and two page to enter account information on.

Requirements at the speed of development
Most analysts I know are extremely anal about their requirement documents. They will spend hours upon hours making sure every aspect is covered in excruciating detail so that in their minds it makes perfect sense - so much so that even a developer would understand it! This makes the move to agile for most a painful one in my opinion. Mostly because it brings the developer into concert with stakeholder, no longer placing the wall up between the two. It also makes it painful because we don't have these long timeframes to write and Re-write (using WordUnit of course http://www.waterfall2006.com/beck.html) requirements. The transfer of knowledge happens much more quickly, coming from the stakeholder directly to your developer/analyst.

So how do we accomplish requirements so quickly? I would venture that we have to stop thinking of requirements in the same way that we have for the past several years. The document that used to hold pages upon pages of value mappings, or data constraints, etc., are no longer very useful. What is useful is a database that has value mapping and data constraints. As opposed to the mammoth SRS that used to hold these and never get read, the database would hold these and would constantly be read. If development and the product owners are constantly meeting with each other, then that long document is no longer needed, just the database that does what the document was going to tell it to do.

It is getting late, but I think I'm going in a solid direction for now. I'll pick this up later after I've made my next big mistake. :)

Agile QA

Before I begin this post, let me point out that I believe QA must exist within Agile. My discussion here is on how I think it should exist if it is expected to be agile.

As every BA knows, he/she will undertake QA to some degree. This is very clear in the waterfall process as the BA is often tasked with writing the tests that will usually be performed and ultimately automated by the QA team. In waterfall, this happens very distinctly at the end of development. Once developers finish what they set out to write, it is tossed over the wall to QA, and the waiting for bug reports ensue. Its a very well laid out process, and has successfully helped delay and hinder projects along with all the other steps of waterfall. Unfortunately, there is very little written on the web that discusses this so I'm tossing an idea out there.

When I look at the QA process in my curent project, it is unfortunately still bogged down with the waterfall process. We have really good QA testers, but we are still following a process whereby a lot of documentation is created up front about what will be tested, and then testing it after the software is developed. While we have focused our attention on the rest of the team to incorporate agile into our process by doing daily builds, increasing discussion with the product owner and building our 'self-organizing' teams, we are still in the mindset of 'tossing' a finished development effort over to QA to find bugs. Even though we are doing daily builds and handing over software iteratively (finished to a point but not yet complete), we are still not getting QA in the loop as soon as I think they should. This process seems wrong, both from an agile standpoint, and from an effort to improve software as we build it.

Think of it this way: If there are people on your team that are responsible for the quality of a product, you would want them involved at the very beginning, right?. By this I mean that their expertise and input are gathered at the beginning as development starts writing code, and is continuously sought after during the development process. What this does, I believe, is give the team the direction and insight that they need to build the product that will stand up not only to what the product owner asked for, but will also become a piece of software that the product owner wants, and will do that much sooner than if we tossed it over to be tested.

Imagine these two scenarios:

Scenario 1:

Team begins development on a product by reading the user story and clarifying the acceptance criteria with the product owner. The teams begins developing based on that conversation, and asking the product owner for clarification along the way until they finish writing the code that they think satisfies it. Meanwhile, QA testers are preparing test scenarios and cases that they intend to exercise against the finished version of the software. During these two seperate threads, neither group really talks with the other about what they are doing. Once the software reaches testable phases, all of that is handed off (that should be a flag right there) to QA so that they can begin testing it and seeing if it meets their perception of the acceptance criteria + their QA insight. QA begins testing it and finds all sorts of issues that those developing it didn't really think of (not because they are bad codeers, but because they think in a different way). These issues are then worked on, sometimes argued about, sometimes deferred till later, sometimes fixed, and then re-tested.

This scenario is what I see as a phenomenal waste of talent within the team. If this is the process we follow, then I wouldn't consider the team very agile but instead falling for into scrum-fall process that will lead to problems.

Scenario 2:

Team begins development on a product by reading the user story and clarifying the acceptance criteria with the product owner. The team then decides on what they will start building and gets further input from those that will be testing it. Those people on the team now know what to build based on the acceptance criteria + criteria that will be put into internal tests to ensure quality. As the software is built, further conversations take place for clarification both with the product owners and QA, and the sofware is built to those expectations (Note that I have not said that what QA wants to test is gospel, that is part of the conversation). As the software reaches testable phases, QA runs thier tests against that bit of functionality. When a test does fail, those who develop it more likley know exactly why and what was missed and can fix it (assuming the test was correct).

The difference in this scenario is that at this point those tests should be more of a formality since development was geared towards making them pass from the very beginning. Thus, the amount of QA's valuable time is not wasted testing for things that the developers never realized they would be expected to build in the first place. The time spent up front in the conversation is very minimal when compared to the rework that may be needed if end the tests are not known ahead of time.

Think of it as though you were in school (to borrow Mike Cohn's analogy of acceptance criteria). If you are studying for something that you will be tested on, you want to know what the test will ask so you know what to study. Knowing that, you are better prepared for the test because you geared your study habits successfully to answer those questions.

This scenario gets us closer to a more collaborative team environment. I also believe that it helps improve on the first scenario's waste by letting developers know what they will be tested on ahead of time so their effort takes that into consideration. At the end of each iterative step, less time is spent going back and forth between QA and development because it was mitigated in the beginning. I don't see this as the silver bullet, as QA will certainly come up with testing scenarios as the software is developed that they may have not thought of at the beginning of development, but it should get us a great deal closer to a more agile process.

I still fully expect the role of QA is to make sure that the client gets not only what they asked for, but what they want. I also expect that QA should try to break the code as much as possible. What I'm advocating for here is a more collaborative approach that makes sure those who will be subjected to a test, know exactly what they will be tested on from the very beginning instead of getting surpised at the end.

Requirements: Just in Time - Part Two

In the first part of this post, I tried to walk through what I was thinking about requirements gathering/elicitation in the agile sphere. I noted then that it was subject to chanage, and I'll reinforce that note in this post I'm sure.

As I discussed in the 3rd part called "Detailed examination of the requirement", I talked about when it should be be done - as close as possible to the sprint that the product owner wants it to be implemented. That satisifed my post's title. Now I'd like to talk a little more about what is done for requirements gathering/elicitation as the BA (that is after all, our traditional bread and butter).

When I first started out trying to learn agile, I managed to probably make every mistake I could make, or at least put a significant dent in them. The best one was at the beginning. I had just come from a waterfall job, so I was still in the frame of mind that tomes of documentation were a good thing even though the agile preference found value in less of them. So while I knew that the developers would be working from User Stories that barely fit on a notecard, I was convinced that all Agile BAs had a dirtly little secret that they kept detailed requirements hidden away somewhere that backed up each of the user stories (you know the kind... Section 1.A.1.a.ii). As I began asking around the internet of other Agile BAs, I quickly found out that they weren't keeping that secret, that they weren't in fact keeping large amounts of detailed documentation. What they were doing was documenting acceptance criteria in place of those requirements.

To top off the acceptance criteria, some retain artifacts such as use case models or workflows to better describe the acceptance criteria as well as the overall "vision" of the product. This minimal documentation along with the developers communicating with the product owner, does more for a developer than a 75 page SRS can do. This is because something like a simple use case model or workflow diagram gives the developers context, and the acceptance criteria layout what happens within that context. The additional discussion that takes place with the product owner gives even more insight as to what the user story will require.

So all of this is to say that instead of documenting some missive that explains every exact detail of the system, requirements should now come in the form of a simple User Story statement with the acceptance criteria and perhaps a model or two. That's barely sufficient to get it done. The in-between stuff like implementation details, UI functionality, etc. can be obtained through direct dialogue and documented in Unit Tests. This brings up another point.. Things like implementation and UI probably shouldn't appear in the acceptance criteria. I say that for two reasons: 1.) because sometimes acceptance criteria are written way ahead of time, and 2.) you will often find better ideas of implementation and UI come from discussions between those that will develop them and those that want them (and yes, the BA is part of that dicussion quite often).

How a BA arrives to that User Story and acceptance criteria+models is the hard part to migrate to. I don't necessarily think that this is done using traditional requirement elicitation techniques, at least not all of them. It can, but something tells me there is a better way, and I think it will look more iterative in nature. The argument can be made that SRSs are iterative (just look at the change log at the beginning), but I'm thinking much smaller iterations still. This is the topic I will explore in Part Three.