Agile Methodology When Processes Clash With Principles

Author: Narendran Thillaisthanam
Associate Vice President, Photon
In about a decade since Agile methodology's introduction, it has gained huge acceptance amongst software developers and IT managers. According to a recent Forrester survey, roughly 35 percent of its respondents used Agile methodology1. There is absolutely no doubt that Agile methodology has crossed the chasm.
While Agile brings in significant benefits and cost-savings to the consumer, it also comes with significant challenges. Agile methodology is built on several key principles that are fundamental to the methodology3. A lot of companies rush to adopt Agile methodology ignoring these key principles as well as the cultural and technical nuances associated with it. What results is not controlled chaos, but utter chaos and stress.

Cultural Challenge: One of the fundamental principles of Agile is the concept of self-organizing teams. Teams of software engineers are empowered in unprecedented ways, reducing the role of project managers (e.g. scrum master) to one of facilitator4. Motivation and team work are central tenets to the process.

When project managers that are used to command and control the style of management are entrusted an Agile project, they continue to push their heavy handed management style, which flies in the face of Agile’s self-organizing principle.

Similarly, software engineers who are used to strong oversight and management are tested by the sudden empowerment and freedom that comes with Agile. Lack of documentation, lack of fine-grained planning, and once a day scrum calls work when there is enough team-work and motivation. Agile is a mind-set and every team-member has an equal role to fulfill.

Contracts and Outsourced Software Development: Several outsourcing vendors enter into legal contracts with the customers where the payments are tied to traditional waterfall model. A sample contract could read, "25 percent payment for requirements completion, 25 percent for architecture completion, and so on."

If typical Agile methodology were to be followed, then the above contract does not make any sense. For example, requirements gathering will be completed only during the last sprint by which time typically 70-80 percent of the entire project would have been completed. Thus, the delivery teams are caught in a bind to interpret the above payment clauses and simultaneously execute projects in pure Agile sense. This often times leads to chaos.

Collocation, Pair Programming and the Importance of Teamwork: It is no secret that Agile promotes team work and collocation5. Based on this tenacious assumption is the notion that working software is preferred over reams of documents. The implicit assumption made here is the fact that collocated engineers will talk to each other as and when required. Informal communication is the underlying principle, than formal meetings, specification documents, and SLAs.

Many companies overlook the importance of small teams and collocation and continue to plan large projects with diverse locations. The overhead of ‘educating’ team members in diverse locations on new changes (especially when there is sparse documentation) is a daunting task. Underestimating this aspect results in chaotic product development and causes a lot of stress.

Staffing and Agile Methodology: Agile is a front-loading process and mandates that all key contributers are assembled up-front. This is in sharp contrast to a waterfall model where staffing follows a normal curve. The following diagram illustrates the difference in staffing across Agile and traditional waterfall methodology.

Several project managers who come from traditional methodologies underestimate the significance of this and try to staff the team in a pure waterfall style. This results in project delays and sprints not meeting the goals.

The Definition of Shippable Product: The definition of task completion is clear in Agile6. Agile mandates that across every sprint, the team completes a 'shippable' product. Thus, from the very first sprint, customers can get a deployed version that they can actually make use of.

This is another area where there is continued misunderstanding. Engineers continue to claim features to be 'complete', whose meaning is not clearly established (that is, in true Agile sense). Project managers (read scrum masters) fall pray to this age-old conundrum and they continue to track feature completion without passing the key litmus test, namely shippability. ‘We can demonstrate feature X’ is very different from feature X ‘working’ at customer deployment and the customer is ‘educated’ to work with that feature.

Change Management and Sprint: It is true that Agile methodology can adapt to changes more easily than other methodologies. The cost of change curve diagram below actually indicates that Agile methodology has a flatter cost for changes compared to traditional waterfall methodology. But, it should be noted that the cost of change is not zero.

Several customers engage with vendors with the assertion, "We are still gathering requirements with our end-customers and thus we anticipate a lot of changes along the way. We prefer that you execute the project in Agile to incorporate these changes." While there is nothing wrong with this statement and this is perhaps a way of life in modern software development, the problem comes when the stakeholders and project managers push the change management to the extreme.

Agile recommends that within a sprint changes are well understood and nailed down7. It does not say that you can change requirements within a sprint. Agile recommends that if there are major revisions in requirements within a sprint, the sprint should be abandoned and a new one constituted.

A second pertinent point is that Agile promotes the notion of shippable product. So, every sprint is supposed to move the product towards the shippability continuum. Thus, changes to earlier sprints are perhaps as expensive in Agile as it is in the waterfall model (perhaps even more). Not understanding this crucial difference can make change management equally hard in Agile.

Conclusion: Agile methodology is here to stay, no doubt. However, as more companies rush to adopt Agile, they ignore the key principles and tenets that are behind the institutionalization of Agile methodology. This article highlights some of the common pitfalls that companies have to be aware of as they roll out Agile methodology within the organization. By far, the biggest challenge seems to be cultural, where the incumbent development methodologies clash with the Agile mindset. This problem is exacerbated when roll-outs happen without adequate training and when there is a lack of emphasis on shift in mindset.
Previous  article
Next article
Write your comment now

Email    Password: 
Don't have SiliconIndia account? Sign up    Forgot your password? Reset
Reader's comments(2)
1: From: Mrs. Mary David

This mail may be a surprise to you because you did not give me the permission to do so and neither do you know me but before I tell you about myself I want you to please forgive me for sending this mail without your permission. I am writing this letter in confidence believing that if it is the will of God for you to help me and my family, God almighty will bless and reward you abundantly. I need an honest and trust worthy person like you to entrust this huge transfer project unto.

My name is Mrs. Mary David, The Branch Manager of a Financial Institution. I am a Ghanaian married with 3 kids. I am writing to solicit your assistance in the transfer of US$7,500,000.00 Dollars. This fund is the excess of what my branch in which I am the manager made as profit last year (i.e. 2010 financial year). I have already submitted an annual report for that year to my head office in Accra-Ghana as I have watched with keen interest as they will never know of this excess. I have since, placed this amount of US$7,500,000.00 Dollars on an Escrow Coded account without a beneficiary (Anonymous) to avoid trace.

As an officer of the bank, I cannot be directly connected to this money thus I am impelled to request for your assistance to receive this money into your bank account on my behalf. I agree that 40% of this money will be for you as a foreign partner, in respect to the provision of a foreign account, and 60% would be for me. I do need to stress that there are practically no risk involved in this. It's going to be a bank-to-bank transfer. All I need from you is to stand as the original depositor of this fund so that the fund can be transferred to your account.

If you accept this offer, I will appreciate your timely response to me. This is why and only reason why I contacted you, I am willing to go into partnership investment with you owing to your wealth of experience, So please if you are interested to assist on this venture kindly contact me back for a brief discussion on how to proceed.

All correspondence must be via my private E-mail ( for obvious security reasons.

Best regards,
Mrs. Mary David.
Posted by: mary lovely david - Monday 26th, September 2011
2: Hi my dear,
My name is Mounace, i would like to establish a true relationship with you in one love. please send email to me at ( i will reply to you with my picture and tell you more about myself. thanks and remain blessed for me,
Your new friend Mounace
Posted by: mounace love love - Thursday 09th, June 2011
More articles
by Kaushal Mehta - Founder & CEO, Motif Inc..
The retail industry is witnessing an increased migration of customers from traditional brick and mortar retail to E-commerce (online retail)...more>>
by Samir Shah - CEO, Zephyr .
You probably do because you are on the phone with them! For all of you working in some technical management capacity here in Silicon Valley,...more>>
by Raj Karamchedu - Chief Operating Officer, Legend Silicon .
These days are a mixed bag for me. Of late I have been considering "doing something bigger and better," in my life, perhaps seriously though...more>>
by Madhavi Vuppalapati - CEO of Prithvi Information Solutions .
IT Services Rise of Tier II companies The Indian IT outsourcing industry is going through very exciting phase in its business life...more>>
by Bhaskar Bakthavatsalu- Country Manager, India and SAARC of Check Point Software Technologies.
Data loss occurs every day through corporate email. In fact, given the sheer number of emails an organization sends every day, data loss inc...more>>