What Is Agile Software Development?

Agile and agility are two commonly used words in software development today. Agile originated with the “Agile Manifesto” which is a set of values and principles laid out by a group of software development leaders in 2001. Those values and principles have had a profound impact on the way software development teams and, increasingly entire organizations, work together to deliver value to customers.

While agile started in the technology industry, increasingly, business and management leaders talk about the need for organizations to be agile. In fact, according to the State of Application Development Report 2019, agile is the primary approach companies are investing in to accelerate application development.

Approaches to speed up app delivery - state of app dev

For these companies, agile is viewed as a management approach which can help organizations drive digital transformation efforts. For many business organizations, it represents a desire to work faster and more efficiently while being more responsive to customer needs. As a result, cross-functional teams use agile software development methods to define how to work together.

When people talk about agile software development methodologies, they often refer to approaches focused on: 1) working in short iterations, 2) meeting frequently for short periods, 3) visualizing their work, 4) collaborating across business and technology functions, and 5) building applications as small functional components.

Benefits of Agile Software Development

Traditional software development used a “waterfall” approach where teams would define everything at the start of a project, then build it, then give it to customers. The problem with that approach, especially in today’s fast-moving, digital world, is that by the time customers receive the product, tastes and technology often change making products obsolete.

In contrast to the waterfall approach, agile software development methods aim to define the specific software requirements weeks or days before it is built rather than defining everything ahead of time. This approach maximizes the advantages of agile software development to ensure that the team can incorporate changes and new ideas, which deliver value to the customer, into the product quickly and at a lower cost versus traditional approaches.

In addition to the increased customer value delivered with agile, it provides many other benefits to improve how teams work. Agile requires teams to work together differently. Specifically, agile enables high-performing teams to increase internal and external collaboration while achieving greater decision-making flexibility. Because IT and business people are working closely together, the improved ways of being agile spill over into other areas of the organization, further multiplying its benefits.

When to Use Agile Software Development

Most teams benefit from using agile software development methods regardless of the nature of their work. Agile is especially beneficial when teams are creating a new product or process or when teams are focused on driving organizational change. In some cases, a traditional waterfall approach may be best, for example, when implementing a packaged software solution which requires adherence to clearly defined, sequential steps.

Each team should use their own knowledge, experience, and expertise to determine the best approach, whether agile or waterfall, to achieve success, but should be careful not to fall back on traditional approaches because they are easy or familiar.

Agile Software Development Methodology

The benefits seem clear, but what do we mean by agile software development methods exactly?

Teams adopt a wide range of strategies and approaches to implement an agile software development methodology, and call themselves “agile”. However, in the case of software development, people frequently use these terms incorrectly. They confuse the broadly defined values and principles which guide our behavior in the Agile Manifesto with specific frameworks, like Scrum and Kanban, or customized approaches which bear no resemblance to agile.

The real value and beauty of agile is that there is not a single, prescriptive guide or framework for digital teams to follow in order to achieve agility. Agile is not a single book or training course which will magically transform your team.

While many great frameworks and methodologies exist to help you along the way, teams can find their own agile software development process and adapt it to meet the unique needs of their organizations, industries, and customer needs.

Step one in achieving agility requires establishing the correct mindset. Agile requires flexibility and trust and open mindedness intensely focused on the goal of achieving customer value frequently.

Building High-Performing Agile Teams

Building high-performing, customer-focused agile teams does not happen by accident. It starts with finding individuals with the right mindset to succeed with agile. Team members on a high-performing agile team answer an emphatic “yes” to the following questions:

  • Are you flexible and open to change?
  • Can you collaborate across functions?
  • Can you learn continuously?
  • Do you have a level of trust?
  • Can you communicate openly and honestly?
  • Are you comfortable being part of a team?

Before starting a project, take time to find the best individuals who have the right mindset so you are able to quickly turn them into a high-performing agile team.

Once you have the right individuals, it’s critical to define, as a group, your shared goals and expected way of working based on agile principles. With that framework in place, teams can start their Tuckman’s Stages of Group Development.

As they evolve through the Forming, Storming, Norming, and Performing stages, agile teams share many similar characteristics and have found ways to:

  • Understand and adapt the agile values and principles
  • Build self-organizing teams; let the people doing the work define how to do the work
  • Regularly stop to reflect on how to improve their process, collaboration, and quality
  • Create an environment where collaboration includes business, non-technical, and technical points of view
  • Have the discipline to keep things simple and not build things that are not needed

To support their evolution from Forming to Performing, agile teams need to define team practices like: ways of working, technical practices, and scaling strategies. Agile teams frequently use one of many existing frameworks to support those practices or choose to define their own practices to meet their unique needs.

Introducing Agile Software Development Frameworks

Agile is not a single framework or methodology. There is not a single book that you can read or steps you can follow to “become agile”. This is both the challenge and the opportunity to achieve success with agile.

Achieving agility requires a deep knowledge of a wide range of frameworks, strategies and approaches. It also requires the ability and willingness to experiment with new ways of working and to learn from those experiments. The learning outcomes enable agile teams to define their customized version of agile that enables them to succeed.

Agile software development uses a wide variety of methodologies and frameworks. At a high-level, the frameworks help agile teams determine: ways of working, technical practices, and scaling strategies. Some of the most common frameworks agile teams use include: Scrum, Extreme Programing (XP), and Lean Agile. Many agile teams are using techniques and practices from all three of these approaches whether they realize it or not.

Most Common Agile Software Development Frameworks

As agile teams begin their forming stage, they often look at frameworks to organize how to work. Selecting a framework as a starting point enables teams to define individual roles, plan work schedules, and organize team collaboration. Over time, several frameworks for organizing how to work have emerged.

The following graph from the 13th State of Agile Report shows a sample breakdown of Agile Methods and Practices Agile Teams employ globally. 

Let’s take a quick look into each of these frameworks.

Scrum

Scrum is a popular framework for solving complex problems and providing a structure for teams to deliver high value software products. Scrum describes a set of roles, events, artifacts, and rules that help teams organize and execute their work. Scrum is built on the pillars of: 1) transparency, 2) inspection, and 3) adaptation and defines five key values: commitment, courage, focus, openness, and respect.

This framework is one of the most popular for agile teams (see chart above).  Newly formed agile teams frequently start with Scrum because it defines an easy to understand framework for defining and organizing teams. However, building a high-performing Scrum team takes time and effort to understand and implement the principles in order to maximize the value of the framework. As the Scrum Guide states, Scrum is: lightweight, simple to understand, but difficult to master.

If you want to learn more about Scrum, I invite you to take a look at my previous blog post Agile and Scrum: Understanding the Differences.

Kanban

Kanban is not a methodology or framework, however teams frequently use elements of Kanban to improve their performance. Kanban is a Japanese word meaning “signboard”. The “Kanban Board” is widely used by agile teams to visualize and provide transparency of the team’s work. Each work item is typically represented by a card, sometimes with a user story, requirement or task, with specific information about the work to be done. The Kanban approach uses a “pull system” to move work through a process so work starts on an item after another item is finished. Kanban does not define a fixed period of time for the individual work item or phase of work. 

Kanban is based on five principles: visualize the workflow, limit work in process, manage the flow of an item through a system, make process policies explicit, and improve collaboratively. In addition to visualizing their work on a Kanban Board, teams focus on limiting the work in progress per person and team to set limits to reduce the inefficiency of multitasking and context switching.

Scrumban

As its name implies, Scrumban takes parts of Scrum and Kanban to form a hybrid approach. Often, agile teams start with Scrum and apply Kanban principles on top of it to improve performance and efficiency. Scrum provides the team, event, and artifact structure while Kanban supplies techniques for improving the flow of backlog items through the process.

Implementation of Scrumban can vary widely. For example, some teams may use the team and artifact structures of Scrum, but modify the events in Scrum and eliminate the strict timebox definition. Likewise, teams may choose to use traditional estimation techniques, like planning poker, while others, working outside the timebox construct, may use high-level estimations.

Scrumban is often viewed as a natural progression from the more structured approach defined in Scrum to the self-organization fostered by Kanban.

Lean Agile (Lean Software Development)

Like Kanban, Lean Agile is not a framework itself. Lean Agile principles are often used in conjunction with Scrum, Kanban, and other techniques to improve the efficiency and performance of agile teams. A key goal of Lean Agile is to improve efficiency by removing waste from the agile software development process.

Lean development is based on seven principles:

  1. Eliminate waste
  2. Amplify learning
  3. Decide as late as possible
  4. Deliver as fast as possible
  5. Empower the team
  6. Build integrity in
  7. Optimize the whole

These principles provide agile teams additional tools, techniques and perspectives for improvement.

Teams, at all levels of agile maturity, benefit from applying Lean Agile techniques. For new or maturing teams, principles like deciding as late as possible and empowering the team are important foundations on which to build. As agile teams mature, they benefit from working to eliminate waste and amplify learning to reinforce the continuous learning cycle characteristic of high-maturing agile teams.

Extreme Programing

Extreme Programming (XP) is a development methodology focused on building in short development cycles to reduce the cost of changes in requirements. XP emphasizes customer satisfaction and defines practices for developers to accommodate changing customer needs to maximize value. Most of all, XP emphasizes teamwork and enables key stakeholders: managers, customers, and developers to work together to deliver great solutions.

Extreme programming is based on a set of values, principles and practices. The five XP values are: simplicity, communication, feedback, respect, and courage which guide the overall Team interaction. XP Principles include: feedback, assume simplicity, embracing change; these principles provide guidance on the overall approach to building a software product.  

The number of agile teams using XP exclusively is relatively small (link to 13th State of Agile Report), but many of the functional and technical practices Kent Beck and others defined in XP are widespread. As a result, agile teams frequently use Scrum, for example, to organize their teams, events, and artifacts, while using XP rules to drive their product implementation.

Other Agile Frameworks

Behavior Driven Development (BDD): A software development methodology that relies on continuous communication between developers, QA, and business stakeholders, so features are well understood before development starts. BDD applies the Five Whys Principle to each user story to ensure a business outcome connects with the purpose of the user story.

Test Driven Development: A process that uses small, specific test cases in short development cycles to build applications. Developers start by writing test cases, running those tests and seeing what fails. The developer then writes code and reruns the tests. If the test passes, the developer moves to the next test; otherwise, they continue to update the code until the tests pass. The goal of this approach is to focus first on ensuring any code added to the application passes the appropriate tests.

Feature Driven Development: A model-driven, short-iteration process that consists of five basic activities: develop overall model, build a feature list, plan by feature, design by feature, and build by feature.

Hybrid / Home Grown Systems: In addition to the frameworks described above, teams use a variety of other approaches. Those hybrid structures often combine elements of multiple frameworks into a methodology specific to a company or team. 

These hybrid approaches often fall into a few broad categories: a combination of traditional waterfall and Scrum (so called Scrumfall), iterative development using a waterfall approach (but with shorter iteration cycles), and Dynamic System Development Method.

The degree to which these approaches achieve agility varies widely depending on the skills of the team, the organization’s culture, and solution complexity.

Agile at Scale

The original agile frameworks, principles, and methods were designed for small teams or small groups of teams. Management, customers, and teams realized the need to scale the successes from small agile teams to large groups of teams across the enterprise. To achieve this goal, new frameworks for scaling agile have emerged. Those agile at scale frameworks include: Nexus, SAFe, Scrum at Scale, LeSS, Disciplined Agile, and others. My colleague, Rick Warner, recently wrote a blog post called Scaled Agile - What is it and Why You Need It covering this topic.

You can see in the following chart from the 13th Annual State of Agile Survey, that this is an emerging discipline with many agile Teams not clear on their strategy (“Don’t Know”) or using internally created methods. 

As the data shows, SAFe and Scrum of Scrums are two of the most popular frameworks today.  Many agile teams like a Scrum of Scrums, Enterprise Scrum, and Nexus because they allow teams to leverage the existing skills, training, and infrastructure of Scrum in their organizations.  Nonetheless, agile at scale is a rapidly growing and maturing segment of agile software development space with many changes expected in the coming years.

Conclusion

Agile software development has matured since 2001, but is still a discipline where agile teams can continually learn and improve. The key is to understand that achieving agility is a constant journey of improvement and responding to change. Developing a deep understanding of your current frameworks is key, but remembering that achieving agility goes beyond mastering narrowly defined activities.

To achieve the real value of agile, teams need to understand the unique qualities of each team, organization, product, and project to determine the best approach to a given product development assignment. Agile teams need to look broadly at the frameworks, methods, and tools available and apply those to current circumstances to stay intensely focused on the goal of achieving customer value frequently.

Looking For More Content About Agile?

If you’re looking for more content improve your business agility, here are a few great places to start: