Agile Software Development and VUCA

A big part of software development coursework is examining the many variations on the software development lifecycle (SDLC). We look at what development models are ineffective and what models are most effective for their organizational and project-specific contexts. The methodologies we are most interested in (at least, in my Software Development courses), however, are those of Agile. In this post I will look at Agile from a high-level perspective and I will identify some ways that Agile addresses the Volatility, Uncertainty, Complexity, and Ambiguity of today's business world.

 

What is Agile?

 

I will only scratch the surface of Agile in this post. Agile is a methodology designed for software projects in organizations facing constant change (hence the name, Agile). Greene and Stellman (2014) define Agile as a set of methods and methodologies (i) as well as a mindset (ii) (Ch. 1).

 

i. Agile as a Set of Methods and Methodologies

 

The methods and methodologies of Agile facilitate project management, software design and architecture, and process improvement (Greene & Stellman, 2014, Ch. 1). Methods and methodologies consist of key practices that allow for teams to adopt Agile methods and methodologies (Greene & Stellman, 2014, Ch. 1). I will hopefully explore some of these in a future post.

 

ii. Agile as a Mindset

 

An agile mindset is a mindset dedicated to making planning, design, and process improvement the shared responsibility of the entire team. (Greene & Stellman, 2014, Ch. 1). Agile puts heavy emphasis on the visibility of processes and work and the regular sharing of information among all team members.

 

VUCA, Software, and the need for Agile

 

VUCA

 

Over and over in Organizational Behavior and Management and in Organizational Development (OD) we talk about VUCA – Volatility, Uncertainty, Complexity, and Ambiguity. In this age, VUCA is almost the norm for businesses across just about every industry for several reasons (and there are many more reasons than I present here):

 

i.                 External Factors

 

a.     Globalization – the expansion and integration of smaller economies into the larger global economy – has many implications for the way organizations ought to function to be successful is the global market. Cummings and Worley (2015) explain that while globalization offers new markets and opportunities for innovation and capital, risk is inherited by the economy’s global nature (p. 5). Risks such as Military or diplomatic conflicts and financial crises (such as 2008’s global financial turmoil) contribute to the risk assumed by global organizations (Cummings & Worley, 2015, p. 5).

 

b.     Information Technology is absolutely driving organizational change. In my Fundamentals of Business Information Systems course, I studied how several trends in Information Systems are greatly changing organizational structure. These four interrelated trends were pervasive network connectivity (the ease and scale of which devices are networked), the proliferation of mobile devices, cloud computing, and data science. Each of these trends alone is worth its own series of books but I can exemplify them concisely by examining one company: Uber. Uber’s service model works entirely over networked mobile devices which communicate with Uber’s cloud services to connect users to drivers and help them get rides as efficiently as possible. This is where data science comes into play; Uber’s data scientists and analysts are responsible for automating Uber’s core products with separate subteams for Driver, Forecasting, Global Intelligence, Maps, and more (Data Science & Analytics, n.d.). Still other teams are dedicated to ensuring safety and mitigating risk (Data Science & Analytics, n.d.). Cummings and Worley (2015) also point out the use of virtual teams (p. 5) – a phenomenon that has become ever more common across various organizational levels at my university and many other universities, schools, and businesses across the globe during the COVID-19 pandemic. (And if you’ve been interviewing for jobs, like myself, you’re probably well acquainted with initial screening interviews over Zoom or MS Teams). I study this stuff so I could go on and on about how IT/IS are driving organizational change (care to learn about IoT? :D). 

ii.                Internal Factors

 

a.     Quick and Nelson (2019) suggest that pressures for change are usually identifiable by some symptom such as a significant quarterly loss (p. 314). One cause for internal change, therefore, is a decreased effectiveness of the organization to turn a profit.

 

b.     Another internal force that influences organizational change is shifting employee expectations (Quick & Nelson, 2019, p. 315). These shifting employee expectations may be attributed to factors such as an increasingly educated workforce demanding more of employers (Quick & Nelson, 2019, p. 315). Increasing worker diversity can also prompt internal change (Quick & Nelson, 2019, p. 315).

 

VUCA and Software Development

 

Early Software Development

 

As former president of the IEEE Computer society and Hewlett Packard Enterprise distinguished technologist Michael Martinez explains in an IEEE Computer Society blog post, “Long before there were waterfall and agile methods to developing software, there was something of a Wild West” (50 years of Software, 2021). What development probably looked like during the early decades was this:

Figure 1. The build-and-fix model. From “The effect of requirement changes on the development of e-learning products.” by R. S. Bhattacharyaya and P. K. Das, 2013, 27(3), 643, p. 14.

 

There were not common principles or practices and programmers often lacked formal training, but possesed extensive domain knowledge and aptitude, often developing on large nonnetworked machines (Hoda, Salleh, & Grundy, 2018, p. 2). As Hoda et al. explain,

 

. . . software engineering developed as a discipline to provide engineering rigor to the profession. In the ‘70s, ‘80s, and early ‘90s the growth of software systems, range of domains of applications, number of developers, advent of the web, and diverse range of challenging software engineering problems resulted in a set of principles methods, practices, and tools to assist the engineering of such systems (p. 2).

 

They argue that the ”human side” of engineering software was being lost in the software engineering mainstream (Hoda, Salleh, & Grundy, 2018, p. 2). In other words, while structured processes and practices existed in the realm of software engineering, they did not emphasize the effectiveness of human interaction. Earlier software development lifecycle (SDLC) models, for example, were very rigid and sequence based, did not involve the customers or clients in much of the lifecycle, and did not allow for adequate risk management. Furthermore, software development was often conducted in more siloed work environments where developers and businesspeople did not have effective channels of communication. I will likely try to expand on different SDLC models including their drawbacks and their advantages in a later post.

 

The Emergence of Agile

 

In 2001, some of the current giants in the world of software engineering including Kent Beck, Alistair Cockburn, Martin Fowler, Robert C. Martin, Ken Schwaber, and Jeff Sutherland, among others, published the Agile Manifesto. In response to the weaknesses of pre-existing SDLCs, they sought to develop a more flexible framework for software engineering, retaining the rigor of engineering processes while emphasizing customer collaboration and the interactions of individuals and teams over processes and tools.

 

Agile: Addressing VUCA

 

I will by no means explain the ways in which Agile meets the needs of its customers (and its implementing companies) in exhaustive detail but I will concisely present some of the most striking aspects of Agile as laid out by Martin Fowler in an article he wrote about Agile in 2001.

 

Welcoming Change for Customers’ Competitive Advantage.

 

Agile is designed to welcome change, even late into development, rather than resist it (Fowler & Highsmith, 2001, p. 3). The nature of Agile processes makes it feasible for development teams to act on feedback from their clients to make changes late in the game. It’s easy to imagine why this could give customers a competitive edge.

 

Frequent Delivery.

 

Work in Agile is done in increments or iterations with multiple and frequent deliveries of parts of the software product (Fowler & Highsmith, 2001, p. 3). Agile advocates delivery of software every two weeks to two months with a preference for the shorter timescale (Fowler & Highsmith, 2001, p. 3). This frequent delivery process allows for a consistent feedback loop between development and clients to ensure the right product is being developed.

 

Collaboration with Business Partners.

 

Agile seeks daily collaboration between business partners and developers. In explaining the need for this collaboration, Fowler (2001) examines,

Many folks want to buy software the way they buy a car. They have a list of features in mind, they negotiate a price, and they pay for what they asked for. This simple buying model is appealing, but for most software projects, it doesn’t work (p. 3).

Instead of following the aforementioned model, Agile does not require detailed requirements up-front (Fowler, 2001, p. 3). Rather, to develop successful products, the development team will work with their business partners to adapt high-level requirements into working features of a software system through frequent interaction (Fowler, 2001, p. 3).

 

Pragmatic Communication.

 

Agile strongly favors face-to-face conversation over documentation. The main concern of communication, Fowler (2001) argues, is to give understanding which is better gained through conversation (p. 4). Particularly, it is because of the nature of tacit knowledge that communication should be favored (Fowler, 2001, p. 4).

 

Sustainable Development.

 

In being a people-centric development framework, Agile recognizes the need to maintain a sustainable development pace over the course of a software project. Fowler (2001) argues that longer work hours do not lead to higher productivity but more mistakes (p. 4). He advocates for a working pace of around 40 hours per week, so team members remain alert and creative throughout the lifetime of a project (Fowler, 2001, p. 4).

 

Self-Organization.

 

Agile philosophy states that “the best designs (architectures, requirements) emerge from iterative development and use rather than from early plans” and that innovation and creativity emerge the most from self-organizing teams with high interaction and few process rules (Fowler, 2001, p. 5). This view really underscores why Agile has a heavier emphasis on people over process.

 

Tuning Behavior.

 

Agile is about being flexible and that means that it emphasizes trusting people individually and as groups to monitor, reflect on, improve, and modify their processes to generate better results (Fowler, 2001, pp. 5-6).

 

Conclusion

 

I hope by now you have a high-level understanding of what Agile is and why it is useful as a set of methods and methodologies as well as a mindset for Software development in today’s business environment. Agile prioritizes people over process while preserving the rigor of engineering practices with the ultimate goal of delivering quality software increments to customers quickly while maintaining a sustainable pace.

 

 

References

 

Cummings, T. & Worley, C. (2015). Organization Development & Change. (10 ed.) Cengage.

 

Data Science & Analytics. (n.d.). Uber. Retrieved November 6, 2021, from https://www.uber.com/us/en/careers/teams/data-science/

 

Fowler, M., Highsmith, J. (2001, August). The Agile Manifesto. Software Development Magazine. Retrieved from http://www.sdmagazine.com/documents/s=844/sdm0108a/0108a.htm

 

Hoda, R., Salleh, N., & Grundy, J. (07 2018). The Rise and Evolution of Agile Software Development. IEEE Software, PP, 1–1. doi:10.1109/MS.2018.290111318

 

Quick, J., & Nelson, D. L. (2019). ORGB 6th Edition. (6 ed.) Cengage.

 

Stellman, A., & Greene, J. (2014). Learning Agile. O’Reilly.

50 years of Software: How It Began, Where It's Going. IEEE Computer Society 50 Years of Software Comments. (n.d.). Retrieved November 6, 2021, from https://www.computer.org/publications/tech-news/trends/50-years-of-software.