The following is the second in a three-part series. For part 1, click here.
Part 2: Growing the Movement
In technology – and especially digital technology – the origins of a given innovation may be easy to pinpoint, but its often lightning-quick dissemination through society is usually cloudy. For this follow-up to part 1 of our series on Agile Software Development, we consider some key trends over the past two decades within the nebulous methodology; this is by no means an extensive look at the history of agile software development in the 21st century.
Agile as subject of research
Even prior to the accepted origin of the Agile Software Development philosophy, i.e. the release of the Manifesto for Agile Software Development in 2001, the drivers behind the movement were prescient enough to introduce the new concepts and ideas in academic and engineering circles. Thus before Agile software development was codified and branded as a new school of thought, a cadre of adherents was already forming.
Among the earliest academic-style papers was “The SCRUM Software Development Process” by Jeff Sutherland and Ken Schwaber, the first piece to codify any sort of Agile programming, was presented at a conference in Austin, Texas, in 1995. Since then, according to the paper “The Rise and Evolution of Agile Software Development” published in the journal IEEE Software in July 2018, “the phenomenal growth of agile practice is mirrored by agile research becoming a significant subdiscipline of software engineering in the last two decades and continuing today. In April 2018, a search for the [search term] ‘agile software development’ in Google Scholar for a period up to 2001 produced just over 13,000 results. The same search led to over 260,000 results for today….”
Further, “the rise and sustained growth of agile research has been chronicled by 19 years of the International Con-ference on Agile Software Development and 15 years of the North American Agile Conference, two of the largest dedicated annual agile conferences, and by numerous regional agile conferences and events around the world…”
And the steadily-growing world of Agile has also come to apply some principles of the methodology to the Agile software development industry itself, via the “State of Agile Report” released annually, which provides feedback to developers from within and without the community.
Scrum: First in past, first in present
As stated above, the Scrum methodology of software development goes back to the 1995 and the Austin conference; since then, Schwaber and Sutherland’s baby is now a cottage industry in and of itself, spawning books, lecture series, classes and seminars.
Like the early Agile movement of which Scrum’s creators were part, Scrum came about as a direct response to the old-fashioned “waterfall” style employed in all engineering disciplines and so square-pegged into software engineering work processes. As Sutherland noted in early published papers, Scrum is also a response to the spiral and iterative methodologies, systems deliberately designed to cover the shortcomings of waterfall methods, but still frustratingly linear for a world increasingly demanding quicker results.
Fundamentally, Scrum first defines all work processes as “theoretical”, i.e. those composed of fully defined actions and a linear timetable for delivering results, or “empirical” (a.k.a. “black box”), i.e. those creative elements which are determined by prevailing – and changing – conditions. The former essentially represents both waterfall and spiral methods, and the latter to Extreme Programming (XP) and other Agile methods.
The real genius of Scrum is in its combination of theoretical and black box processes, each firewalled from one another. Generally speaking, the scrum methodology calls for a series of three processes, with theoretical methods sandwiching a round of black box processes of free-form brainstorming, development, streamlining and quality control. It’s easy to see the appeal of Scrum and other Agile software development philosophies: business management types can receive their deliverables by schedule format whereas the development team need not feel constrained by a checklist of documentation to produce along with results.
Covid pandemic accelerates growth of Agile popularity
When the Covid-19 pandemic hit in 2020, few areas of the international workforce were able to rebound and adapt as quickly as those using Agile methodology in business management. According to the 2021 State of Agile Report, some 16% of the nearly 1,400 respondents stated that they were working for a fully remote workplace before pandemic conditions set in.
It’s the other responses to this particular bit of the survey that are most revealing, however. At that time, another 25% of respondents stated that they will remain fully remote while 56% expect to be working a hybrid approach combining virtual and in-office time. A miniscule 3% indicated that they would be returning to an office environment full-time post-Covid restrictions.
Even more incredible is the finding which revealed “significant growth in Agile adoption within software development teams, increasing from 37% in 2020 to 86% in 2021.”
The Agile philosophy in the 21st century has come full circle from revolutionary “Extreme Programming” to mainstream business philosophy to re-capturing the imaginations of software developers. This tendency had grown steadily for 25 years already, but necessity caused by the Covid crisis has led to innovative Agile software development leading the way for public and private enterprises.
