Lean software development provides a firm foothold for the longer term
Successful organizations are adapting at an increasingly rapid rate and more profoundly than ever before in order to continue to provide value to today's and tomorrow's customers. The rate is increasing because of the exponential growth of options made available by technology and connectivity. This is also having a major impact on our conduct in an increasingly globalized community. Changes are becoming more profound because growth towards becoming a high-performance organization permeates into every nook and cranny of an organization's cultural paradigm and sometimes even its revenue model. The implications associated with stagnation or decline are self-evident – no one wants to buy from you, no one wants to deliver to you and no one wants to work for you.
IT can act as a partner to business in this respect and provide opportunities and options, assuming of course that its own value streams are in order and that it listens to customers and understands what they are saying. Although they may intend other-wise, lamentably enough many IT organizations often offer their customers a second-rate compromise stating that development of high-end applications is extremely time-consuming, that it is necessary to specify exact needs and requirements in advance (while customers are still uncertain exactly what they want), that implementing modifications during development is complex and costly, and that error-free software is non-existent. The use of lean software development principles and techniques is becoming increasingly important for many IT organizations. At the moment, it perhaps provides a competitive advantage, but there is every reason to believe that lean software development may rapidly become an essential skill. What does lean software development entail, how can it be applied and where does it lead?
Lean principles teach us that everything starts by viewing matters from the customer's perspective. It then becomes crucial for IT organizations to learn how to be adaptive and to learn on a continuous basis along with the customer. Critical to this ability are short feedback loops. The advantage of short feedback loops include:
- Improved ability to provide what customers actually want
- Improved systems while they are being developed
- Improved developer technical skills
- Greater team motivation
- Greater cohesiveness between stakeholders
In general, the shorter the feedback cycle, the faster an organization is able to respond to its customers and the sooner organizations can ascertain which innovations interest its customers. Ultimately, this is all that is relevant. The paradigm then becomes one of solid software builds, their rapid deployment, targeted evaluation of their business value, and either their improvement or rejection. Radically shortened post implementation feedback loops are beneficial to software teams insofar as they provide rapid insight into the perception and experience that customers have of the system's value. This can act as a springboard to further development. Feedback loops during development provide teams with an early-warning signal about unforeseen development costs relating to increasing technical debt or an application that is scarcely used and has lost its value even during implementation. They also provide an improved mechanism for teams to manage scope (including quality), planning and costs. This type of adaptive approach ensures that business problem-related IT solutions under development evolve in direct response to the feedback being received from the system's stakeholders.
In practice, matters are virtually unmanageable, often keeping software teams locked in an iron grip, e.g. fixed project budgets, pre-defined scopes, deployment dates cast in stone and zero options for putting customer feedback systems into place. It is common for organizations to stick to familiar territory by defining functions, roles and individuals in silo-like structures. Whereas most of the world is only five handshakes away from the US President, the engineer is often ten away from the actual customer.
The need to work differently can be felt, now that pressure is increasing to achieve shorter times-to-market and as uncertainty grows about what an application will ultimately look like, how it can best be built and how it will ensure that at a given point in the future the business will feel that it is being adequately supported. The fundamental premise is to transform ideas rapidly into applications, to measure how customers respond to the features provided, and to ascertain what is required to become or remain successful. IT should be fully capable of accelerating rapidly towards an adaptive approach incorporating short feedback loops, but all too often this is not the case. This is all the more reason to take a closer look at Lean Software Development (LSD) principles and practices.
The rip-roaring success of lean insights and techniques in the automotive industry does not mean that applying lean methods to software development and delivery is as easy. This is due in part to software's complexity, the intangible nature of data and information, and variable productivity levels among knowledge workers. Other challenges include understanding how to apply lean themes such as value, flow, pull and perfection to software while still under development. A multitude of causes undoubtedly lies at the root of such challenges. One possible reason may be that IT framework (ITIL® and CMMI®) availability and knowledge, and rapid adoption of agile software development methods have not directly influenced a sense of urgency in the same way as cost reductions have. Lean software development reinforces the invaluable elements embodied within these methods plus a focus on eliminating waste and evaluating value streams as a whole. LSD provides an appealing path-way towards improved product/service quality, faster and cheaper development, more meaningful knowledge worker deployment and even possibilities for a reduced carbon footprint.
LSD has not yet prescribed any best practices. Adoption and implementation of its principles and practices within an organization's culture and its way of working is primarily custom work requiring a learning-based, iterative approach. No one recipe exists for a lean transformation, but the foundations for a sustainable approach are being laid based on lessons learned within numerous IT organizations in the Netherlands and abroad. LSD came onto the scene only five years ago and is currently focusing on improving process flows for software development. A prime example of a typical lean success is Kanban software development in which the battle against defects and the batch-and-queue mentality is fought upstream into the requirements management domain and downstream in the form of continuous delivery. Anyone who thinks leanly experiences the power of the philosophy that dares to ask questions about every form of efficiency in current-day organizations regardless of their maturity and assumes nothing to be immutable. For some people, lean is a way of life!
Peter Drucker stated that the most important contribution made by management in the 20th century was achieving a fiftyfold productivity increase among the manual workforce, but that the most important contribution in the 21st century would be achieving the same productivity increase among knowledge workers. Any approaches or management techniques that are based on logic and man-as-machine behavior are denying the human condition. Human beings are emotional, social and group-oriented – individual behavior is susceptible to stress and fatigue. Software development takes place in an arena that is largely unpredictable and is characterized by fluctuations whose peaks and troughs cannot always be eliminated. The way in which people work together should ensure that teams can respond adequately to unforeseen circumstances. If the means are acceptable, then individuals will commit themselves to team objectives whenever their prospects (financial or otherwise) are commensurate with their individual contribution. This may be in terms of their professionalism, as well as innovation, creativity and opportunities for continuous improvement. Actual motivation lies in achieving a balance between autonomy in performing tasks, sense of purpose, and the opportunity to grow and excel. Furthermore, this balance is different for each individual.
Origin and Development
A set of general principles forms the basis to lean software development. The origin of these principles is widespread, ranging from Toyota Production System to Total Quality Management (TQM) and from Shigeo Shingo to Poppendiecks. The Lean Systems Society (LSS) based in Washington DC is dedicated to sharing lean knowledge and expertise. The LSS has developed a set of values and principles that can be useful for lean follow-up development and that are currently being put into practice by many advocates. LSS' message is that an IT organization's success lies in the acceptance of the fact that complexity and uncertainty are intrinsic to social systems and knowledge workers. This is why it is necessary to opt for a holistic approach and to derive knowledge and insights from multi-disciplinary sources. Creating a work environment based on respect for the psychological and sociological aspects of individuals and teams is an essential prerequisite for an LSD community. A group of people whose individual contributions are simply combined do not make a winning team. A winning team is made up of individuals who have committed them-selves to achieving a common goal. Several important aspects of LSD have been discussed below.
Optimize the Whole
A focus on only one part of a system of value streams ultimately results in sub-optimization and creates waste. This is why lean principles are a form of long-term thinking and start by paying attention to an organization's value streams and value propositions as a whole, whereby the software (application or embedded) is merely one of many components. It is important to realize that customers initiate both value streams and failure streams. A proper understanding of the actual nature of the customer's requirements and the variations therein should serve as the starting point for making improvements. Every effort to create software in a lean fashion is counteracted if equal attention is not paid to continuous integration and delivery of development results.
Genuinely reliable information about an IT project's progress is information about demonstrable, working software. Ultimately, it involves achieving these performance levels in a predictable fashion. The challenge is to keep learning on an ongoing basis and to develop the ability to respond to developments as they occur.
Respect for Others
Too much attention paid to social aspects causes paralysis and results in a situation in which nothing is achieved. Processes and people need to be synchronized and in balance. This is primarily a task for management. Respect plays an important role in this process. Respect is free and can achieve a lot. Organizations build long-lasting relations with people by respecting their suppliers, their customers, their staff and their managers. Respect is far more than a slogan; it is something that is genuinely felt by every individual within the organization. It is important that applicable rules and procedures be communicated explicitly and are perceived as being consistent. This also entails treating one another with respect and banishes paralyzing cultures of fear. Above all, a lean organization strives for ongoing and never-ending improvement and this entails allowing free expression and creativity for all staff members.
There is a famous scene in the first Matrix movie known as The Morpheus Proposal in which Neo, the main character, has to choose one of two options – a red pill or a blue pill. Either he opts to see the naked truth about the world around him or he opts to live on in ignorance. Morpheus states, 'The matrix is every-where. It is all around us. Even now, in this very room. You can see it when you look out your window or when you turn on your television. You can feel it when you go to work, when you go to church, when you pay your taxes. It is the world that has been pulled over your eyes to blind you from the truth.'
This is the case when learning to view waste as the squandering of scarce resources. Lean software development focuses on simple planning methods that lead to zero communication, coordination and rectification overheads. Visualizing quality and implementation contributes to efficiency – (Kanban being the best-known application for visualizing flow), limiting work in progress and to make throughput times ever shorter and more predictable on an ongoing basis. It is essential to outline invisible knowledge work by visualizing its flow until the value stream has progressed all the way from start to finish. Even more important than this is allowing people to communicate personally and directly with one another to work on the introduction of new features or process improvements, e.g. at the start of the day, or during sit-ins or other coaching interventions.
Improve Methodically and Continuously
Successful lean organizations constantly evaluate the conditions in which work is being carried out, and how LSD systems are performing, by gathering data and analyzing this information. A deeper understanding of variability and the fundamental causes of waste leads to a flood of improvement ideas that can often be adopted at a team level, but occasionally have to be escalated to senior management level depending on the impact and feasibility of a possible solution. Improving methodically and continuously entails basing improvement ideas on facts and analysis, and implementing them in a disciplined fashion. A plan-do-check approach is often sufficient to tackle simple opportunities. For more complex problems and existing processes, the Six Sigma DMAIC (Define-Measure-Analyze-Implement-Control) cycle is an option. Communicating and reporting improvements should be kept as clear and concise as possible. A3 Thinking and/or use of advanced DMAIC-Kaizen sheets support this endeavor. Scrum retrospective meetings can also serve as excellent input to a series of Kaizen events. Continuous improvement is an integral part of working and should be on the agenda every single day.
Show Lean Leadership
Leadership can be shown by anyone within an organization. Leaders define success and map out a path showing how others can make their individual contributions. They set the tone and tempo whether they have been appointed as the leader or spontaneously step up to the mark when the situation dictates. In larger, more complex organizations, leaders should be trusted and have the authority to take a lead in innovation, problem solving and continuous improvement. If you want great people at every level of an organization, then you also have to have great leaders at each of these levels. These leaders perform their work with passion and pursue change and innovation. They also set a good example by conducting themselves in a respectful fashion, as well as having the ability to recruit, develop and lock new talent into the organization. When time permits, great leaders make room for others by stepping aside and helping them find their own creativity and achieve their own personal success.
Fast, High-Quality Delivery
The prerequisite for fast delivery is a thorough under-standing of the value that customers require. This can be translated as functional and non-functional product and service requirements. Lean thinking can often appear counter intuitive. Empirical evidence has shown that drastically reducing software development throughput times significantly reduces the number of errors, improves quality and reduces costs. Customers are then more satisfied with the final result because it is more in line with their expectations. Traditional approaches to software development and delivery assume that freezing specifications prior to the start of development contributes towards improved quality levels, predictability and planning. Such an approach generally leads to reams of requirements or code that require testing, software errors that are never detected or detected too late due to a lack of feed-back, and major miscommunication and additional coordination efforts.
Delayed projects constitute a loss of value for the business due to the ongoing lack of a solution to its problems or challenges. The focus on time efficiency – one of the most important productivity factors for an IT organization – can be supported using easily applicable techniques such as value stream mapping and cumulative flow diagrams. An important aspect of LSD is the concept of building in quality at source. The later in the product's life cycle errors are found, the higher the risk and the higher the rectification costs. This is the reason behind implementing means to allow changes to features (or rather experiences) to be made more easily within system architectures. LSD also places an emphasis on testing as the ultimate proof of the formulated system hypothesis (testing is the specification) that is developed prior to or simultaneously with the code, as is the case in eXtreme Programming. A well-developed structure for test-driven development and automated testing (or rather specification by example) is a characteristic of successful lean software development.
A Firm Foothold for IT Organizations
While agile methods serve as good examples of lean thinking in action, LSD principles also provide IT organizations with a firm foothold for sustainable improvement over the longer term and across value streams as a whole. An IT organization is lean if virtually zero waste occurs during development and delivery of its products and services. It is also lean if the value perception of its customers results directly in greater levels of satisfaction and loyalty. Staff members feel highly motivated to help or to learn how to help their organization improve on a continuous basis. The lean journey is a path towards perfection and has no set destination – lean means always looking for improvement.
NOTE: This article was inspired by Leading Lean Software Development – Results Are Not the Point by Mary and Tom Poppendieck, 2010.