[Hint] If you’re using Waterfall project management, you’re doing it wrong. Deliver critical insights more quickly and with less effort and cost using Agile Scrum project management.
According to Gartner, Business Intelligence (BI) is an umbrella term for applications, tools, and best practices that enable access to and analysis of information that improves and optimizes outcomes.
Serving as a traditionally trained project manager, I have led dozens of BI projects – most delivered relatively on-time and within budget – all deemed highly successful at the time. But knowing what I know now about how to attack these types of projects, I admit that those “highly successful” projects I helped lead in the past were far less successful than they could have been. BI project management has evolved.
As a business leader, there are lots of reasons why you may want your organization to embark on a BI project:
- You know you have the data; you just need tools in place to leverage it
- You have grown rapidly and need to consolidate data from multiple ERP systems and possibly more than one General Ledger
- Your reporting is manually intensive and time-consuming
- Reporting is causing performance problems or data security concerns with your critical transactional systems
- You are experiencing “Excel Hell” – your managers are spending more time gathering and formatting spreadsheets than they are using the information
- You desperately need a Single Source of Truth (SSOT)
- All of the above
All of these reasons share a common goal: To gain improved insights (actionable information) from data. In other words, you want to discover what you do not know. Fortunately, there are lots of data warehousing solutions, reporting tools, and analysis tools available to help you solve these challenges. You can also enlist any of the multitudes of technology and consulting firms out there (like Saxony Partners).
But on their own, none of these tools and partners can help you know what you do not know. This can only be discovered through insights. Insights that will naturally point you to the need to learn more.
Under waterfall project management, BI projects typically play out like this:
- Analysis Phase: Assess the current environment and identify tools and techniques to address problems
- Requirements Definition Phase: Document a list of reports and objectives
- Design Phase: Document what each report will look like, how users will interact with them
- Build Phase: Build the documented solution
- Test Phase: Ensure everything works as documented
Once all the routines, reports, and tools are tested – then development work is done. The tools are integrated into the production environment for the user base to begin using. Time to celebrate, right? Maybe not.
As someone who has successfully executed the waterfall project management approach several times for several large organizations, I can attest that most ended in hearty handshakes and congratulations. But none were as successful as they should have been.
I led a data warehouse project for one of the world’s largest online trading firms where we defined over 100 standard reports and selected some very expensive latest-and-greatest graphical analysis tools.
I led a large data mart project for a public natural gas utility in which we defined and designed layouts for numerous online ad-hoc and structured reports.
I managed a data warehouse project for a major public manufacturer, which had grown via acquisition and was trying to run their business using two general ledgers. At the start of that project, we identified more than 75 standard reports they were positive they needed.
At the conclusion of each of these projects, my teams and I were applauded for delivering what we said we would when we said we would. In hindsight, I realize that all these projects could have been completed more successfully and quickly for far less cost if we had utilized an Agile approach.
Notice that the determination of what needs to be built is decided at the start of a typical waterfall project. However, the reason for any BI project is to gain insight into what is not already known. As you gain that insight, you will want to know things you did not know you wanted to know. Unfortunately, you cannot know what you really need to build until after you have already built some pieces.
There is a better way to structure BI projects. Rather than waterfall project management, Agile principles and the Scrum software development framework embodies quick releases of production-ready deliverables.
You can read more about the Agile principles over on the Project Manager blog, but here is the shorthand:
- Customer satisfaction through early and continuous software delivery
- Accommodate changing requirements throughout the development process
- Deliver working software frequently
- Collaboration between the business stakeholders and developers throughout the project
- Support, trust, and motivate the people involved
- Enable face-to-face interactions
- The primary measure of progress is working software
- Agile processes to support a consistent development pace
- Attention to technical detail and design enhances agility
- Simplicity is key
- Self-organizing teams encourage great architectures, requirements, and designs
- Regular reflections on how to become more effective
These principles, combined with user-ready reports and tools, can help you quickly gain valuable insight. This initial insight will often point you to questions that you did not know to ask at the start of your BI project.
Speaking of reports and tools, I have noticed from past waterfall BI projects that many reports and tools have gone unused. Conversely, there were always a few key deliverables that were particularly insightful. The latter were surprises, more-or-less.
Under the Agile-based Scrum project framework, we would have discovered critical insights earlier. Doing so would have set the stage for future releases – and we would have saved the time and cost that went into building unused reports and tools. Our clients would have uncovered their desired insight quicker and more cheaply.
Another fundamental component of the Agile-based Scrum framework is delivering production-ready code within time-boxed intervals called sprints. Sprints should never be longer than two months; most project teams generally target a two-to-four-week cycle. Each sprint focuses on the highest-priority items that can be delivered in that sprint. Throughout each sprint, new items and priorities are identified – and these are carried over into subsequent sprints.
Such an approach is why the Agile Scrum framework works so well for BI projects. As you learn and gain insights from earlier sprints, those insights are used to drive what is built in subsequent sprints. The right tools and reports are consistently prioritized, again eliminating the time and cost otherwise spent on unused resources.