It takes a village
The old ways of software development went like this:
Business analysts, technical managers, architects, and key business owners would gather in a room and discuss over a many meetings what they wanted in a product. There was no involvement of the people actually doing the work until all the requirements were said and done. After the requirements were gathered, then it was the design document that needed to be scoped. After a few weeks/months on that, we finally got to development, testing, and finally production. That software development cycle is called Waterfall.
You can see how there could be many issues in this process. What happens when requirements change? What if you have the wrong people in the room? How come there are no business partnership past requirements? These are all issues that occur during waterfall. There’s a different approach and one that has changed how new products and services are rolled out. It’s called the Agile methodology. In agile we approach projects with faster, iterative cycles, and with partnerships with all the people in the project. Where am I getting at, you ask? Well, that same process can be used when building out a data product. Maybe that data product is a BI or data platform.
I’ve titled this post as “It takes a village” because it takes the effort of both technology and business partners to implement a successful data product. I am putting both BI and data platform in the same group because if you don’t understand your data (what you have and what you need), you won’t be able to empower users to use your BI platform.
Let’s walk through an example on what happens when the right people are not at the table:
The Tech department in your organization is planning on creating a data platform. You, a manager in the business analysis team, only hear whispers of it but you are not sure what is going on. A few months down the road, you receive and email from someone in tech asking for your team’s help to test the new platform. You volunteer because you are curious of what this means for you and your team. When you go test, the data is all wrong! it’s missing data that is important for your team. You realize that there was no quality test done on the data. Your trust on this new Data Platform is out the door!
This could have been easily avoided, if the right people were at the table. So, how do you know who are the right people? Putting your product or project management hat on, let’s think of this through the lens of the end user.
Survey the Users
Start off by figuring out who are the users of the current data. Is it finance, product development, accounting, human resources, sales, etc. Meet with every team lead. I know that may be difficult, especially with larger organizations. Starting with existing systems (database users or intranet site users) and gathering usage metrics will give you a starting point. If you are in tech, align yourself with your business analysts and start with them. They are most likely your biggest users as they use data as part of their job.
For each group you meet with, gather the following information:
list of data sources they use and whether that data is something the data platform will contain ( depends on what you are trying to accomplish with the platform)
Prioritized tasks/asks - You don’t need to build everything at the first go. Figure out what is important day 1 of launch ( aka the MVP)
Start small
Agile methodology teaches us that we implement changes in small increments. Figure out from your priority list and ask yourself where can I start? What is the foundation work that you have to do? Is it an ETL(Extract, Transform, and Load) framework to move data from one database to another? If so, how can you create that framework in small iterative increments?
Agile is all about taking that big end goal; that north star, and slicing it into bits that you can work on and release in a cadence. Why is that important? Well, if you remember what I said about waterfall, with agile you are no longer waiting months before your product is out in production, your end users can start using it and providing feedback sooner than in a waterfall method. Remember, it takes a village even if it’s a small one to make sure that your projects are successful.
Sharing is Caring
As part of your project, make sure that you are keeping the end users engaged. Once you have some working version of a product, show them. Demonstrate that their input matters. At the end of the day, they will be the users of the product that you build. But, be mindful of what is possible within your project timeline. Don’t over promise. Tell your team what is realistically possible so that no one thinks a requirement was missed. The “missing” requirements could just be future features. The key to this is communication and collaboration around what you are going to deliver.
Build a community
I sound like a broken record when I say that collaboration is key. In this case, I’m talking about a data village (if a village is not a community then I don’t know what is). Think about it this way, the BI/Data Platform you build will be used by many. If you’re a developer, you will contribute to the platform, if you are a business user, you will be using the tool everyday. You need to be able to communicate, collaborate and work together.
Promote the idea within the teams that it takes everyone to make this work. Have user group meetings within your organization that enable people to share what they are doing with the platform. Have a collaboration tool (messenger app/email distribution list/website) that allows all users to share and collaborate with each other. That will ensure user retention and adoption of your tool.
If you are building out a new data product, BI tool, or Data Platform or if you are just improving it, remember to go on this journey with your business users. The journey will not be perfect; it will have its struggles, but at least you are working through it together.