Two of my favourite pyramids are the Data Science Hierarchy of Needs and the Minimum Viable Product. Combining them helps us build effective artificial intelligence (AI) proof of concepts in businesses. It also supports building AI competency at the same time as demonstrating Return on Investment (ROI).

TL;DR

• Combining the AI Hierarchy of Needs and the Minimum Viable Product gives us a visual way of describing organisation competency, direction, and indicative workload.
• Reaching higher tiers of the AI Hierarchy of Needs is harder when the lower tiers are missing or poorly implemented.
• When we build Minimum Viable Products we should be delivering working solutions that delight, fulfil a need, and provides learning or capability feeding into future work.
• Building MVPs in data science and AI when these are new competencies differs from an MVP software project build where all competencies exist.
• To minimise the risk of failed data science and AI MVPs, deliver a data and business intelligence MVP first and consider strengthening that competency before moving on to the next.

The Data Science Hierarchy of Needs

Monica Rogati introduced the Data Science Hierarchy of Needs in the 2017 Hacker Noon article, The AI Hierarchy of Needs. Rogati uses the pyramid to explain that like in Maslow's Hierarchy of Needs, the essentials are required before you can move towards the ultimate goal.

This is entirely true.

You cannot build data science products, or AI products, that your staff can trust if you don't use data they can trust. Proving your data is safe is the basis upon which your entire use of AI will rest.

Additionally, you don't need algorithms like deep learning for all analytical or predictive tasks in the organisation. In most businesses, simpler algorithms will be far more widespread as they can take less time to implement, granting important breadth of coverage. They'll escalate the complexity of the algorithms in use to gain incremental benefit beyond what the simpler implementations offered.

Simpler is usually better.

The AI Hierarchy of Needs

The Minimum Viable Product

Popularised by the Lean Startup, the Minimum Viable Product (MVP) is:

The minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort Eric Ries

The MVP has become the staple of software engineering; partly as it helps frame a definition of "done" for early work and gives defined reflection points, and partly as when we called things prototypes they lived forever anyway!

One of the pitfalls that software engineers would fall into when building an MVP is focusing on the basics of the breadth of functionality, but not spending time on the what makes sites and applications usable, like User Experience (UX).

Alternatively named the Minimum Delightful Product, the aim for an MVP is to build something that meets expectations and minimum quality whilst showcasing the core functionality.

It shouldn't do a huge amount, but what it does do should be enjoyable to use.

The Minimum Viable Product explained visually by @jopas

Combining the AI Hierarchy of Needs and the Minimum Viable Product

I've discussed previously an organisational AI competency model that describes for manufacturing the ability to use increasingly sophisticated algorithms to support the business. Each one has increasingly more stringent data management requirements.

Unlike with the MVP pyramid where the top tier must also be included we can derive value from slicing our pyramid a number of ways.

A simplified version of Ragoti's AI hierarchy of needs with Data storage at the bottom, then Data pipelines, then Business Intelligence, then Data Science, and finally AI

Business Intelligence

If business intelligence (BI) is new to your organisation, then being able to work out what happened and when in an area of your business is the first MVP you should be building.

The BI MVP delivers on three tiers - Data storage, Data pipelines, and Business Intelligence

Doing enough data storage, cleaning, and reporting in an area of the business should show ROI in terms of how problems can be identified sooner, and decisions can be made based on recent patterns of activity.

This MVP might be a single department, but if it proves valuable there's a whole tranche of activity there in rolling out similar BI MVPs across the business until a complete view of the business is possible.

Many organisations have developed a robust, comprehensive data storage and data pipeline solution to support business intelligence across the organisation.

Many businesses have gone through this process of building a comprehensive view of their organisation. This typically never reaches 100% coverage as businesses are constantly innovating, changing, and adding new data sources, but that's a topic for another post.

Data Science

For a data science project you might be able to build your MVP in a department or area that already has the data part sorted. If you're trialling a new area, however, you may need to include some data collection, cleaning, and monitoring dashboards too.

The Data Science MVP needs data storage, data pipelines, and business intelligence to be successfully delivered.

ROI of a data science project usually comes from insights that cause people to amend processes, or they provide a means of prediction inside an existing product or activity that improves something like profitability.

Like with BI, you can work towards rolling out data science techniques across your organisation. This typically has decent gains and by working on making improvements across many departments, you can start seeing a virtuous cycle.

Artificial Intelligence

Some business challenges need AI; it could be needing to recognise brands in videos, translate text, or personalise content. There are many challenges that machine learning techniques like deep learning will be more effective at than other tools in your analytical toolbox.

One way of going about an AI MVP is to buy an off-the-shelf AI solution, like Microsoft Cognitive Services, to perform a task like text translation for you. This is a great route if you do not need something custom.

If you do need something tweaked, there are also customisable options inside these off-the-shelf products, but they will require data. This brings back your AI MVP to needing a solid foundation.

The off-the-shelf AI MVP needs data storage, data pipelines, and business intelligence to be successfully delivered.

If you're implementing an off-the-shelf/customised AI MVP you can avoid a data science component to the project. You shouldn't neglect a BI component as you will not know what impact your AI MVP is having.

If you need to build something bespoke, then you will need to include some data science work to either help in the development of the more sophisticated solution, or to provide a baseline for measuring ROI.

The bespoke AI MVP needs data storage, data pipelines, business intelligence, and data science to be successfully delivered.

Reducing project risk

Attempting to go from no organisational capacity, to building a bespoke artificial intelligence minimum viable product, is TOUGH. Needing to get many tiers working at once is hard and makes the time to MVP longer. This differs from the software MVP, where each tier is a skill or capability a qualified engineer or group of engineers should already have.

If you get experienced consultants to build your bespoke AI MVP and they have to work on all the tiers, then you'll pay AI consultants to do work that can be done much more cheaply. Additionally, you might not be able to support their work due a lack of internal skillsets.

If you have analytical staff internally, then trying to learn multiple new skills simultaneously makes things more difficult, and increases chances of a botched project.

My preferred route is to build incrementally, gaining value at each step.

The bespoke AI MVP is better delivered in incremental MVPs, starting with a BI MVP, then a Data Science MVP, then an AI MVP.

Each MVP minimises the lower tier work needed to support the new tier's MVP. Attaining each new tier is where most of the learning should be for an organisation growing competency in Data Science and AI. The incremental MVPs, therefore, balance the need to validate learning, realise ROI, and build trust across the business.

Conclusion

Combining the AI Hierarchy of Needs and the Minimum Viable Product gives us a visual way of describing organisation competency, direction, and indicative workload.

Reaching higher tiers of the AI Hierarchy of Needs is harder when the lower tiers are missing, or poorly implemented.

When we build Minimum Viable Products we should be delivering working solutions that delight, fulfil a need, and provides learning or capability that feeds into future work.

Building MVPs in data science and AI, when these are new competencies, differs from an MVP software project build where all competencies exist.

To minimise the risk of failed data science and AI MVPs, deliver a data and business intelligence MVP first, and consider strengthening that competency before moving on to the next.

If you'd like to discuss suitable MVPs for your business, you can book a chat with me using my booking link.