Success is a multifaceted and sometimes elusive concept. It is definitional. Its meaning varies from one situation to another and among different stakeholders.
It’s important to define what success is at the start of your effort, whether launching new site features, testing, or reaching business goals. In my experience, if you wait to define success until “when you see what happens” you can easily end up with conflicting opinions about the project’s outcome. For example:
- One can have a product of significant interest to users but has poor usability.
- Alternatively, your application’s interface could be perfect, where every user finds what they are looking for and no user ever makes a mistake. However, that application only calculates yesterday's date. It is likely that few will ever care about that. In this case, your usability measures are great, but your product interest is low.
- Your links can generate great click-throughs but lead the users to a dead end. Your CTR is great, but your overall performance is poor.
- A product can generate a great deal of activity but contributes little monetary value to the business.
Different success definitions answer different kinds of questions. Did the product meet its revenue goal? Are users engaged with the tool? Did recent changes to the links increase their click through rate? Complicating things, the same metrics (Unique Users, Page Velocity, Repeat Use, etc.) are often used to measure the different types of success. Those metrics may have a different meaning depending on the success question that is being asked.
In deciding on the success measures for a project, it helps to be aware of what type of success is being goaled. In my mind, there are 5 common optimizations which we measure. These are just different types of success:
- Business optimization
- Feature optimization
- Campaign optimization
- Process optimization (usability)
- System optimization
Business optimization
Business Optimization is concerned with the economic success of the company and the metrics are highly correlated to business revenue or strategic goals. The bottom line is the key driver of success. For example, publishing sites might measure success in terms of ad page views, sponsorship visitors, or paid subscriptions. In contrast, e-commerce platforms primarily focus on sales and profit margins.
Feature optimization
Feature optimization is concerned with the success of a specific product, or application, or a feature within an application. Metrics are focused on customer interest in the product and its parts. Is the product something your customers want? Is the feature set complete or needlessly complex? Feature optimization is about fine-tuning the product to meet customer expectations and demands.
Campaign optimization
Campaign optimization is concerned with maximizing the effectiveness of a discrete marketing effort over some period of time. These are the marketing measures used to track reach, acquisition, etc. SEO fits into this category. An optimized campaign ensures that the campaign reaches its intended audience and the audience coverts.
Process optimization (usability)
Process optimization delves into user interaction with a product or application. Optimization enhances the user experience by improving the ease of using the product. Can users efficiently operate the tool? Can they find what they are looking for without frustration? Process optimization aims to create a seamless and user-friendly experience.
System optimization
These are used to monitor the operational conditions of the site. For example, was there a sudden drop in traffic due to server failure or an increase in 404 errors? What is the load the CMS is handling? Is the video platform working as expected? Is the page load time large or small? System optimization helps ensure the overall health and functionality of the digital infrastructure.
Understanding the type of success measure that aligns with your project's goals is crucial. Different stakeholders may be interested in different measures of success. The type of success measure that is of interest often tends to map to the responsibility of the “metric consumer”. A person responsible for programming links might tend to look at process and campaign measures while a product Vice President might tend to look at business and feature measures. Planning how you will measure project success ahead of time will help account for these different needs.
Care should be taken to identify what type of goal you are measuring, not just the metric used to measure that goal. This can help avoid confusion both for you and others in the organization. Often you will be looking at more than one measure of success. With multiple goals, you may get different outcomes for the different types of success. In that case, you will need to have prioritized your goals.
In conclusion, success is not a universally defined concept. It takes on different meanings depending on the context and goals of a project. The five common dimensions of optimization—business, feature, campaign, process, and system—offer a framework to identify and measure success effectively. Recognizing the type of success you are targeting and selecting the right metrics for your goals can help ensure clarity, prevent confusion, and enhance your chances of achieving success in any project.
