Clarify win-loss speculation with post-mortem interviews

Flickr image source: http://tinyurl.com/ngyurwt

After yet another desperate claim from an exasperated Sales team that we were losing deals because of presumed product gaps, I decided that I needed hear it first hand from the customers.

I look forward to the days when I can sit with our Sales team on a regular basis (at least quarterly) and formally walk through our wins and losses. I say this somewhat selfishly because I have seen our company overreact repeatedly to lost deals. There is a tendency to find scapegoats and Product is often an easy target.

I know we have sufficient features with clear benefits to close new deals (our win rate is actually pretty high) but it still stings when a customer chooses a competitor. Nobody likes to lose but we need to reflect on these events in a productive way. We need to be truthful with ourselves and learn from our losses.

What drove this decision

A battle was already brewing between Sales and Product around a set of "must-have features" that had been distracting me from the real roadmap priorities.

In fact, I had been cornered a number of times over the past few months, typically by a Sales manager stomping around the office, still sore after getting the bad news that we lost another opportunity.

But warning bells were going off for me after hearing what seemed like unfounded claims that the deal fell apart because our product fell short in a certain area. I was especially curious about one deal in particular which we apparently lost to a couple of vendors with whom we never compete.

Something didn't make sense about these claims and I wanted to reconcile my own hunches with those coming out of Sales.

The decision: Schedule discussions with the lost prospects to learn more about their exact decision criteria.

The Sales team has been winning and losing deals for a long time but aside from our Reps recording one of a dozen pre-defined "reasons" in our CRM, the team had never spent any cycles capturing the details of the sale - and certainly had not been sharing the broader stories in a broader venue.

Our best opportunity to circle back with a lost prospect would have to be soon after they told the bad news. We had 2 such losses in the past 2 weeks and I scrambled with the Sales team to set up separate calls with each prospect to get the rest of the story, from their own mouths.  

Plan of attack

Flickr image source: http://tinyurl.com/od5e5na

My suspicion in both cases was that my Sales team had misinterpreted the customer's needs. In both cases, our folks had assumed that it had been a particular feature or set of related features that our competitor offers that had been the deciding factor. I was skeptical, but mostly because I didn't think our Sales team had a complete grasp of the other offering and were being hasty to find a scapegoat.

We scheduled conversations with each prospect and I had the Rep and their Sales manager sit in, mostly to listen. I shared my hypotheses with the Sales folks up front - specifically that I thought we would ultimately get feedback from the customer that was different from the conclusions they had drawn.

I also made it clear to them that I was motivated by a desire to learn and had no intention to lead our witnesses. 

Interview 1st prospect about competitor's tool

We had been hearing more and more about one particular competitor product in our sales cycles. They (the competitor) knew we didn't have an answer to it and wisely used their own product as a land mine to trip us up. I don't want to take anything away from our competitor but it was clear that they were intentionally guiding prospects to focus on this tool even when the prospect wasn't struggling with the set of problems it solved.

I've seen the tool and it does provide some attractive features. Until recently, we had been assuming that 1 feature in particular was winning over prospects. But the deal notes from our own Sales Rep on this opportunity pointed to something else.

We learned that our assumptions were wrong - the customer was struggling with a completely different problem.

In a 20-minute conversation with the prospect, we walked through their decision process and ultimately came to the competitor's offering. When prompted to identify the actual pain points and how well the competitor's tool matched up with those pain points, we learned that our assumptions were wrong - the customer was struggling with a completely different problem.

After thanking the prospect, I reviewed the outcome of the call, as gently as I could, with the Sales team to make sure they understood the real criteria of the customer's decision. The new information was indeed valuable and we would have to incorporate the findings into our own product strategy but it was also important to set expectations appropriately inside the company so as not to unnecessarily exaggerate our presumed product gaps. 

Interview 2nd prospect about broader needs

The second opportunity was apparently lost to vendors that we never see in our typical sales cycle. That seemed to me to point to a fundamental misunderstanding about what problems that customer is looking to solve.

We ran through a similar post-mortem conversation with this prospect who was all too happy to share their issues again. I find it easy to start conversations with decision makers around their pain and struggles. And even though our Sales team had just spent weeks courting them, they approached this new conversation with our Product team with unexpected enthusiasm.

Our Sales team had clearly wasted way too many cycles chasing an unqualified opportunity AND had drawn unnecessary attention to a presumed product gap.

Since this customer had not yet made a final decision, we steered the discussion to what had separated us from the remaining vendors. What I heard was at the same time validating and frustrating. The customer had legitimate needs but ones that would never be solved by the type of solution we had to offer.

The good news for me was that this lost deal was not going to impact our own ongoing product decisions. The bad news was that our Sales team had clearly wasted way too many cycles chasing an unqualified opportunity AND had drawn unnecessary attention to a presumed product gap.

The impact

Both prospects were not only willing to share with us but actually quite impressed that we would take the time to learn from our lost deals. I would like to think that this customer feedback will only accelerate our plans to formalize our win-loss analysis discussions in the months ahead.

I'm also hopeful that our Sales team will start to acknowledge that they sometimes misread the prospect's ultimate decision and will refrain from rushing to hasty conclusions. Perhaps only time and ultimately, more losses will tell.

Look for more reports from theProductPath around voice of the customer, competitive analysis, and managing stakeholders here on PM Decisions.

More articles from our blog PM Decisions

Recalibrate product priorities for the next 6 months

Flickr image source: http://tinyurl.com/pb94a5u

As we approached the half-year mark, I recognized that I needed to regroup with my internal stakeholders to make adjustments around what we would be delivering for the second half of the year.

You could argue that the half-way point through a calendar year is a somewhat arbitrary juncture, but nevertheless it provided me with a good checkpoint opportunity. Since the beginning of the year, my team had planned, built, tested and delivered product updates through several releases. Before we commenced with the next cycle, I wanted to revisit our priorities and review any significant changes in the business that would require some course correction in the product roadmap.

What drove this decision

I was confident that the Product team had been addressing the full range of demands and that we would continue to balance stakeholders' priorities as we were all committed to satisfying our end customers' needs.

We had already begun to carve out big plans for the next 6-9 months and some of them would require significant investment. I knew we were aligned with individual departments but over time, the various department heads had lost sight of what was most urgent for their peers. It would be more difficult for me to deliver and promote an updated product roadmap without some preliminary group consensus at the topmost level.

The decision: Organize and facilitate a product strategy summit with all the department leaders to revisit and rebalance priorities.

I will be the first to admit that the word "summit" sounds a bit lofty (pun intended) but I assure you this was not some trumped up, off site boondoggle - I'm saving that for my big end of year 1st Annual Product Strategy Retreat.

My goal was simply to get all the department VPs and some key Directors in the same room and run through a formal, Product-focused agenda. I wanted to use the time to review the roadmap choices we had made over the last 6 months (some good, some bad) and continue to push the company to pursue sound product investment decisions.

Balancing 3 related concerns: curb appeal, interior living space, and structural foundation 

In the many discussions I had been having with the different departments leading up to this summit, it became increasingly clear that there was a lack of mutual appreciation, if not basic understanding of what was driving the other departments' priorities. A number of the stakeholders were out of touch.

I wanted to share this insight with the invitees but in a way that didn't offend or alienate a particular group. In the summit invite, I offered the following apartment building metaphor to help explain the different, but complementary perspectives of each group:

Image sources: http://tinyurl.com/nbzkcoz, http://tinyurl.com/pno7n9f

• Sales wants the building to have massive curb appeal, to look good from the outside so as to increase the chances of attracting new customers.
• Customer Success wants all our new tenants to have a smooth move-in experience and to enjoy living in their new home for many years.
• Operations wants to make sure the building itself can safely support all the tenants and their various housing needs.

And of course everyone in the organization should be focused on increasing customer satisfaction as it will lead to renewals and referrals!

Plan of attack

Setting proper expectations: everybody wins! everyone loses!

I started the meeting by setting expectations for the group - which largely meant telling them that no one was going to get everything they wanted. The summit was an exercise rooted in compromise. I told them that it was equally important for them to defend their own priorities as it was to become more aware of what the other departments were struggling with.

The bulk of our time in the summit was spent prioritizing a lengthy list of wish list items. To kick things off, though, I offered an assessment of the past six months to remind everyone why we built the things we did and to get some feedback.

Review the roadmap themes for the year

I first presented the product strategy I had introduced at the beginning of the year that outlined how I would be making decisions for the next 12 months. Specifically, I highlighted the roadmap themes that were established back then and verified with the group that we were still confident in those decision-making guidelines.   

Review 1st six months

Building on the initial validation from the group, I proceeded to show how the recent product decisions we had made were in line with the roadmap themes. This brief, high level review of the advancements we had made in our product portfolio over the past 6 months provided an ideal opportunity to explore queries such as:

• How accurate was the original plan and how close did we come to hitting our targets/metrics?
• Are the major initiatives still on track or have there been new insights that require us to adjust our course?
• What have we learned about estimating work, about dependencies, about our ability to accurately respond to customer demand?

I was certainly pleased to hear the group confirm that the first half of the year was largely productive and that we had been made solid product investment decisions.

First pass at prioritization

From the Trello board I created to capture individual wish lists for the different departments, I pulled the top 30-40 cards to start the prioritization exercise. The first step was to bucket the items into either one of only two lists: High Priority or Low Priority.

This first pass proved to be relatively easy for the group. I was pleased to see the Low Priority list ultimately accumulate many more cards which meant the final pass would be that much easier.

Overall, it was a constructive discussion. None of the participants were particularly unreasonable and did not fight excessively for their own items.  I did occasionally use the following prompt when we seemed to be getting stuck: "Can we get through the second half of the year without spending any time on this item?" That was often the only nudging that was required to properly bucket the topic.     

Crash course on product constraints

I had prepared a break in the summit schedule in case we needed a cooling off period - and intentionally scheduled the catered lunch to arrive during the break. It turned out there hadn't been that much contention so I instead used the break to revisit some of our constraints and to remind the group about why the next half of the exercise was going to be frustrating for all. Specifically, I talked briefly about the following topics:

• Our true development capacity - what we should really expect from the current size and skill level of our Engineering teams. Read more about setting expectations around Engineering capacity here.
• Software's total cost of ownership - how each line of code we add requires more of our company's resources from Product, Engineering and QA all the way through documentation, support, and training.
• Opportunity costs - each time we agree to start building something new, we must recognize how that will limit our ability to respond to the next "must have" requirement (yeah Sales, I'm talking to you!)

I can't be sure if all the messages landed on point with the entire group but it was my meeting and I was going to take advantage of this ideal product soapbox opportunity.  

Final pass at prioritization

In the home stretch, the group now turned its attention to reviewing the much shorter High Priority list (14 items, down from 40). One by one, I went around the table to get individual input. I guided the group by having each of them select 1 primary item and 1-2 backup items.

When we had finished the final pass, we had arrived at a prioritization list that had mutual support from all departments. And I had a clear indication of how I would be adjusting the Product Roadmap for the second half of the year.

The impact

Not only did we finish the full exercise (on time no less) with a shared prioritization list but I was told afterwards by the group members that it was one of the more amicable and productive meetings around product direction in which they had participated in recent memory.

I made it clear to the group that the summit was not designed to immediately produce a new or updated roadmap. It would take me more time to work through the items on the High Priority list and factor everything into the Product Roadmap for the second half of the year. Still, I would like to believe the outcome of the summit extended beyond achieving consensus around particular product priorities. I think it represented a milestone for advancing inter-departmental agreement. Maybe next time, I'll refer to it as a Product Peace Accord.

Look for more reports from theProductPath around feature prioritization, product strategy, and managing stakeholders here on PM Decisions.

More articles from our blog PM Decisions

Cut down the scope of the initial product by 20%

Flickr image source: http://tinyurl.com/pg4zl82

After receiving push back from users in the first rounds of testing, this startup decided to eliminate a significant portion of its product offering.

For the past year, I have been observing the early stages of new business, started here in Chicago. The technical co-founder, a close friend, has also been serving as the Head of Product, not uncommon for small companies. I have been keenly interested to watch how this team would bring their vision to market and all the twists and turns that path would take.

One interesting development occurred after they put their initial version of the product in front of their target users. The feedback was immediate and in my opinion, quite sobering. The team would seem to have gone too far in one direction and was now facing a critical decision to scale back the initial scope of its product.   

What drove this decision

“This is too hard. I don’t want to create more work for myself or my client.”

— Early prospects

The originally envisioned product was intended to utilize a 3-step process. Users would complete a high-level questionnaire followed by a more detailed assessment to produce a curated product catalog where they could then make purchases in a familiar e-commerce scenario.

After developing a reasonable first pass at each of these components, the team put the product in front of a sample set of target users. The feedback was prompt and to the point: this is too hard.

The team had to make a change.

The decision: Drop an entire section of the user workflow

The (initial) users had spoken. The workflow had to change. It had to be simpler to complete. This startup team needed to step back and review the customer flow.

Plan of attack

The team ultimately decided to refactor the 3 step process and eliminate 1 of the questionnaires. After that, it would be time to test again:

• How would the new process change the results?
• Was there still a clear path to the purchase?
• What would the users find challenging about the new workflow?

Rethink the customer flow

The refactoring exercise began by decomposing the workflow process into finer grained steps, then exploring alternative paths for leading the user to the product registry at the end. The team focused on keeping what was needed most and trimming away the rest.

Redeploy the process with 1 fewer components

Having established the revised flow, the team proceeded to put the pieces back together to create a more streamlined activity for the users. The objective: a simpler process with fewer steps that produced the desired outcome.

Image source: https://www.flickr.com/photos/mmetcalfe/3874486791

Retest with users

Armed with an updated and improved product, the team scheduled the next round of user testing to validate their work.

The impact

Ultimately the team verified that their users could successfully complete the process without the extra steps. It takes courage to put an unfinished product in front of your customers and even more guts to go back to the drawing board when you get frank criticism.

Removing working code from a brand new product can be hard, especially if you look at the work required to build that code as having been wasteful. But I would argue that learning early leads to less waste. I applaud the startup team and encourage all product-minded people to follow their lead.

Look for more reports from theProductPath around product validation, feature prioritization and voice of the customer here on PM Decisions.

More articles from our blog PM Decisions

Find a good stopping point for a postponed feature

Image source: http://tinyurl.com/lckcjk6

After the principal customer backed off a requested new feature, I decided to quickly wrap up our discovery work.

Large feature requests require more time to plan and necessitate more resources. Collecting and sorting through customer requirements may stretch out over weeks or even months. And once these initiatives get going, they can gather momentum making it more difficult to shelve everything when the team's priorities change.

What drove this decision

One of our largest customer had asked us to help them with a problem that was tied to how they were using our software. It was a new problem for us but one that we felt would likely benefit other customers.

We had acquired a solid list of initial requirements but we knew better than to rely on only 1 source. So the Product team started researching the similar needs of other customers to begin the work of scoping out a new feature.

Not long into this effort, the original customer reprioritized some of its own, internal initiatives. The urgency around this problem diminished and the feature request dropped much further down the list. 

The decision: Suspend ongoing product discovery and shore up the current state of requirements.

I wanted to make sure that if and when the feature came back around, we could resume quickly and regain some of our initial velocity. We had made good headway and I needed to preserve as much of that as I could, especially if a different team would be picking up the feature next time.

Plan of attack

I was confident that the next team to work on the feature should not have to start from scratch. 

Requirements change, technology changes, and priorities certainly change. And I was sure that if we restarted this effort even 2-3 months down the road, we would have to revisit our assumptions. Still, I felt confident that we had been on the right track and wanted to preserve our progress as best I could.

Review the current hypotheses

I started by capturing the hypotheses I had developed around the proposed feature. I made sure to describe the thinking that went into the current hypotheses, to highlight assumptions I was in the middle of testing, and to list some of the ideas I had already ruled out.

Obviously some of these assumptions would - and should - be revisited but the next group would certainly benefit from the validation work I had done for the parts around which I had more confidence. For example, rather than build an entirely new scheduling tool to orchestrate and run a proposed background process, I had intended to leverage one we already had, even if it didn't completely accommodate all of the new requirements.

Create a list of next steps

I was forced to pause in the middle of the feature discovery but I had been working through a rough plan that would have resulted in a path forward. I felt it was important to document that plan, showing where I had left off and where the next team might pick it up and continue. I tried to anticipate and answer questions they might have such as:

• How far into this effort did the previous team get?
• How would they have scoped the work, knowing what they knew at the time?
• What were the dependencies if any that would guide the order of the proposed work?
• Did they identify technology limitations that would have dictated what was feasible at the time?

Preserve the notes in a convenient, accessible place

Flickr image source: http://tinyurl.com/lo8esfw

An obvious measure perhaps but I did not want to make it hard for the next team to find my notes. At the moment, our team uses Confluence for capturing early product discovery work. It is hard to believe that we would have moved on to another tool but anything is possible. So I made sure to create prominent links to the repository from Trello, the tool I use to corral Executive input; from Slack, the new tool of choice for our Dev Ops and Customer Support teams;  and in the primary email thread we had been using to communicate with the original customer.

The impact

Part of me feels that there was some wasted effort both in the original work and the steps to wrap it up for a future phase. But that feeling is not as strong as the one that is happy not to have invested even more time building a feature that may have been excessive or unneeded.

With a pay-it-forward mindset, I tried to think about helping the next PM to inherit this feature as I went through these steps. I know I would appreciate someone doing the same for me. And if it is me that opens the product feature time capsule down the road, I promise to be kind to my former self when writing up that experience.

Look for more reports from theProductPath around product teams, feature prioritization, and product investments here on PM Decisions.

More articles from our blog PM Decisions

Head off a new product idea before it gains any more steam

Flickr image source: http://tinyurl.com/jw7pso7

In recognizing that a few, excitable executives were intent on charging ahead with plans to build an "answer" to a competitor's product, I stepped in to introduce some rational thought.

I love exploring new product ideas, and that's especially true if those ideas and products are likely to help my company grow while delighting my customers. Brainstorming with other groups in the organization is healthy and I relish deep dive opportunities to learn more about their own struggles. Indeed, we have made some fantastic product breakthroughs over the past few years by collaborating across departments, thinking creatively and rationally about how to improve the business.

This was not lining up to be one of those situations.

As our organization was looking to head up market and seek out larger customer deals, we inevitably began to run into a new set of established vendors. One of these vendors seemed to be getting good traction in the sales process with a distinct feature that was noticeably absent in our own offering. And we quickly became fixated on that feature gap. And by "we" of course, I mean "Sales".

What drove this decision

“We’re getting our butts kicked!”

— Sales team armed with a single-digit data point and still smarting from a recently lost opportunity

Our Sales team was anxious about a particular competitor and quickly became convinced that we had been losing deals - and would continue to lose deals because we lacked a comparable feature in our offering.

What was really missing was actual data to back up these arguments. It is fascinating to watch how quickly a Sales team can work themselves into a frenzy and then how other internal decision makers can get caught up in the hype culminating in an escalating product emergency.

At first, I thought I could just let the frenzy flame out on its own. For surely, if there really was strong customer demand, then I would begin to see more and more frantic sales people banging on my office door. Unfortunately, neither scenario came to pass and so, with not even a little reluctance, I was forced to contain (and ultimately smother) the flames by helping the Sales leaders more formally assess the actual opportunity.

The decision: Get the Exec team to back off their urgent new product ploy

To be clear, I try to never dismiss a product proposal outright and indeed, this particular suggestion seemed to have real merit. What I refuse to do is barrel down a "new product" path without having first made the solid case that the investment has a good chance of producing returns.

You can learn more about the Multi-Tool here. 

At this point, I have to give a big nod to Marty Cagan who in my opinion, best summarized the technique for validating product opportunities in his book Inspired: How to Build Products Customer's Love. I feel so strongly about Cagan's Opportunity Assessment approach that I spent 6 months building a tool that helps Product Managers (and others) both develop and pitch Assessments to stakeholders.

My plan here was to use a simplified version of the Opportunity Assessment to push back on this most recent mania while at the same time, improve the way our management team approaches product decisions.

Plan of attack

My original idea of simply waiting it out and hoping the idea would die on its own, was failing. This became very clear when one of the execs unexpectedly called for a meeting where in just a few hours, we could "hash it out, create some wireframes/mockups that we could then outsource to an external development team". He made it sound so simple but what really concerned me was that no one seemed ready to question the approach. Inaction wasn't working - I needed a new approach.

A PM always has a few tricks up the sleeve to shut down a discussion like this. For example, you could go with "we don't have enough people", or "there are missing dependencies", or "the technology is not ready", etc. But these are short-term tactics. I had to make sure this type of request wouldn't pop up again in some whack-a-mole-style product game.

So, in the spirit of cooperation, I suggested that the assembled team be prepared to start that "hash it out" meeting with the responses to these points.

First make sure we understand the Pain

It all starts with a problem. You have to identify a particular pain you are trying to address and zero in on it. The pain can't be "our competitors have a feature we don't have" - it has to be customer-focused. We are in the business of identifying problems that customers have and will pay us to solve for them. 

Let me start by saying this is harder than it appears. To do this well, you need to be able to separate particular actors from the problem. So instead of saying "Customers want to use their phones to check the status of their transaction", you downplay the user to better highlight the problem: "it is difficult to verify the status of a funds transfer when away from a computer."

Then make sure we understand who has the pain

Note in most cases, you will address the first two points at the same time. I have found it more useful to keep these separate so that you can independently verify that you understand the problem and that you can find the users that struggle with it.

After you've identified the problem, you then nail down exactly who has that problem. Is it college students? Is it college students between the ages of 18 & 20 that live on campus and whose parents deposit money once a month...before the weekend?

The closer you get to identifying a specific persona, the better off you'll be. It will not benefit your team's opportunity assessment if you define your target customer in terms that are too broad, as you will see in the next section.

Is there a real market for a product that solves this 

If I could get the team to first agree on the problem(s) and then, who we believe has that particular problem or problems, then it's all downhill from there. Well, sorta. But we certainly can start to size the available market based on that persona. Sometimes it is easy to back off of an idea early on if you can determine that there really are not enough people to sell the product to. Or that you'll have a very hard time reaching them. Or that the customer's buying process is incompatible with your own sales cycle. And so on. 

Really what I'm hoping to discover of course is whether or not there would be a positive revenue impact as a result of pursuing this product or enhancement. After all, we are asking the company to make a product investment. And smart business leaders should know better than to make investment decisions without some level of due diligence.

The impact

In this particular situation, I was certain that the answers to these (and any remaining) questions would lead to the conclusion that the new product idea was ill-founded. Indeed, we shut down the discussion in short order and went back to more productive work. But the bigger win for me and the Product team was that we had introduced a new tool that would ideally lead to better decision making, at least when it came to new products. 

There are more questions to answer in the Opportunity Assessment (10 questions in Cagan's version) and certainly more than one sequence through the questions. I like starting with the plan outlined above because it is an efficient way to filter out premature and unfeasible product ideas early on.

But is can be used at any time in a product's lifecycle. In taking on the the "Head of Products" role, I inherited a product portfolio that I felt was too much for a company our size to manage. I have since begun the arduous process of reviewing each of our existing products and using the Assessment framework to defend or defeat each one. 

Look for more reports from theProductPath around opportunity assessments, product strategy, and managing stakeholders here on PM Decisions.

More articles from our blog PM Decisions

Assess the Product Portfolio

Flickr image source: http://tinyurl.com/nl2c4vh

After recognizing that our development team would remain inundated with far too many product requests, I decided to examine our entire collection of products with an eye toward slowing or reversing ongoing product investments.

Like any Product team, we are never in short supply of feature requests, product suggestions, looming technical debt, must-have components, urgent customer pain points, etc. And we do our best to stay on top of the things, trying to keep all parties happy-ish. But over the past few months, I had started to see important product enhancements slipping from each release and was able to attribute those slips to other competing product priorities.

Because there are quite a few moving parts in a typical software development shop, there are any number of places you could start the analysis. Our problems could be related to bad grooming practices, poor scoping efforts, or even an understaffed engineering team. But I know the problems we were having were not limited to a single product or any one PM. It was simply proving too difficult to keep all our balls in the air.

We had launched a generous number of products over the past decade but poking around at the ground level was likely to be an unproductive exercise. I needed a model to help me and our team evaluate decisions across our entire portfolio.

What drove this decision

I was getting anxious about the number of product-related discussions the Management team was having and becoming convinced that they had lost some perspective on what we had already committed to. In fact, there were signs that the company's entire product strategy might be in need of a refresh so I scheduled a mid-year strategy meeting with the management team to do just that. But I felt there was a need to connect the day-to-day challenges we were having with each product release to the product discussions that were happening (at an increasing rate) at the executive level.

I needed to create a backdrop for our high-level product investment decisions that would also drive real benefits at the troop level.

The decision: Create a model for evaluating current and future product portfolio decisions

Over the past decade or so, we had made decisions to create new products and not all of them turned out to be winners - no real surprise I guess. The bigger problem was that we had managed to sell each product to some subset of our customer base so we had an obligation to support it through troubleshooting, bug fixing and occasionally, even enhancement-ing. This obligation grew non-linearly with each new product. 

I knew the customer usage patterns were irregular and more importantly, the revenue we derived per product was inconsistent and unpredictable. Despite this, we had no plans to sunset any of the products thus perpetuating our dilemma indefinitely.

I needed to spark some productive conversation. I needed to shed some light on the impact of our past decisions and try to head off some questionable proposals that were brewing in the company.

Plan of attack

The diagram that I built only needed to be as complex as the audience I was targeting. I felt I could communicate a great deal with coarse-grained data elements, where conclusions could be drawn (no pun intended) by counting boxes or showing the relative size or positioning of boxes. I ended up with something along these lines (pun fully intended):  

View fullsize

Simplified Product Portfolio Model

1 - Plot the products on a simple graph based on revenue vs. customer use

I strongly recommend that you organize your graphs so that the most desirable position is in the top right corner. Both X and Y axes should go from low to high, least to most, etc.

I started by placing each product on the chart based on how much customers were using it and how much revenue it generated. In using these axes, I was hoping my products would be stacked up and to the right. If plotted accurately, it should be easy to defend ongoing and future product investment decisions with this simple graph.

In an attempt to communicate more information with fewer charts, I decided to use the relative size of the product "boxes" as an indicator of our current investments. Without aiming for exactness, I was able to quickly explain where our resources are being deployed. A slightly more complex approach would have been to use other graphical elements to break down the current investment by Product, Engineering, Operations and Support.

2 - Clarify the ratio of Products to Product Resources

Our company currently has more than 15 products in our portfolio and I'm sad to report, not nearly enough Engineers and PMs to manage a portfolio of that size. We have found ourselves in this situation after years of adding new products because at the time, we believed they were crucial to our company's success.

But like many eager, young software companies, we never seem to plan well for the long-term maintenance and support of each new product. I can't help but make the comparison to the decision of adding a new baby to your family.  Everyone thinks it a wonderful idea at first but after spending a number of grueling months bringing it into the world, you find yourself responsible for taking care of another living thing for the rest of your life (or your life at the company at least).

With a few, carefully placed text overlays, I listed on the diagram, the relative sizes of our current teams which revealed the stark truth about our current staffing situation. This helped set the stage for a productive conversation about acquiring more Product/Engineering resources or reducing the size of the product portfolio or both.

3 - Future investment and product goals

I used the same model to spell out some of our short and long term product strategy. With the boxes already arranged on the chart, I added arrows to indicate where we would expect to see movement in the months ahead. Specifically, I predicted the movement of various products over the next 6-12 months in terms of increased/decreased usage and increased/decreased revenue.

By adding a few crude marks over some of the product boxes, I was able to show which products in which I intended to reduce our investment and even more, which I thought we should retire in the months ahead. 

The impact

There are more messages that can be highlighted and communicated in a chart like this. Seeing all the products intelligently placed next to each other on a single page inevitably invites more strategic conversations like:

• Why are we still supporting this product if the usage rate is that low? It's time to retire it.
• When is the last time we updated the technology behind those products? They're due for a refresh.
• How long has it been since we last sold this product? Let's try lowering the price.

Armed with this new model, I was able to drive a strong Product-focused agenda at the Strategy Summit and was even able to successfully head off some premature ideas that could have led us further down the wrong path.

Look for more reports from theProductPath around charts, product portfolios, and managing stakeholders here on PM Decisions.

More articles from our blog PM Decisions

Evaluate a product integration with a potential partner

Image source: https://www.flickr.com/photos/emanuelec/5869072769

After some considerable prodding from the Sales team to help advance the largest deal in the pipeline, I decided to formally assess the key vendor integration options that could help us win the business.

It is rare that any single product can wholly satisfy a customer's needs and quite common that multiple products will be used in concert to get the job done. Cloud-based products for example, like the one my company offers, are often combined to accomplish set of use cases. Think of opening your Dropbox files in Microsoft Office 365 to make changes. Or pulling Flickr photos into a Slack channel to help the team troubleshoot a problem.

Through an investment in both SOAP and REST APIs, we have made our cloud product open to integration as well. We often point partners (and customers!) to these full-featured web services when we encounter requests for functionality that hasn’t been productized. But since it may be necessary to initiate the integration from our end, we also provide hooks in the platform for outbound calls.

I’ve always been intrigued with software integrations. Part of that interest is derived from exploring the forethought that goes into making any software product extensible and the rest is observing how the combination of two or more products results in a new creation that may never have been seen before.

What drove this decision

Sales was getting anxious - more anxious than usual

We had been salivating over a large opportunity in the sales funnel, a big deal in the hopper - a hit-your-quota kind of transaction. For weeks, the Sales team had been working hard to address the prospect’s long list of requirements and to zero in on the areas where there was lingering concern.

This particular prospect was asking for more functionality than we could - or would ever likely provide. The Sales team was eager to determine whether another vendor could be introduced into the deal (always a risky proposition) to fill the functional gaps.

It was now my job to confirm whether or not this was technically feasible and if so, how it would be done. But in order to have all the parties understand the proposed solution which would likely be unfamiliar and quite technical, I was going to need to explain it with some pretty pictures.

The decision: Create a graphical model highlighting the integration points for the customer's use cases to help validate a new partnership

Diagrams help to communicate with a broader audience 

Joining the conversation late, I needed to orient myself to better understand the other vendor's product. I set up a few phone calls with both the vendor and the prospect, had some internal discussions with the Sales team, and scribbled a lot of notes. When I thought I had a good, initial model, something worth sharing, I polled each of the groups again, passing around a 1-page diagram where I had squeezed some boxes and lines together to create a crude chart.

These early rounds of validation prompted some tweaks to the diagram and I was able to tighten down the overall integration story. What also became clear was that, to make this happen, some non-trivial product investment would be involved on both sides. I knew we would soon be having a number of internal and external “business case” or ROI-type discussions and that the crude chart would not suffice. I decided to step it up and invest in a more robust model.

Plan of attack

My goal with this improved model was to capture and communicate the concepts that would best help with the downstream decision making. This meant not just determining the number of integration points to fulfill the customer's use cases but the relative complexity and level of effort of each, as well. Clarifying to all parties which use cases required which integration scenarios meant that we could better prioritize the relative work involved.

HIGHLIGHT the origin of each integration point

As I reviewed the use cases, I looked for places where the functionality of one of the products ended and where it needed to be picked up in the other product. This made it easy to determine which of the products would initiate a new integration scenario. On the diagram then, I drew uni-directional arrows between systems that are themselves, usually represented as fancy boxes.

I usually label the arrows with numbers or text to help direct the reader to the start of the flow. I also find it valuable to identify the “trigger” itself. For example, “a user clicks the upload button to start the process” or “the process is initiated when a user is added to the address book” or “each new PDF document added to the folder will initiate a call to the other system."

When more than one back and forth call is required to accomplish a particular scenario, the diagram get more interesting. Does the calling application wait for an immediate response (synchronously) or will the reply come back some time later (asynchronously)? What happens when the response never comes back - which party is responsible for resolving the loose ends?

Identify dependencies

In identifying the origins of each integration point, I’ve invariably highlighted new dependencies between the products. Dependencies must be evaluated carefully because changes to the callee’s interface will affect all callers. It is common for Product Teams to update their interfaces (we do it with our own web service interfaces) and must consider backward compatibility to minimize the impact on existing integrations.

For the sake of simplicity and because these would start to clutter up the sample diagram, I’ll skip over the security considerations, the deployment models (e.g. desktop, on premise, private/public cloud, etc.) and any differences in how each company licenses their products.

View fullsize

Sample Integration Diagram

Identify alternatives for each Integration point

As the saying goes, there is more than one way to skin a cat. This can apply to product integrations as well. I always look to indicate any possible alternatives that I discover so that the decision makers can better evaluate the full range of possibilities - even if the alternatives seem less practical at first.

For each alternative, I try to include the relative levels of effort, costs, etc. I usually have a good reason for choosing the primary option so I spell out that rationale for my first choice. Included in the rationale could be viability concerns about the integration technology choices, the types of connection(s) required, overall speed/performance, error handling, etc.

Distinguish between what's available now and in the future

In this particular effort, I learned that some of the integrations would be dependent on features that the other vendor had yet to release. To highlight that important variable, I made special, prominent notes about what would be feasible today vs what was yet to be built to support the integration.

The impact

I delivered the updated, more formal integration model to the teams and continued to participate in the partner discussion for a few more weeks. After both vendors had settled on a viable approach, we jointly presented to the prospect. The additional work to provide an integrated solution was spelled out along with the additional cost, some of which would be subsidized by the vendors. The final proposal, that included a modified version of my diagram, was well-received.

I often create and share models like this to weigh important decisions. I find it easier to grasp the complexity of a proposed solution when you are able to visualize the number of boxes and lines. Unfortunately, a diagram can't always communicate the longer term return on investment of a product integration. It doesn't tell you how many customers will use it or when you can expect sufficient revenue to cover the implementation costs. But that's another article....

Look for more reports from theProductPath around product validation, charts, and product strategy here on PM Decisions.

More articles from our blog PM Decisions

Getting ready to groom

Flickr image source: http://tinyurl.com/ogerhjp

In recognizing that too many of our user stories were not truly ready to be implemented, I decided to revisit the entire path to grooming

For months, I had been watching the Product team wrestle with getting their particular features out the door and in front of our customers. And it wasn't just one PM who was struggling or an exceptional delay in delivery - it was borderline chronic. More often than not, I could trace the issues back to poor preparation by the Product team.

I have been working with our entire tech team to continuously improve our own Agile development process and we certainly have more work to be done. But the problems I needed to address were further upstream, before the stories entered the Engineering and QA queues. The Product team was creating unnecessary thrashing by failing to do our part well. 

What drove this decision

When your product releases are scheduled less frequently (think months apart vs. weeks), there is a greater impact when you miss a delivery date. I was frustrated at having to drop features from a release because they weren't ready or seeing enhancements slip to the next sprint.

And while it is tempting to spread the blame across the myriad variables that go along with shipping product software, I became convinced that my own Product team had to step up to support our share of the contract.

Some product planning failures are easy to measure. Are features not being completed? Are your stories slipping over successive sprints and even releases? Are your Engineers and QA teams stalled waiting for last-minute clarifications around designs and requirements?

The decision: Formalize the team's story preparation process leading up to sprint planning

The bad/good news was that we had accumulated sufficient instances over the past few months to see some a pattern emerging. Early on, we tried using note cards and a whiteboard to better track the larger product initiatives over sprints and releases. This seemed promising at first, probably because the team appreciated an exercise that was both tactile and collaborative. But we ultimately failed to capture any incremental advancement of any initiative using the cards. Moving the cards across the board from sprint to sprint didn't make us better at preparation and worse, this exercise did not help us identify deficiencies in the planning process.

We needed a different approach, one that helped us focus on maturing product requirements BEFORE we gave them to the team. We needed a better product grooming process that preceded the actual backlog grooming sessions.

Plan of attack

The tactics described below were prescribed only for our larger product initiatives and would likely be overkill for bugs and smaller feature enhancements. We began by focusing on the problems that occurred most frequently, then folded in additional steps to mature the pre-grooming process.

STEP 1 - Include the fully baked UX designs 

Perhaps the most obvious problem for our team was that we were often scrambling to get our front-end designs in place by the time the Engineers were ready to estimate the level of effort. We had become less and less stringent about completing the UX work and ensuring that formal designs were attached to a story or epic prior to backlog grooming or worse, sprint planning.

The lowest hanging fruit, then, was to re-establish this specification for all our user stories. Going forward, no stories would be allowed into a backlog grooming session unless the designs were complete. Obviously, there is no real definition of "fully baked" as teams will inevitably tweak designs during development and even testing but this was a relatively easy first stake in the ground.

STEP 2 - Review the acceptance criteria with internal stakeholders

I'm of the mindset that you can never stop improving on your ability to define product requirements. I encourage my Product Managers to continue to work on this skill with each new assignment. But there is an easy to tell whether you are actually improving or not by gauging how often your stories are being kicked back by the Engineering or QA teams.

Flickr image source: http://tinyurl.com/le8jhxv

And even if you are scoring well with these groups who are much closer to the code, you probably have other stakeholders inside the company that can help you measure your aptitude. In our organization, the Sales Engineering and Professional Services teams are great early sounding boards. Another important group (for us and most organizations) is the Customer Support team. Each of these groups is intimately familiar with your customer base and can help validate how well you have done in capturing the happy path(s) and the larger set of user variations.

In our updated process, we have a specific gate for reviewing stories or at least the acceptance criteria with our internal stakeholders. This has the additional benefit of not catching them off guard during a Release Preview for example, when the feature is already "complete". 

STEP 3 - Weave in usefulness testing with end users

When it comes to testing products with end users, excuses usually outnumber opportunities. Like most Product teams, we would prefer to have more time to spend with our customers but scheduling these interactions always seems more difficult that it should be. Still, the team and I firmly believe this is a critical part of the grooming process - especially as the requirements and designs come together.

Going forward, we have made this an explicit stage in the lead up to backlog grooming. Of the steps listed here, it is the most challenging to implement but will likely have the biggest payoff. I can think of nothing better for removing doubt and minimizing surprise than having end users validate your product throughout your development process.

The impact

The Product team is still adjusting to the new approach although we all recognize the improvements that will come with better story preparation. We've already seen some of those benefits: in the past few weeks, we withheld a number of stories from backlog grooming that had not met the new criteria, saving us hours of guaranteed thrashing with the Engineering and QA teams.

Additional refinements will be necessary. As we advance and extend our customer research and user testing, we will introduce new gates in an effort to further solidify the product features and push them through our Agile process. 

Look for more reports from theProductPath around product validation, backlog grooming, and product culture here on PM Decisions.

More articles from our blog PM Decisions

Pull a large MVP-less feature from the next release

Flickr image source: http://tinyurl.com/pl7xl4w

In recognizing that an important but poorly scoped product initiative was slipping further behind, I decided to push it farther back on the roadmap.

I would argue that our organization is actually quite Agile-minded but we have more work to do before we can start scheduling product releases in units of weeks rather than months. In our current state, we have to treat releases with a bit more care as they don't come around that often. So when we target a major product initiative to be completed and deployed in a future release, it is critical that we drive toward the date. Any slip could delay the feature by a month or more and that can have significant customer implications.

I had recently given one of the PMs on my team a real juicy assignment. We had determined, through some solid customer research, that one of the core features of the platform was in need of an overhaul. In matching up usage data with anecdotal user feedback, we recognized that the problems would require more than a few simple enhancements. This was indeed a large product initiative and was lining up to be one of our strategic product achievements for this year. 

One of the core features of the platform was in need of an overhaul.

What drove this decision

We had been wrestling with all the usual challenges of developing a new product feature on a tight schedule. Some requirements required further clarification. Web page and email designs required rounds of tweaking and tuning. Plus the Engineers were being pulled away periodically to troubleshoot and fix other, non-related bugs. And then there was the Operations team who had planned a major infrastructure upgrade that created additional chaos, especially as we depended on those same resources to deploy the new feature.

I could go on, listing distractions and blaming other parties. But the number one reason for the slippage was simply subpar product planning. The PM had not done a good job in decomposing and staging the required work. After a few months, we had created some great new software but no shippable product. 

The decision: Drop the feature from the next major product release and reinvest in some proper scoping

About a month before the next big release, it had become increasingly clear that we would not finish developing the feature in time to validate it thoroughly with our users. In our culture, we place a high value on both sticking to our release dates and only shipping product features that are user tested. The first constraint often governs the second.

The decision to drop the feature at this point in the release schedule may have been a bit precarious. But it was really only the first step in a series of related actions that kept the Product and Engineering teams busy for some time. 

Plan of attack

As it turned out, our poor product planning had resulted in even more work than you might expect from what seems like a straightforward decision. We desperately needed to put this particular feature back on the right path but there were now some additional tasks that would enter the work stream.  

REVERT TO THE OLD FEATURE

So confident had we been in our ability to complete this feature (including functional, performance, and user testing) that the team had started dismantling the legacy functionality.

Now we had to re-mantle it. Or more specifically, we had to back out some of the new enhancements in order to restore the old feature. No self-respecting Engineer enjoys this kind of work and indeed it cost the Product Team dearly in terms of credibility.

ADD A FEATURE TOGGLE

In order to preserve the progress that had been made, we invested some additional time in creating a true feature toggle that would allow us to enable the new version for individual customers. This provided us with the option to conditionally test the new functionality, as soon as it was available, with some of our more curious and perhaps more forgiving users.

RE-SCOPE AND RE-PLAN THE FEATURE

The bulk of the work came from decomposing the originally planned feature into smaller stories and creating a revised, incremental delivery plan that would help us deliver a true MVP first and then, in subsequent iterations, a series of progressive improvements.

Flickr image source: http://tinyurl.com/on6nch5

I will not use this report to describe the detailed process of scoping a product feature. There are many good discussions around discovering the Minimum Viable Product or MVP. To be sure, it takes practice to learn how to be ruthless. Good PMs find ways to whittle away at a product or feature until there are no nonessential elements remaining.

But it takes a different set of skills to then carefully plan the gradual introduction of those elements over time. It can be tempting to load up on enhancements, especially as product adoption increases among customers and users. Discipline and care are traits to be nurtured and celebrated in your Product Managers.

RESET EXPECTATIONS

As I mentioned earlier, this feature, when complete, is intended to make a big splash with customers. In making the decision to postpone its deployment, I now had the delicate task of resetting expectations with our Sales team, with Professional Services and Support teams, and with the few customers and prospects we had been engaging with for testing and research.

The impact

The existing customers and new prospects that were pining for this feature were disappointed with the decision to pull and postpone though I believe the blow was cushioned in two ways. First, we notified them of our product decision several weeks ahead of the release instead of only days before or even worse, after the release.

Second, we took care to explain that we did not feel comfortable pushing a significant new product feature without appropriate user testing. Not to say that this relieved all the anxiety but it is more difficult to argue with the choice of product quality over speed to market.

One final note. The Product team did consider moving ahead with a nearly complete version of the new feature and including it in the upcoming release. The thought was that we would have the ability to turn it on for a few, select customers in exchange for some heavy usability testing. I will publish more on that approach in a separate article.

Look for more reports from theProductPath around planning products and releases, MVPs, and product culture here on PM Decisions.

More articles from our blog PM Decisions

Publish and promote a more consumable product roadmap

Flickr image sources: http://tinyurl.com/otrq3r6, http://tinyurl.com/qyf68uo

As interest in the future capabilities of our products began to increase, I decided to invest more time in making the roadmap more accessible to all parties.

I could spend hours tinkering with a roadmap. There are so many variables to play with, dependencies to work through, customer feedback to fold in, unanticipated hiccups that challenge the schedule, new bugs that surface, and on and on. Without any prompting, I will frequently create several versions of the roadmap to emphasize different details or highlight specific risks. But the list of people that want or NEED to get that close to those details is quite short.

The roadmap can be an indispensable tool for communicating plans to those outside the product team. In my experience though, geeking out with the product roadmap is just not for everyone - most people just want the simple story.

What drove this decision

Over the past few months, I had been fielding a steady stream of inquiries from our Sales team who in turn, were being asked by our customers and prospects to share our product roadmap. These requests were prompted by a desire to see where the company were headed, product wise, especially when there was some set of requirements that would seem to stretch the current limits of our platform.

Most people just want the simple story.

I was also being pulled into discussions with other departments across the organization where, inevitably, the conversations were centered around how our products would (or wouldn't) address a variety of concerns. During these meetings, I often felt as if I should have been carrying a copy of the roadmap with me but the versions I had were too elaborate and were more likely to have generated more questions that it would have answered.

The decision: Socialize a lighter weight product roadmap for inside and outside audiences

Many of the questions I was answering were simple in nature and could be settled with a quick clarification or by referencing a feature in an upcoming release. Unfortunately, there was no accessible model or illustration to reference. I needed to create an alternative, more accessible artifact.

Plan of attack

You have different considerations when you produce a version of your primary artifact that is suitable for external audiences. Not only will you have the challenge of reducing the overall complexity of the content to make it more accessible to those with less context, but you may also need to put it within appropriate physical reach to accommodate the timeliness of their needs.

For me, this meant paring down the content to eliminate details that would only be needed or appreciated by the Product and Engineering teams. It also meant distributing the new artifacts to new channels that puts them within arm's reach and encouraged self-service (vs. disruptive, ad-hoc inquiries).  

Create an attractive artifact that's easy to follow

The first step was perhaps the most difficult for me. I started with the assumption that this extended audience would have a very limited attention span. They would not be that interested in exploring all the potential paths, dependencies and implementation details of the 12-month product roadmap. They were looking for quick answers to broad questions.

Image source: https://www.flickr.com/photos/freshwater2006/693945631

In an attempt to remove the "clutter", I focused on highlighting outcomes, specifically the key dates when they could reasonably expect the big ticket items to become available. I retained the high level  themes, the broad commitment dates and other key elements that go into creating a solid product roadmap. I also embellished this version a bit, altering the names of certain features to line up better with customer use cases. For example, instead of listing a simple "reminder" feature, I described how the reminders could be used to better support a particular business process lifecycle.

Finally, I included and emphasized disclaimers about the accuracy of the roadmap commitments to help set/adjust expectations. This was a bit more CYA than anything else as I want to be able to defend future product decisions that will inevitably conflict with any previous "promises".

Make it accessible for self-service

Not that I don't appreciate being pulled into product conversations but a great number of these do not require the Product Head Honcho. To help people answer their own questions, I took care to put the light weight Roadmap in easy-to-find places throughout the office and beyond:

• Outside my office - My own office is on a well-trafficked path making it convenient to post a copy of the Roadmap on the wall. Several times, when people have barged into my office, I have had to gently usher them outside to point out the "self-service" display. 
• On monitors - Our IT team has placed large, flat-screen monitors throughout the office. I have inserted an image of the new Roadmap into the standard rotation for these monitors to help keep it top of mind.
• Shared repository - I also found it useful to put a copy of the Roadmap artifact in the shared "folder". We have been coaching people to begin any new search there before emailing or picking up the phone.

Present early and often

Now that the distilled Roadmap was ready, I started to share it with the teams at every opportunity. At "all hands" meetings, at our monthly Release Preview, during weekly Sales and Marketing meetings, and at customer events, I would present the artifact to show incremental progress, update everyone on recent product decisions and even show sneak peaks of ongoing development initiatives. 

The impact

I am pleased to have stemmed the tide of inbound roadmap-related queries. And while it does take a few extra cycles for me to keep everyone up to date, I am confident that we have further empowered our Sales team and are helping them move past would-be obstacles with customers and prospects in the sales cycles.

The simpler, lighter weight rendition of the Product Roadmap is now a staple of our planning efforts. We will continue to produce and promote the roadmap through this artifact.

Look for more reports from theProductPath around product management tools, product roadmaps, and product culture here on PM Decisions.

More articles from our blog PM Decisions