Triage an almost bungled feature upgrade

Flickr image source: http://tinyurl.com/pp7daju

In recognizing that our pre-deployment testing had raised unexpected alarms with some customers and our own Support staff, I decided to straighten out the mess by creating a better feature migration path.

We were winding down the days to our next major product release and I thought the planning had been going well. There are always a number of moving parts in a big deployment and it takes no small bit of coordination across teams to pull off a trouble-free rollout.

One of the big announcements this time around was how we had overhauled one of our more prominent features to make it easier to use. And while we had usage data that showed it wasn't exactly mission critical for our existing customers, we still wanted to ensure a smooth migration.

What drove this decision

The upgrade approach we had planned was simply swapping in the new feature in the place of the old one. Any account that was actively using the old feature would continue to work for the 30 days following the release but after that time, all accounts would be have been automatically switched over.

This seemed straightforward enough and indeed, our internal testing and QA efforts proved that this was a clean transition. Where we went wrong however, was not effectively communicating the plan, especially to our internal stakeholders.

The most obvious blunder was in the Release Notes (I wrote) that we circulate internally, weeks before the go-live date. I had made a copy/paste error using some outdated text and reported the exact opposite intent around this migration! As you might imagine, the Customer Success team freaked out when the old feature "suddenly just disappeared".

All fingers were rightfully pointing back to me and my Product Team. What I thought had been weeks of good planning and internal discussion were negated by what was now a small crisis. I needed a good plan to set things straight.

The decision: Instead of forcing a more immediate upgrade on our users, I would stretch out the feature deprecation period and give customers more time and more autonomy to make the transition.

It is worth clarifying that nothing was indeed broken in any customer account and nothing would break as a result of pushing the new feature in the upcoming release. Actually, I could have tried to smooth over the current unrest and push forward as we had originally planned. After all, no actual customer had experienced any problems in their accounts.

But in this case I chose a more cautious path. Even though it would be more work for our tech teams - actually, it would be re-work since we had already removed the old feature and now had to quickly retest the reverted code - I knew the extra effort to create an extended upgrade period would cost us a lot less than having to soothe any confused or disgruntled customers.

Plan of attack

Patience is a virtue - especially in times of crisis. When the voices are beginning to elevate, when email threads are growing exponentially in a matter of hours, and when emergency meetings are being scheduled first thing in the morning, it helps to be able to keep a calm head.

I would have to calibrate our response based on the actual size of the problem.

One of the things I wanted to make sure was that, in attempting to address the problem, we didn't make things any worse than they were already perceived to be. The key to that, in my opinion, was to properly scope the effort. In short, I would need to be able to calibrate our response based on the actual size of the problem.

Understand the full impact of the change

When we did assemble the interested parties in a meeting room and reviewed the situation, one of the first questions I asked was whether we knew the number of customers/accounts that would be affected by the deprecated feature. My own Product team had gathered some numbers weeks earlier but I wanted to know if we had been off in our estimates.

Because there wasn't enough certainty, I had the Engineers run a new set of exhaustive queries to get our hands around the total affected population. Early totals were supplemented by more thorough search results but all reports showed that the impact was far less significant than the recent hype would have suggested.

Flickr image source: http://tinyurl.com/ol3x3jr

Call attention to the deprecated feature

With the Engineering team's help, I committed to reinstating the deprecated feature into the product to help affected customers with their inevitable migration. But in doing so, I insisted that we incorporate some blatant visual cues for the end user so they understood that some action was required.

With minimal work, we were able to introduce a glaring design element intended to draw attention to the deprecated feature. This new call to action would be reinforced with other messaging inside the product and externally as well.

Broadcast (better) to internal and external users 

The steps required for a customer to move to the new, improved feature were straightforward and would likely only take a few minutes to complete. However, I have learned the hard way about underestimating users and so, to err on the side of caution, I committed to providing multiple migration artifacts.

I had the Product team scramble to develop new collateral including a 1-page cheat sheet and a short instructional video to help guide customers on the migration process. In both, we explained how the deprecated feature would be disappearing in the subsequent release to be replaced by the improved option available now and we included a recommended course of action to avoid any problems in their account.

The impact

In the post-mortem discussion, all the teams agreed that the new feature is a much better option for our end users - we had just stumbled a bit in our plan to roll it out to our customers. It was a good lesson for me and one that I'm happy to have tackled before the official release where it could really have escalated.

Look for more reports from theProductPath around product releases, product culture, and PM credibility here on PM Decisions.

More articles from our blog PM Decisions

Cut down the scope of the initial product by 20%

Flickr image source: http://tinyurl.com/pg4zl82

After receiving push back from users in the first rounds of testing, this startup decided to eliminate a significant portion of its product offering.

For the past year, I have been observing the early stages of new business, started here in Chicago. The technical co-founder, a close friend, has also been serving as the Head of Product, not uncommon for small companies. I have been keenly interested to watch how this team would bring their vision to market and all the twists and turns that path would take.

One interesting development occurred after they put their initial version of the product in front of their target users. The feedback was immediate and in my opinion, quite sobering. The team would seem to have gone too far in one direction and was now facing a critical decision to scale back the initial scope of its product.   

What drove this decision

“This is too hard. I don’t want to create more work for myself or my client.”

— Early prospects

The originally envisioned product was intended to utilize a 3-step process. Users would complete a high-level questionnaire followed by a more detailed assessment to produce a curated product catalog where they could then make purchases in a familiar e-commerce scenario.

After developing a reasonable first pass at each of these components, the team put the product in front of a sample set of target users. The feedback was prompt and to the point: this is too hard.

The team had to make a change.

The decision: Drop an entire section of the user workflow

The (initial) users had spoken. The workflow had to change. It had to be simpler to complete. This startup team needed to step back and review the customer flow.

Plan of attack

The team ultimately decided to refactor the 3 step process and eliminate 1 of the questionnaires. After that, it would be time to test again:

• How would the new process change the results?
• Was there still a clear path to the purchase?
• What would the users find challenging about the new workflow?

Rethink the customer flow

The refactoring exercise began by decomposing the workflow process into finer grained steps, then exploring alternative paths for leading the user to the product registry at the end. The team focused on keeping what was needed most and trimming away the rest.

Redeploy the process with 1 fewer components

Having established the revised flow, the team proceeded to put the pieces back together to create a more streamlined activity for the users. The objective: a simpler process with fewer steps that produced the desired outcome.

Image source: https://www.flickr.com/photos/mmetcalfe/3874486791

Retest with users

Armed with an updated and improved product, the team scheduled the next round of user testing to validate their work.

The impact

Ultimately the team verified that their users could successfully complete the process without the extra steps. It takes courage to put an unfinished product in front of your customers and even more guts to go back to the drawing board when you get frank criticism.

Removing working code from a brand new product can be hard, especially if you look at the work required to build that code as having been wasteful. But I would argue that learning early leads to less waste. I applaud the startup team and encourage all product-minded people to follow their lead.

Look for more reports from theProductPath around product validation, feature prioritization and voice of the customer here on PM Decisions.

More articles from our blog PM Decisions

Pull a large MVP-less feature from the next release

Flickr image source: http://tinyurl.com/pl7xl4w

In recognizing that an important but poorly scoped product initiative was slipping further behind, I decided to push it farther back on the roadmap.

I would argue that our organization is actually quite Agile-minded but we have more work to do before we can start scheduling product releases in units of weeks rather than months. In our current state, we have to treat releases with a bit more care as they don't come around that often. So when we target a major product initiative to be completed and deployed in a future release, it is critical that we drive toward the date. Any slip could delay the feature by a month or more and that can have significant customer implications.

I had recently given one of the PMs on my team a real juicy assignment. We had determined, through some solid customer research, that one of the core features of the platform was in need of an overhaul. In matching up usage data with anecdotal user feedback, we recognized that the problems would require more than a few simple enhancements. This was indeed a large product initiative and was lining up to be one of our strategic product achievements for this year. 

One of the core features of the platform was in need of an overhaul.

What drove this decision

We had been wrestling with all the usual challenges of developing a new product feature on a tight schedule. Some requirements required further clarification. Web page and email designs required rounds of tweaking and tuning. Plus the Engineers were being pulled away periodically to troubleshoot and fix other, non-related bugs. And then there was the Operations team who had planned a major infrastructure upgrade that created additional chaos, especially as we depended on those same resources to deploy the new feature.

I could go on, listing distractions and blaming other parties. But the number one reason for the slippage was simply subpar product planning. The PM had not done a good job in decomposing and staging the required work. After a few months, we had created some great new software but no shippable product. 

The decision: Drop the feature from the next major product release and reinvest in some proper scoping

About a month before the next big release, it had become increasingly clear that we would not finish developing the feature in time to validate it thoroughly with our users. In our culture, we place a high value on both sticking to our release dates and only shipping product features that are user tested. The first constraint often governs the second.

The decision to drop the feature at this point in the release schedule may have been a bit precarious. But it was really only the first step in a series of related actions that kept the Product and Engineering teams busy for some time. 

Plan of attack

As it turned out, our poor product planning had resulted in even more work than you might expect from what seems like a straightforward decision. We desperately needed to put this particular feature back on the right path but there were now some additional tasks that would enter the work stream.  

REVERT TO THE OLD FEATURE

So confident had we been in our ability to complete this feature (including functional, performance, and user testing) that the team had started dismantling the legacy functionality.

Now we had to re-mantle it. Or more specifically, we had to back out some of the new enhancements in order to restore the old feature. No self-respecting Engineer enjoys this kind of work and indeed it cost the Product Team dearly in terms of credibility.

ADD A FEATURE TOGGLE

In order to preserve the progress that had been made, we invested some additional time in creating a true feature toggle that would allow us to enable the new version for individual customers. This provided us with the option to conditionally test the new functionality, as soon as it was available, with some of our more curious and perhaps more forgiving users.

RE-SCOPE AND RE-PLAN THE FEATURE

The bulk of the work came from decomposing the originally planned feature into smaller stories and creating a revised, incremental delivery plan that would help us deliver a true MVP first and then, in subsequent iterations, a series of progressive improvements.

Flickr image source: http://tinyurl.com/on6nch5

I will not use this report to describe the detailed process of scoping a product feature. There are many good discussions around discovering the Minimum Viable Product or MVP. To be sure, it takes practice to learn how to be ruthless. Good PMs find ways to whittle away at a product or feature until there are no nonessential elements remaining.

But it takes a different set of skills to then carefully plan the gradual introduction of those elements over time. It can be tempting to load up on enhancements, especially as product adoption increases among customers and users. Discipline and care are traits to be nurtured and celebrated in your Product Managers.

RESET EXPECTATIONS

As I mentioned earlier, this feature, when complete, is intended to make a big splash with customers. In making the decision to postpone its deployment, I now had the delicate task of resetting expectations with our Sales team, with Professional Services and Support teams, and with the few customers and prospects we had been engaging with for testing and research.

The impact

The existing customers and new prospects that were pining for this feature were disappointed with the decision to pull and postpone though I believe the blow was cushioned in two ways. First, we notified them of our product decision several weeks ahead of the release instead of only days before or even worse, after the release.

Second, we took care to explain that we did not feel comfortable pushing a significant new product feature without appropriate user testing. Not to say that this relieved all the anxiety but it is more difficult to argue with the choice of product quality over speed to market.

One final note. The Product team did consider moving ahead with a nearly complete version of the new feature and including it in the upcoming release. The thought was that we would have the ability to turn it on for a few, select customers in exchange for some heavy usability testing. I will publish more on that approach in a separate article.

Look for more reports from theProductPath around planning products and releases, MVPs, and product culture here on PM Decisions.

More articles from our blog PM Decisions