Evaluate the available data for making better product investment decisions

In recognizing that many of the decisions we had made over this past year were not driven by or supported with real evidence, I decided it was time to change how the Product team gathered and used data in our product development process.

Metrics and the related analysis are as important to Product teams as they are to any other department. But in my experience, it can be hard to nail down exactly what data your Product team should be monitoring. Because we interact with so many other parts of the organization, we may find ourselves gravitating toward (or flat out borrowing) directly from what other departments are (or should be) measuring. For example, it may seem obvious for us to latch on to customer conversion metrics from the Sales funnel. But then we also recognize that Customer Success is interacting directly with actual paying customers so maybe Net Promoter Score is something we should be tracking.

I have not found a single, universal measure that works for every Product team at every stage of its growth. I think good Product teams must continuously evaluate what data they need to help them make the best decisions. It is safe, if not self-evident, to start with metrics that ultimately tie back to and support the organization's high-level goals. But settling for superfluous or vanity metrics would seem to go against the prevailing wisdom which suggests adopting metrics that are truly actionable.

Product teams must continuously evaluate what data they need to help them make the best decisions.

I was interested in advancing my own organization's capabilities around gathering and using product data. In this article, I will share what came out of the exploratory discussions I initiated with members from the Product, UX, Engineering, Operations, and Data Analytics teams. In a companion article, I will report on how this analysis led to the next set of decisions to improve our overall proficiency.

What drove this decision

If you were to ask me to defend any of the decisions I've made with the teams over the past 10-12 months, I would not be able to reach for charts or graphs that clearly show one or more metrics trending up or down over time. In fact, if pressed, I might rattle off mostly anecdotal evidence like:

• We've experienced little to no push back from internal stakeholders about this past year's product roadmap
• There has been a noticeable absence of customer complaints around this year's product releases
• The Product team and I collectively lack any real regrets in our decisions so far (i.e. no major screw-ups)

Oh, and we closed two of the largest deals in the company's history this past year!

Had we been getting lucky by landing indifferent customers? Had the Product team simply taken the easy path or picked the low hanging fruit to avoid complexity or confrontation? I don't think so. I think the more likely story is that we had sensibly chosen to tackle the most pressing/glaring issues that were the highest priorities for all parties.

But it is not as though we were operating in a bubble. Along the way, we had certainly been talking directly with end users and also indirectly with the internal folks who themselves, talk with customers. But many times we pushed forward without adequate data and that hindered our ability to understand the impact of the changes we were making to the products.

I knew we could make better decisions if we had more information with which to work.

The decision: Confirm with the team exactly what data we were missing and what we would need to do to get our hands on it.

I want to be clear that this was an internal Product team endeavor driven by a desire to improve our own capabilities. Good or bad, the organization had not reached a point where my team was responsible for reporting KPIs or similar measures to our senior leadership or other internal stakeholders.

But before I could dream about some impressive analytics dashboard that might make Edward Tufte smile, I thought we could start small and work our way up from there.

Plan of attack

There was unanimous agreement that the Product team would benefit from collecting both qualitative and quantitative data. Thanks to an outstanding UX team, we had been making steady progress with capturing and evaluating qualitative data from customer interviews, surveys and the like. That work had indeed driven some major initiatives including the release of our new product earlier this year.

We determined that we would focus instead on complementing what we already had with more cold hard numbers from our own customer databases.

Focus on understanding feature usage to assess the impact of changes

The team concluded that the most obvious place to start would be to get a better grip on which customers were using what features and how. Much of our ongoing work would continue to revolve around revamping the existing platform components to provide a better user and administrative experience for our customers' primary use case.

Our challenge was knowing the size of the impact of making changes. Which customers would be affected? What migrations would be necessary? How much revenue would be at risk?

I shared the following, relevant story with the team to drive home the point:

Earlier this year, I had to push back on an anxious Customer Support team who was, as it turned out, overly worried about an upcoming feature migration I had scheduled for the entire customer base. Based on no real evidence, they had gotten themselves all worked up over what they believed would be a disruptive conversion. After running some simple reports, I concluded that less than 50 customers would be affected and most of them would barely notice as they had not really invested much in the (soon to be) legacy version of the feature. In the end, we pulled off the feature migration without a hitch and never received a single customer complaint.

Update our product hypotheses to include outcome-based metrics

The Product team had been getting better at creating problem hypotheses to help drive individual enhancements and sometimes, even entirely new products. The hypotheses helped us focus on real problems and generally followed this pattern:

We believe that [user persona] is struggling to [complete this task, achieve this outcome, ...]

I suggested to the team that we look to extend the hypothesis statement to incorporate a measurable result as suggested in this Medium article by Chris Abad:

“We believe if we provide [solution] to [customer], it will result in [outcome] as measured by [measurable success metric].”

If we could tie measurable outcomes to the problems we were attempting to solve, it would certainly give us some specific metrics to zero in on.

Help the team with product and design (re-)discovery

I am convinced that the 10-year-old software platform I inherited had accumulated much more code than was necessary to attract and retain customers. So in an attempt to reduce the size of the product set, I committed to devoting some time in every upcoming release to begin paring down the code base.

But we needed to be prudent about how to do this. As the teams looked to rebuild single pages or overhaul entire features, we desperately wanted to know what had to stay and what could be eliminated. We agreed that we would be able to draw good data from our existing customer base to help make smarter design decisions and prevent us from (re-)building stuff that is not needed.

The impact

I felt a little deflated at the end of this week's exercise. In the back of my mind, I had known that we should be operating better but it did not truly hit home until we assessed our current situation. It was a more than a little frustrating that we had so little data to go on.

Ours was not the first Product team at the company and our predecessors apparently had tried similar efforts in the past. I tracked down a previous Head of Product and learned from him that this had been a struggle for him in the past as well.

I am not discouraged though and am prepared to lead the team through the next stage in this process. Look for the companion article that describes the results of our push to acquire the product data identified here.

Look for more reports from theProductPath around product data, metrics & analysis, and product culture here on PM Decisions.

More articles from our blog PM Decisions

Practice humility in product pursuits

Flickr image source: http://tinyurl.com/od9nlno

In recognizing that the string of recent product successes had (perhaps!) inflated my ego and that that might likely affect my judgment, I decided to devote some time to more sobering aspects of being a Product Manager.

It is easy to get swept up in your team's triumphs, to celebrate (even minor) victories, and to take excessive pleasure in the praise others shower on you when times are good. I am as guilty as anyone of basking in the glow of success. Hitting your goals is satisfying indeed and it is tempting to linger there awhile in that satisfaction.

But it can affect your judgment if you get too caught up in your own hype. I was starting to feel like that was happening to me and I wanted to course correct before I began making poor product decisions. Surely there were some facets of my job or details of the product that were not going so well. I wanted to make sure I wasn't ignoring or just glossing over them. One careless misstep could effectively evaporate all the admiration.

What drove this decision

Credibility is crucial in this role. Product Managers must find a way make forward progress with little to no authority and to do that effectively, we must establish ourselves as capable decision makers. The challenge, of course, is that it takes much longer to build credibility than it does to lose it.

I have found that it is often a long march to establish good standing for yourself in the organization. But you can't get there - or hope to stay there long - if you misrepresent yourself by glossing over the many mistakes you make along the way. You have to acknowledge failures to your stakeholders, to your teams and to yourself.

I did not want to lose any of the forward progress I had made and that meant coming down out of the clouds and remind myself that I was still fallible.

The decision: Balance out all the congratulatory feedback and even my own eternal optimism by purposefully exploring activities that would knock me down a notch.

It seemed to me that a good way to knock myself down from the pedestal would be to engage in the kind of activities where there was no actual way to win. I would, therefore, seek out situations that would test my confidence but where the outcomes were not likely to be favorable for me.

Plan of attack

I found good opportunities to help me reset my ego. During this particular week, I tried out some new ideas that would serve up some humility. I think part of what made these exercises effective for me was that they were somewhat disconnected from my role in the organization. I did intentionally seek out product-related activities, but I would not be taking advantage of my job title to affect the respective outcomes.

Taking a turn at being the interviewee

I volunteered to help a colleague who was running a set of user tests for a mobile application. Specifically, he wanted to interview me as a target user of the app. His product initiative was unrelated to my company which meant less personal risk for me. In fact, I convinced myself that, should there be any missteps, my blunders would only be exposed to one other person.

And boy did I have some blunders! Well, technically, it is hard to say that you can get anything wrong in a usability test. As my colleague reminded me, "we are testing the app, not you." I have used that exact phrase in trying to set proper expectations when leading my own experiments, but it is hard to shake the feeling that, as an interviewee, you're not performing adequately.

At one point in the exercise, I thought I had successfully completed the task although it took me longer than I thought it should have. Must be the app I thought. Then, when I was politely informed that I actually had NOT accomplished the task, I was truly embarrassed.

when I was politely informed that I actually had NOT accomplished the task, I was truly embarrassed.

Being on the other side of the table helped me better understand the anxiety that comes with being interviewed. In the long run, this experience will undoubtedly make me a better interviewer, but I will admit that it had the sobering effect that I was looking for.

Reeling from a wave of negative customer feedback

In this same week, I had started to receive some strong reactions from our customers about a recent feature overhaul and none of it was positive. I can't think of anything much more humbling to a Product Manager than hearing that you may have got it wrong: "Can we just go back to the way it was before?"

The feedback was being collected by and funneled through our internal Customer Success team and we took it very seriously. Of course, my instinct, shared by the entire Product team, was to talk directly to the customer. We had, in fact, been doing that throughout the entire product cycle and had already incorporated earlier feedback into the feature through incremental releases.

"Can we just go back to the way it was before?"

Immediately, we went back to review our notes from these previous conversations which included several sessions with one of the customers who was now griping the loudest. I will admit that it is difficult to keep a cool head when intense emotions are involved.

I found this to be the perfect opportunity to revisit my convictions as a Product Manager, to reflect on how much better I still needed to be, and to remind myself how this particular job really never gets easier.

Proceed over a panel in a roomful of peers

And just to pile on even more, I also volunteered to participate as a moderator in a panel discussion for a group of Product Managers from the local community. I had never been a panel moderator and had never worked with this organization or its members. 

Eleanor Roosevelt is credited with saying "Do one thing every day that scares you." Well if you are short on ideas, let me suggest trying your hand at moderating a panel of experts, most of whom are far more experienced than you, and guiding the conversation for a roomful of practitioners in your field.

The impact

Flickr image source: http://tinyurl.com/ph3qp6j

I certainly felt the impact from each of the activities. Any one of them might have been sufficient but combining them all in a single week certainly helped me achieve my goal. And even if you're not working toward the exact same objectives, I can still recommend following some of these pursuits:

• Literally put yourself in your buyer's/user's shoes to help you better develop empathy
• Prepare yourself and your teams for confronting product mistakes so you will know how to move forward
• Do something that scares you, especially around your peers

Look for more reports from theProductPath around product culture, credibility, and the PM role here on PM Decisions.

More articles from our blog PM Decisions

Celebrate product wins big and small

Flickr image source: http://tinyurl.com/ppparbe

It's easy and obvious for everyone to come together and share in the company's larger victories like closing a big deal, but I decided to devote some time to acknowledge, if not revel in some of our minor triumphs too.

Last month, we executed the largest sale ever in the company's history. And while there was no small amount of boisterous high fiving going on over in Sales, it was a great opportunity to bring all the departments together and spend a few moments cheering as a single group.

A big product win too

I was celebrating of course but for a slightly different reason. I was particularly encouraged by this deal because of how it reflected on our current product. It would seem that we had met the customer's needs exactly. After concluding a long sales cycle (not uncommon for an enterprise software company), we signed a contract with the customer without having to promise any changes in the core product or adjustments to the roadmap (quite uncommon for an enterprise software company).

Rarely have I been able to avoid the uncomfortable product roadmap discussions with well-meaning prospects on one side and hovering, commission-obsessed sales reps on the other, trying to move the deal along by finding the right words that would address the urgent needs for "missing" features.

Product Managers dream of building the exact right product for their target customers and when a big one lands, you have to feel good about it.

So this was indeed a victory for Product and also for our dear friends in Product Marketing. Product Managers dream of building the exact right product for their target customers and when a big one lands, you have to feel good about it. But those victories can be short-lived as all eyes inevitably turn toward the next challenge.

What drove this decision

It is hard to say if or when we'll ever top this milestone, but it was another event that happened this past week that made me stop and think about the smaller accomplishments that also deserve recognition.

We had recently rolled out a new product which has been well-received by customers. One, in particular, an early and avid adopter, had recently become frustrated when a change to their own production environment temporarily "broke" our app. They called up and asked us to help remedy the problem because they couldn't go back to the way things were before using our app. 

I immediately thought of that great 1-question survey popularized by Sean Ellis that helps Product people test for this exact outcome: if enough users respond by saying that they would be "very disappointed" if they were not able to continue using your product, then you should feel confident that you are the right track.

We had experienced that exact outcome and even if it was only 1 customer, I still call that a victory! And I wanted to play up those wins with the troops too.

The decision: Recognize and applaud the lesser product successes too.

As a senior team member in the company, I have more visibility than most when it comes to departmental activity. I see the accomplishments being made every day and now I wanted to share those positive results with the appropriate parties who were not always directly connected to the action.

Plan of attack

My loose "plan" was simply to keep an eye out for opportunities where I could funnel reactions and reports back to the folks who would otherwise not hear of them. Along the way, I was a little curious to learn that there were not enough regular channels or venues for doing this and that sometimes, I had to get a little creative. 

Share glowing feedback from a significant product demo

Early in the week, I delivered a custom product demo for an important, external party who I would position somewhere between future business partner and potential investor. Several times during the demonstration, I received compliments including an amazing, unsolicited comment from one of the women who was "weeping a little," wishing she had had the chance to use our product while working for her previous company.

At the very next Engineering standup meeting, I relayed the flattering remarks with the team. Some rolled their eyes at the glowing hyperbole, but I could tell they appreciated the message. 

Appreciate engineering feats

Flickr image source: http://tinyurl.com/olfywby

Later in the week, we reached what I called a golden spike moment, referring to the 19th-century achievement of connecting the two sections of the First Transcontinental Railroad. In our case, two Engineering teams had connected a major new feature with its corresponding new configuration page.

We were now able to demonstrate how to build a new configuration and then immediately use that to launch the end-user tool. Achieving this (long overdue) task meant that we could now address a key pain point for our customers. I celebrated by delivering artisanal pastries to the teams at the sprint planning meeting.

Join in the "New features" huddle

The Product team was very excited to have rolled out two new, high-profile features in the last major release. However, we knew to temper our enthusiasm as it often takes several weeks or even months before customers latch on to and begin utilizing them in any significant way. Such is the pace of B2B software.

But one early indicator of success is when our own internal implementation team picks up the new stuff and starts incorporating into their projects. During this same week, we were pleased to see our folks schedule a huddle to share some early tips and best practices with each other - a true indication that we had delivered something of value. 

I invited the Product team to listen in on the discussion and together we basked in the indirect validation as we heard stories about how the new features would ultimately make their jobs easier and deliver better value for our customers.

I celebrated the win with the Product team over a nice lunch (which I ultimately turned it into a working lunch to conduct a customer journey mapping exercise - ha!)

The impact

You don't need always need to celebrate with free food, but I have found that people do like to feel appreciated. Our teams have certainly responded to the positive reinforcement and I will continue to keep an eye out for opportunities to share good tidings.  

Look for more reports from theProductPath around product culture, product feedback, and validating products here on PM Decisions.

More articles from our blog PM Decisions

Brainstorm a product solution for customers

Flickr image source: http://tinyurl.com/okhuqz2

After collecting far too many stories about how customers had resorted to stitching together different parts of our product to solve the same problem, I decided to regroup with the extended product team to revisit how we might address the actual needs of our users. 

I can measure the age of some of our primary product components in terms of years. This has positive and negative implications. On the plus side, I have a wealth of real-world application to draw upon as generations of users have been recording feedback with us about what is and isn't working (it is not uncommon for us to work with multiple Administrators at the same company due to turnover). All this input gives the team and I a wealth of information to sift through to find patterns of use.

On the other hand, I now also have both a significantly hardened code base that is more difficult to change and a larger user base that would be affected by any serious product revisions.

What drove this decision

I first started seeing the patterns emerge when I was embedded with the Sales Engineering team. Customers were asking for a way to start the same business process from two different points. We had built a tool specifically to accommodate the first case but unfortunately, to address the other starting point, we and ultimately our customers, were forced to reach for an alternative component.

Administrators found that configuring this second approach was harder but what's worse is that the resulting activity provided a completely dissimilar end user experience!

When I spoke with our Professional Services team who often assist new customers with their initial implementation, I confirmed that this was a common scenario and there was indeed pent up demand for a better solution.

The decision: Go back to the drawing (white-) board to review this prominent customer use case and find a more suitable way to solve this for our users. 

I didn't feel like we needed to start with a blank page. After all, our customers had all found a way to make our software work and many of them had taken the exact same approach to get there. What was bothering me was that we hadn't given them an obvious solution path and more importantly, the end user experience was disjoint and reflected poorly on the product. 

By gathering in one place the sharpest minds from each relevant department in our company, I would have an opportunity to apply some good, creative thinking to the problem. But leading a group discussion like this, especially with multi-disciplinary lineup would require some planning and coordination.   

Plan of attack

I had been gathering details around this problem for months, and so I set about reviewing my notes to create some initial hypotheses. I poured through my notebooks, Jira stories, customer interviews, and related research to organize my thoughts.

In the end, I had some solid starting points for the discussion but stopped well short of outlining a final solution for fear of stifling the creative process.

Prep ahead of time to streamline the conversation

I had some presumptions for how I thought (hoped?) the brainstorming conversation would go. The day before, I stayed after work and put some initial thoughts on a big whiteboard. My intent was to use this as a backdrop for the dialogue.

In my experience, I have found that it is much easier to get people talking when they already have something to bounce ideas off of even if the starting material is somewhat inaccurate or incomplete. 

Assemble a good quorum

I was genuinely interested in securing multiple points of view, but I also wanted to cover my bases so that the outcome could be vetted by all interested stakeholders. To that end, I invited participants from Engineering, UX, and Product as well as from Sales and Professional Services.

The latter was crucial for validating and clarifying the customers' challenges. The former ultimately helped me discover a product solution that was feasible and usable.

Have open-ended questions ready

Brainstorming often works better when there is some amount of prompting involved. I had already planned to facilitate but in a larger group, it can take actual prodding to make sure everyone's voice is heard. I prepared questions, some technical and others around usefulness, to provide sufficient opportunities to participate.

Even if I had already known the answers to the questions, I would still have incorporated them as collaboration tactics to get the ideas flowing and to promote a healthy discourse.

Stay on topic

Brainstorm sessions are expected to wander a bit which can actually encourage involvement and inspiration. But I had scheduled a short meeting so I also had to make sure I got what I needed from the group to be able to move forward. The trick was to limit the number and scope of conversation topics.

I started with a clear topic outline and stated what I needed the group to help me achieve. A few times I had to gently steer the discussion to get it back on track or to park issues that would have derailed us or sapped valuable minutes.

The impact

My meeting of the minds was ultimately a success. We confirmed several of the core hypotheses and as a result, I was able to proceed with my product planning. I was also pleased to have several new ideas enter the mix as well. In fact, there was one outcome in particular that I did not anticipate and that ultimately proved to simplify the resulting solution.

The approach I described here is a more formal approach to brainstorming, but it worked well given my constraints. With a bit more preparation up front and some lightweight oversight for the actual discussion, I was able to achieve positive results in a relatively short period of time.

Look for more reports from theProductPath around product strategy and product culture here on PM Decisions.

More articles from our blog PM Decisions

Guide and advise fellow Product Managers

Flickr image source: http://tinyurl.com/plxquqy

After many rounds of discouraging conversations with less experienced and struggling Product Managers both inside and outside my own company, I decided to assemble and share some helpful resources for advancing one's own PM career.

I believe that we are still in the early development stages of the Software Product Manager (SPM) role and also, that there is a general lack of clarity around this particular breed of Product Manager. That would certainly explain the flood of books, articles and similar resources all aimed at helping PMs understand and navigate this career path. But along with those are an even greater number of online conversations that seem to add to the confusion by attempting to explain it.

Here are a few of the top links returned when searching for "What is a Product Manager":

• What, exactly, is a Product Manager?
• Product Manager 101: What Does A Product Manager Actually Do?
• A Product Manager's Job
• Product Managers: Who are these 'mini-CEOs' and what do they do?

For someone who has recently landed their very first Product gig or is just looking to transition into becoming a Product Manager, this visible and honest industry conversation is simply an indicator of how confusing it can be to get started. And while there are some terrific resources being developed, most new PMs don't even know where to begin. 

What drove this decision

Closer to home, I knew of three individuals at different stages in their own PM careers who were each struggling with the similar problem.

Arthur was already pursuing a UX calling and was a semester away from earning an advanced degree in that area but had spent enough time around Product people to become intrigued with the role and was now curious about whether he could do both.

Brandon had been tapped by his current company 2 years ago to take on a Product Management role after proving himself as a reliable project manager and was now convinced that he wanted to find his next Product opportunity to continue down this path.

Carl had been a Product Manager for several years but had received no formal training and while he had mastered the more tactical activities of getting customer requirements through Engineering to deliver features to customers, he didn't know where to start to improve beyond that.

I found myself in impromptu mentoring conversations with all three in the span of a single week. I took that as a good sign to revisit my own collection of PM material and pull together suitable resources for at least these three practitioners.

The decision: Create simple orientation assignments for Product Managers who have a genuine appetite for advancing their careers. 

The challenge was not finding relevant material but rather curating the collection to zero in on the items that would have the biggest impact. I was looking for artifacts that would be easy to consume, that would provide good direction in the short term, and that might inspire them to explore topics even further on their own.  

Plan of attack

These three colleagues were in different stages of their careers but I knew that all of them would benefit from some strong, foundational material. There is a growing body of informational resources for the SPM but I didn't have a place to point them to get started.

So I published my own list.

Required Reading (AND LISTENING)

Inspired: How to Create Products Customers Love by Marty Cagan

The first question I always ask Product Managers who are just getting started is whether they've found and read Marty Cagan's book, Inspired - How to Create Products Customer Love. I have received nothing but rave reviews from every person to whom I've recommended this book.

I consider it required reading -- and re-reading. It is easy to consume and is filled with great advice for PMs at all stages. But perhaps the greatest benefit for an early-stage Product Manager is that Cagan clearly outlines the role itself and distinguishes it from the related but often overlapping functions of Project Manager, Product Marketer and UX Designer.

And If you need a break from reading, I can't recommend enough the amazing podcast, This is Product Management, created and hosted by Mike Fishbein. Each episode provides a fresh and unique perspective on this incredible role. Mike recruits inspiring guests who generously share their experiences working in and around the Product function. 

Your New PM Job

Gopal Shenoy provides a Getting Started guide for PMs

I then turned our attention to what Product Managers should be thinking about in their early days on the job. Some years back, I came across this wonderful article written by Gopal Shenoy called Software product manager’s first 30 days at a new job …. In it, he lays out a great guide for what good PMs do (or should be doing) in their first few weeks. The list is extensive and a little intimidating - perfect for setting a new PM's expectations.

I encouraged Arthur and Brandon to digest Shenoy's 30-day list and create a distilled version for their own use. And if they made it through that, I further suggested that they read his companion article on what to do in the next 45-90 days.

Brandon is actively interviewing but had not yet found some of the great resources that the PM community has contributed for job hunters. I encouraged him to take a look at one particularly interesting app at http://www.thepminterview.com. The author leverages material from Cracking the PM Interview and Decode and Conquer to create an online, interactive exercise where you can practice your answers to sample interview questions.

Formal Training Programs and Peer Groups

I pointed Carl, who was already embedded on a Product team, to look into cursory workshops and/or full, multi-week courses created specifically for Product Managers. There has been an increase in the number of online or self-guided options from companies like Udemy, but I encouraged Carl to look at local options such as General Assembly's 10-week course or the Pragmatic Marketing program. I explained to him that he would enjoy the added benefit of meeting and networking with peers in his own community.

And speaking of networking, I am a big fan of getting plugged into monthly meet-ups and similar informal, after-work forums related to your career choice. Every month, more and more opportunities spring up for Product Managers to meet and share ideas. I told Arthur, Brandon, and Carl to sign up for an upcoming event in our city and even offered to accompany them.

On the Job Training

Fall in love with a product framework

I prefer to give tactical advice when I can. So I followed up on my previous recommendations (read this book, take this class, join this meet-up, etc.) with a single proposal that I would expect each of them to incorporate differently on their own product paths: fall in love with a product framework.

Over the years, the greater Product community has created and shared many formal templates, authoritative checklists, proven models and reusable processes. A few of these attempt to address the entire product lifecycle but I believe that would be exhausting for an up-and-coming PM to consume, much less apply in practice. Instead, I advocate that new Product Managers zero in on a more discrete area, by starting with what interests them the most.

I used this approach when I was first getting my bearings. I was overwhelmed by all the material and product experts I encountered and was struggling to find any method that I could confidently apply. I eventually discovered Marty Cagan's Opportunity Assessment and used this as a foundation for my building my own method.

And I wouldn't discount the searching process itself. You will inevitably stumble onto new ideas just by sifting through the still disperse product knowledge base to find frameworks that interest you.

Volunteer to do a PM's Job

And what do you do with this new framework you've recently embraced?

I was once told that the best way to get into Product Management is to start doing it. This guru urged would-be PMs to hunt down the closest Product team (or whomever is performing that function) and offer to assist. As someone running a Product team, I genuinely like that approach. But I would argue that showing up empty handed is not as attractive as arriving with expertise around some useful tool that the Product team can use to improve discovery, planning, or execution.

The impact

Ultimately, it is up to Arthur, Brandon, and Carl to move forward at their own pace. I'd like to think my advice is helping them to navigate but I feel like we're still far away from having any kind of proven curriculum. Despite that, I'm seeing more and more people gravitate to this kind of role - and I'm encouraging it. The Product community is fortunate to have professionals from so many disciplines join our ranks, making us a much stronger collective.

Look for more reports from theProductPath around Product teams, PM credibility, and the PM role here on PM Decisions.

More articles from our blog PM Decisions

Fire up the Sales team

Image source: https://www.flickr.com/photos/spacelion/3008520385

After being invited to participate in our mid-year, internal Sales team rally, I decided to deliver a rousing product presentation to get the troops pumped up.

Over the course of my career, I have spent months embedded with various Sales teams and I would recommend it for any aspiring Product Manager looking to better understand their stakeholders. Some of their customs, however, may still be a bit peculiar, at least to me. In particular, there is this practice of routinely gathering the entire Sales team for an entire day or two, typically somewhere away from the office with the intent of re-energizing everyone.

I know that the Sales team, like most other teams, have their own set of priorities and that they don't often stay as connected to the products as I would prefer. So I find it valuable to take advantage of occasions like this to better "align" myself and our products with the Sales team.

What drove this decision

We had recently hired some new folks into the Sales ranks over the past few months and the entire team had been focused on rolling out an updated sales process with our revamped Marketing team. This meant less bandwidth for absorbing the last few rounds of product updates.

Everyone in the company is invited to attend our monthly Release Previews of course but Sales attendance, in particular had been somewhat spotty for awhile.

Now, with this mostly captive audience, I had a chance to focus everyone's attention on our Products and to remind the team how much progress we have made in the last six months.

The decision: Weave together a powerful before/after story that relates the numerous product enhancements as well as our impressive rate of innovation to stir the team and inspire them to carry that message to our customers.

I can't blame our Sales reps for settling into a comfortable routine and sticking with a story that works for them. Asking them to change up their narrative every time we release new software is a little unrealistic. But I knew that the product improvements we had made over the past six months had been significant and would have a big positive impact on the way we all were engaging with our customers.

Our company had indeed devoted a great deal of time to understanding our customers' pain and I knew we could speak to that confidently. But I had sat in on enough product demos to know that there were plenty of rough spots - areas of the product that didn't show particularly well. Much of the Product effort in the first half of the year went to address these specific issues and our story needed to be refreshed to incorporate every one of those Product improvements.  

Plan of attack

They gave me 60 minutes to dazzle the troops.

They gave me 60 minutes to dazzle the troops. Even the best slides would not likely keep their collective attention for that long so I supplemented a PowerPoint presentation with live product demos and tantalizing prototypes to help keep them captivated.

The goal for me was to have everyone walk away with renewed pride in our company's product and more importantly, for each of them to feel even more confident that we were truly the best option for solving our customers' problems.

Create a backdrop for the main narrative

The first step was to set up a familiar context around which I could piece together the different elements of the story. As a framework for my slides, I decided to use a 1-page diagram I had created months earlier as a sales aid to help orient our prospects and drive productive sales discussions. I began by highlighting a number of improvements I had made in this iteration of the diagram to help me grab the team's attention at the very start.

My (abbreviated) presentation flow for the Sales team tells a before/after story using the familiar customer process as the backdrop

Show old screenshots highlighting the known bad spots

The next step was easy. I found old screenshots from the recent past and positioned them on top of the diagram. This had the intended effect of reminding everyone how hard it once was to brag about our solution. During the presentation, I intentionally exaggerated the struggles associated with this familiar but outdated description of our product - but concluded that this awesome team was still able to sell that version. The good news is that it wouldn't get any worse!

Replace with new screenshots

Then, one by one, I swapped in the new hotness. Gradually, I unfolded our updated story to the group showing how all our pain points had been (or would soon be) addressed. Even better, the Product team had introduced entirely new features that helped to strengthen our overall story. The resulting picture not only improved on the known problems but gave the Sales team even more talking points and would help them address customer issues that we might have dodged in the past.

Tease with early prototypes

But wait - there's more! Stacking up all the existing enhancements on a single slide was certainly effective but to really energize the crowd, I then switched gears and started demoing some of the new stuff that was just around the corner.

Using live code that was still working its way through QA along with fancy, clickable prototypes created by our UX team, I began to weave an even stronger narrative for our Sales team to use immediately with their prospects and customers.

The impact

My talk had the intended effect and I was certainly encouraged by all the positive feedback from the team. I do realize that the real payoff for the company will only come when they carry this updated story to the front lines. 

I remember days in recent past where Sales Engineers would brush past certain areas of the product or skip them entirely hoping that the prospect wouldn't notice. I also remember the collective cringe when a savvy customer would ask to explore the next level of detail behind a particularly well-manicured product demonstration.

With this presentation, I made it clear that we would no longer have to avoid those areas of our product. Instead, I urged our team to intentionally stop and even linger at certain points in the demo to prompt more productive conversations. I wanted our customers to ask those tough questions and engage further with us. We had even more reasons to be proud of our company's products and what's more, could now point to an impressive rate of product improvement over the past six months.

Look for more reports from theProductPath around product review, product culture, and socializing product roadmaps here on PM Decisions.

More articles from our blog PM Decisions

Triage an almost bungled feature upgrade

Flickr image source: http://tinyurl.com/pp7daju

In recognizing that our pre-deployment testing had raised unexpected alarms with some customers and our own Support staff, I decided to straighten out the mess by creating a better feature migration path.

We were winding down the days to our next major product release and I thought the planning had been going well. There are always a number of moving parts in a big deployment and it takes no small bit of coordination across teams to pull off a trouble-free rollout.

One of the big announcements this time around was how we had overhauled one of our more prominent features to make it easier to use. And while we had usage data that showed it wasn't exactly mission critical for our existing customers, we still wanted to ensure a smooth migration.

What drove this decision

The upgrade approach we had planned was simply swapping in the new feature in the place of the old one. Any account that was actively using the old feature would continue to work for the 30 days following the release but after that time, all accounts would be have been automatically switched over.

This seemed straightforward enough and indeed, our internal testing and QA efforts proved that this was a clean transition. Where we went wrong however, was not effectively communicating the plan, especially to our internal stakeholders.

The most obvious blunder was in the Release Notes (I wrote) that we circulate internally, weeks before the go-live date. I had made a copy/paste error using some outdated text and reported the exact opposite intent around this migration! As you might imagine, the Customer Success team freaked out when the old feature "suddenly just disappeared".

All fingers were rightfully pointing back to me and my Product Team. What I thought had been weeks of good planning and internal discussion were negated by what was now a small crisis. I needed a good plan to set things straight.

The decision: Instead of forcing a more immediate upgrade on our users, I would stretch out the feature deprecation period and give customers more time and more autonomy to make the transition.

It is worth clarifying that nothing was indeed broken in any customer account and nothing would break as a result of pushing the new feature in the upcoming release. Actually, I could have tried to smooth over the current unrest and push forward as we had originally planned. After all, no actual customer had experienced any problems in their accounts.

But in this case I chose a more cautious path. Even though it would be more work for our tech teams - actually, it would be re-work since we had already removed the old feature and now had to quickly retest the reverted code - I knew the extra effort to create an extended upgrade period would cost us a lot less than having to soothe any confused or disgruntled customers.

Plan of attack

Patience is a virtue - especially in times of crisis. When the voices are beginning to elevate, when email threads are growing exponentially in a matter of hours, and when emergency meetings are being scheduled first thing in the morning, it helps to be able to keep a calm head.

I would have to calibrate our response based on the actual size of the problem.

One of the things I wanted to make sure was that, in attempting to address the problem, we didn't make things any worse than they were already perceived to be. The key to that, in my opinion, was to properly scope the effort. In short, I would need to be able to calibrate our response based on the actual size of the problem.

Understand the full impact of the change

When we did assemble the interested parties in a meeting room and reviewed the situation, one of the first questions I asked was whether we knew the number of customers/accounts that would be affected by the deprecated feature. My own Product team had gathered some numbers weeks earlier but I wanted to know if we had been off in our estimates.

Because there wasn't enough certainty, I had the Engineers run a new set of exhaustive queries to get our hands around the total affected population. Early totals were supplemented by more thorough search results but all reports showed that the impact was far less significant than the recent hype would have suggested.

Flickr image source: http://tinyurl.com/ol3x3jr

Call attention to the deprecated feature

With the Engineering team's help, I committed to reinstating the deprecated feature into the product to help affected customers with their inevitable migration. But in doing so, I insisted that we incorporate some blatant visual cues for the end user so they understood that some action was required.

With minimal work, we were able to introduce a glaring design element intended to draw attention to the deprecated feature. This new call to action would be reinforced with other messaging inside the product and externally as well.

Broadcast (better) to internal and external users 

The steps required for a customer to move to the new, improved feature were straightforward and would likely only take a few minutes to complete. However, I have learned the hard way about underestimating users and so, to err on the side of caution, I committed to providing multiple migration artifacts.

I had the Product team scramble to develop new collateral including a 1-page cheat sheet and a short instructional video to help guide customers on the migration process. In both, we explained how the deprecated feature would be disappearing in the subsequent release to be replaced by the improved option available now and we included a recommended course of action to avoid any problems in their account.

The impact

In the post-mortem discussion, all the teams agreed that the new feature is a much better option for our end users - we had just stumbled a bit in our plan to roll it out to our customers. It was a good lesson for me and one that I'm happy to have tackled before the official release where it could really have escalated.

Look for more reports from theProductPath around product releases, product culture, and PM credibility here on PM Decisions.

More articles from our blog PM Decisions

Write the Release Notes

Flickr image source: http://tinyurl.com/ozc9zcy

After realizing my goto resource would be on vacation during the next big deployment, I decided to step in and take the lead on producing the key product documentation.

Our company publishes Release Notes for every major product upgrade. We create this formal documentation for existing customers mostly, as we look to highlight improvements and bug fixes to our software products and any changes in how we deploy, license, or support those products.

I have regularly contributed to this document but have never owned the task of generating and assembling the Release Notes for our product deployments. With each major product release, I have also looked for opportunities to incrementally improve the structure and details of this important piece of product collateral.  

What drove this decision

Flickr image source: http://tinyurl.com/njjshos

One of my Product Managers had scheduled some time off of work for a much-needed (but poorly timed:-) family vacation. Up until this time, the Product Team had leaned on him to lead the effort of producing the Release Notes. And he had always come through.

Each time of course, it had been a team effort as each Product Manager was ultimately responsible for capturing and communicating their team's respective work. But "Vacation Guy" had been the one to pull it all together and ultimately deliver it to the eager masses.

His absence would put us in a tough spot - but it also presented a good opportunity for me to pitch in.

The decision: Roll up my sleeves, dust off the technical writing skills and crank out the next iteration of our product release notes.

I was confident about writing the document itself and fully aware of the entire scope of the upcoming release. But beyond the heads-down task of cranking out the words and pictures, I wanted to use this occasion to advance the state of the product communication we use with our stakeholders.

Our new Product Marketing Manager has been pushing my product team to better connect the dots around product/feature positioning.

On a related note, the company had recently hired our first, full-time Product Marketing Manager (a great move) who among other things, is helping us with the many tasks associated with a product release. For example, she has taken over the Release Communication Plan that is used to orchestrate most of the customer-facing tasks like email notifications, blog posts and related content marketing activities, and promoting the post-release webinars. And much of what she requires for this Plan is derived from the initial draft of our Release Notes.

Among the many improvements she has ushered in has been pushing my product team to better connect the dots around product/feature positioning. For example, the work we do to create testable hypotheses further upstream in the product discovery and early development process were all but lost by the time we delivered the working solution. But that is valuable information that needs to be communicated to customers and end users.

Plan of attack

We generally organizes topics in our Release Notes like this:

TABLE OF CONTENTS

About the Release Notes

Release Overview

End of Life Announcements

Enhancements

• Major enhancement
  • Customer facing notes
  • Content-marketing fodder
  • Internal-only messaging
• Major enhancement
• ...
• Other enhancements
• Fixes

Known Issues

Getting started was not quite as easy as "Save as, Search-and-Replace" but we had developed a reusable structure for the document that made it easier to produce each new iteration. 

Internal first, then external version

I began by creating an internal-only version that contains more information than what was necessary for customers. This initial version provides a diversity of details around each topic and was meant to give our Professional Services and Support teams a better understanding of the impacts, if any for customers.

For example, on the extreme end, I might thoroughly describe the migration steps for users that are transitioning from a deprecated feature to a new enhancement. On the other extreme, I may simply reference existing collateral such as public-facing knowledge articles that are affected by the release and would need to be updated.

I delivered a draft of the Release Notes to the Product Marketing Manager early on so she could start to build the Release Communication Plan. I also circulated it to the rest of the teams to get all the wheels turning for the upcoming release.

More than just an enhancement list

The bulk of the content in the document relates to the product enhancements. Customers (and Sales teams) tend to focus on the shiny new things. I try to accommodate by separating and highlighting the big stories from the much longer list of smaller enhancements.

For each major feature, I started by revisiting the existing customer pain that prompted the product work. Sometimes I even included relevant feedback we received before and during the development cycle.

In describing the solution, I outlined the way the feature is used and included screenshots where helpful. I try to provide at least 1 example to further help orient the user.

With all this information repeated for each big enhancement, the document can often get quite lengthy. To counter reader fatigue, I reuse a consistent layout throughout this part of the document and break up the text with subsections, internal links and generous whitespace to make it easier to consume.   

Updates to primary customer narrative

Along with the general description of the major and minor features included in the release, I also posted refinements around our main use case to help the Content Marketing Team rework their editorial calendar to fold in new blog posts, videos, and other collateral related to the release.

The internal notes and supplemental use case-related material is helpful for connecting the dots between customer needs and shippable products but is ultimately stripped from the final, external-facing version of the document. 

The impact

By most measures, my turn in the driver's seat did not negatively impact our process. I was happy to have made a few, incremental enhancements and as with most product efforts, will be looking for feedback and validation from all stakeholders in the days that follow.  

We will go through several internal iterations of authoring and review the Release Notes before we finally publish it externally to customers, concurrently with the release going live. Often, we will work to refine the Release Notes up until a day or two before the release as last minute details are being ironed out. The finished version is deployed on the company website with any number of links from within the product, from support documentation, from the marketing site, and from outbound promotional emails.

Customers have responded positively to the level of information we provide in our Release Notes. I suspect we will eventually move away from the traditional PDF document to a more fluid web/mobile publishing solution. Additionally, we are starting to favor short videos to introduce each major enhancement but they take longer to create and are more difficult to update if/when the screens/pages change.

Look for more reports from theProductPath around product releases, the PM role, and product culture here on PM Decisions.

More articles from our blog PM Decisions

Unveil new product at company event

Flickr image source: http://tinyurl.com/obvs3hc

In recognizing the work that all the departmental teams had contributed to create this novel product, I decided to orchestrate a group presentation to roll out the new offering to the entire company.

I really enjoy our Release Previews. These are monthly get-togethers I created to have the Product team demonstrate, to the entire company, the latest, shippable product updates and enhancements. These meetings are intentionally distinct from the less formal Sprint Review, typically led by and targeted at Engineers.

That is not to say that Engineers are not part of the Release Preview - actually they are often featured as we are showing off their handiwork in most cases. But the broader messages we deliver to the company at the Release Preview are more value-driven, as in what value are these updates going to provide to our customers and to our own business?

We all know it takes a coordinated effort to build and release new products. During the Release Preview, I try to make sure we recognize everyone who contributes to our incremental progress and that goes well beyond Product and Engineering. But this was not our standard release.

This was the official launch of our new Reference App.

What drove this decision

This was a major achievement for our company and I wanted this particular Release Preview to stand out from the rest. After many months of planning, development, testing, and customer validation, we were ready to unveil a unique new offering that would differentiate our company - and it was important to me and the team that everyone in the company appreciate the collective accomplishment.

The decision: Preview the new product in a group setting where each department contributes to the larger story.

More important than getting everyone up to speed on and pumped up about the new product, I wanted them to all to understand and appreciate the work that all of us had done to make it a reality. We had shown prototypes of the product many months ago and despite small, periodic updates, many in the company had all but forgotten about it.

In the reminder email I sent to the company days before the event, I emphasized how this particular Release Preview was going to be an extravaganza. Now it was time to put together a presentation that lived up to the hype.

Plan of attack

The highlight of our Release Previews are most certainly the product demonstrations. But as I mentioned before, we are very prudent about wrapping those demonstrations with stories about the value that is ultimately delivered to the end customer. This presentation would be no different but the story would be larger and all-inclusive.

Recount How we gOt here

To start things off, I asked our head of Research to provide the back story. This app came out of one of the best customer research initiatives we had ever run and I couldn't think of a better way to set the stage. Straight from our customers' mouths, we heard about the struggles they had been having with our platform. Over and over, we were told the same thing - you have to give us more visibility into what's going on.

We listened, we prototyped and we validated. The reason I felt so confident in this presentation - and in this new product - was because it was practically dictated to us by our end users. 

Get representatives from each department to contribute their parts of the story

Then, one by one, I had representatives from each department stand up and speak to other facets of the product.

Professional Services described the lengths they would go to just to provide a makeshift remedy to the customer's visibility problem.

Engineering and Ops took turns talking about the speed at which we would be able to deliver updates to the product and how much our entire deployment had improved as a result of launching this app.

UX gushed about how the new app leveraged a recently developed component library and how it help to launch a new UI paradigm for the company.

Sales and Marketing weighed in with the overwhelmingly positive response they were getting from the early previews we had been sharing with prospects and customers.

And so on. By assigning each of them a dedicated speaking role and by giving all the contributions equal weighting, I was able to relate a shared victory for everyone.

GET THE CEO TO WRAP IT ALL UP WITH A ROUSING FINISH

I would have been happy enough with just the departmental contributions, but to end on an even higher note, I asked our CEO to supply his own thoughts on our newest product. In a brief but stirring, unrehearsed speech, he shared his appreciation for all our hard work and for the company-wide commitment to product innovation.

It was well-received by all. 

The impact

To say the Release Preview went well would be an understatement. We had the largest employee turnout ever for an event like this and I received a great deal of positive feedback afterwards. One of the recurring messages was that we might want to reserve additional, spillover conference rooms in the future to accommodate more of our folks in the office!

I thanked all the speakers individually for their participation and gently reminded them that we still had more work to do to make the official launch a success. At the end of the day, with all the solidarity we generated, I couldn't help but feel that success was, if not inevitable, then certainly more attainable.

Look for more reports from theProductPath around product reviews, releasing products, and product culture here on PM Decisions.

More articles from our blog PM Decisions

Getting ready to groom

Flickr image source: http://tinyurl.com/ogerhjp

In recognizing that too many of our user stories were not truly ready to be implemented, I decided to revisit the entire path to grooming

For months, I had been watching the Product team wrestle with getting their particular features out the door and in front of our customers. And it wasn't just one PM who was struggling or an exceptional delay in delivery - it was borderline chronic. More often than not, I could trace the issues back to poor preparation by the Product team.

I have been working with our entire tech team to continuously improve our own Agile development process and we certainly have more work to be done. But the problems I needed to address were further upstream, before the stories entered the Engineering and QA queues. The Product team was creating unnecessary thrashing by failing to do our part well. 

What drove this decision

When your product releases are scheduled less frequently (think months apart vs. weeks), there is a greater impact when you miss a delivery date. I was frustrated at having to drop features from a release because they weren't ready or seeing enhancements slip to the next sprint.

And while it is tempting to spread the blame across the myriad variables that go along with shipping product software, I became convinced that my own Product team had to step up to support our share of the contract.

Some product planning failures are easy to measure. Are features not being completed? Are your stories slipping over successive sprints and even releases? Are your Engineers and QA teams stalled waiting for last-minute clarifications around designs and requirements?

The decision: Formalize the team's story preparation process leading up to sprint planning

The bad/good news was that we had accumulated sufficient instances over the past few months to see some a pattern emerging. Early on, we tried using note cards and a whiteboard to better track the larger product initiatives over sprints and releases. This seemed promising at first, probably because the team appreciated an exercise that was both tactile and collaborative. But we ultimately failed to capture any incremental advancement of any initiative using the cards. Moving the cards across the board from sprint to sprint didn't make us better at preparation and worse, this exercise did not help us identify deficiencies in the planning process.

We needed a different approach, one that helped us focus on maturing product requirements BEFORE we gave them to the team. We needed a better product grooming process that preceded the actual backlog grooming sessions.

Plan of attack

The tactics described below were prescribed only for our larger product initiatives and would likely be overkill for bugs and smaller feature enhancements. We began by focusing on the problems that occurred most frequently, then folded in additional steps to mature the pre-grooming process.

STEP 1 - Include the fully baked UX designs 

Perhaps the most obvious problem for our team was that we were often scrambling to get our front-end designs in place by the time the Engineers were ready to estimate the level of effort. We had become less and less stringent about completing the UX work and ensuring that formal designs were attached to a story or epic prior to backlog grooming or worse, sprint planning.

The lowest hanging fruit, then, was to re-establish this specification for all our user stories. Going forward, no stories would be allowed into a backlog grooming session unless the designs were complete. Obviously, there is no real definition of "fully baked" as teams will inevitably tweak designs during development and even testing but this was a relatively easy first stake in the ground.

STEP 2 - Review the acceptance criteria with internal stakeholders

I'm of the mindset that you can never stop improving on your ability to define product requirements. I encourage my Product Managers to continue to work on this skill with each new assignment. But there is an easy to tell whether you are actually improving or not by gauging how often your stories are being kicked back by the Engineering or QA teams.

Flickr image source: http://tinyurl.com/le8jhxv

And even if you are scoring well with these groups who are much closer to the code, you probably have other stakeholders inside the company that can help you measure your aptitude. In our organization, the Sales Engineering and Professional Services teams are great early sounding boards. Another important group (for us and most organizations) is the Customer Support team. Each of these groups is intimately familiar with your customer base and can help validate how well you have done in capturing the happy path(s) and the larger set of user variations.

In our updated process, we have a specific gate for reviewing stories or at least the acceptance criteria with our internal stakeholders. This has the additional benefit of not catching them off guard during a Release Preview for example, when the feature is already "complete". 

STEP 3 - Weave in usefulness testing with end users

When it comes to testing products with end users, excuses usually outnumber opportunities. Like most Product teams, we would prefer to have more time to spend with our customers but scheduling these interactions always seems more difficult that it should be. Still, the team and I firmly believe this is a critical part of the grooming process - especially as the requirements and designs come together.

Going forward, we have made this an explicit stage in the lead up to backlog grooming. Of the steps listed here, it is the most challenging to implement but will likely have the biggest payoff. I can think of nothing better for removing doubt and minimizing surprise than having end users validate your product throughout your development process.

The impact

The Product team is still adjusting to the new approach although we all recognize the improvements that will come with better story preparation. We've already seen some of those benefits: in the past few weeks, we withheld a number of stories from backlog grooming that had not met the new criteria, saving us hours of guaranteed thrashing with the Engineering and QA teams.

Additional refinements will be necessary. As we advance and extend our customer research and user testing, we will introduce new gates in an effort to further solidify the product features and push them through our Agile process. 

Look for more reports from theProductPath around product validation, backlog grooming, and product culture here on PM Decisions.

More articles from our blog PM Decisions