Conduct high-priority product research

Flickr image source: http://tinyurl.com/pmvvurb

To address one of the most critical problem areas for customers using our software platform, I decided to roll up my sleeves and conduct the fieldwork myself.

For many months, we had been recording feedback from our customers, some direct, some indirect around one particular area of the product. Everyone agreed that we had delivered really powerful features - the common challenge was in the setup and configuration. We had made it too complex to use this valuable piece of the product! 

I couldn’t agree more. I knew how hard it was, even for more technical users to configure the tools. I have a deep programming background but even I would struggle trying to ramp up on proprietary schemas with cryptic configuration syntax and poor documentation.

We had been attempting to compensate and even head off frustrated customers by offering to have our own Professional Services team help get customers up and running. But that only served to delay the inevitable as the complaints would then change to “even if you build it for me the first time, I’m not confident I can take it over and maintain it!"

What drove this decision

I had been anticipating a suitable window of time where we could focus on revisiting and rethinking these problems. The teams had been making good progress on some of the pre-requisite components that would be leveraged to rebuild this key feature of our platform. It was time to initiate a round of interviews with customers about how they use the current product, what (if anything) they liked about it and what (again, if anything) they could point to that causes problems for them or their end users.

I wanted to take full advantage of this opportunity to get out of the building, to speak directly to users and reinforce this important research activity with the members of my Product team. Mostly though, I wanted to improve our product and make our customers happy.

The decision: Enlist the help of our User Research expert to coordinate formal interviews with a suitable group of administrators and end users from our active customer base.  

I am fortunate to have a talented resource on our team who has raised the bar around user research for our entire company. She continues to guide us all on the finer points of interviewing users early on around product discovery and later on for product usefulness and usability. With her by my side, I felt very confident that we would get to the crux of the problems and ultimately, be able to prioritize the work to come.

While I was eager to get started and optimistic about what we would learn, I heeded my colleague’s advice and adhered to the plan we made to ensure a favorable outcome.

Plan of attack

It was so tempting to just start writing requirements. But I've been wrong before.

I have used our product internally and I have also demoed it hundreds of times to customers so I felt confident that I knew what was wrong and what would need to be fixed. It was so tempting to just start writing requirements.

But I’ve been wrong before. Let me say that again, I’ve been wrong before. I have felt strongly about how our product should work and went straight to the Engineers with requirements only to find out, after they had spent valuable time implementing the stories that there were no customers lined up to use the new enhancements.

Start with hypotheses and a good prototype

Because I was so familiar with this particular part of our platform, I was able to quickly compose a lengthy list of hypotheses of pain points where I would expect to hear complaints from both heavy and casual users:

• We believe Admins are struggling to initially configure a URL to set up a new instance of [tool] in environments like Salesforce.com.
• We believe end users are frustrated with the number of clicks required to complete the ... process, especially if only one document is necessary.
• We believe Admins are resisting adopting and expanding their use of [tool] because it lacks good error handling, is difficult to test, and has too few options for controlling its behavior for the end user.
• We believe end users struggle with scrolling and otherwise navigating through the different areas of the page, e.g. with long lists of templates or with very long data forms and may find it helpful to track incremental progress as they work through sections of a longer form.
• We believe end users are underwhelmed with the stark layout of the [tool] page, the lack of branding, and the lack of any help text.
• We believe end users struggle with using the [tool] with different browser window sizes, specifically when the buttons move around when the page is resized.
• ... 

I then had the UX team mock up a clickable prototype that addressed some of the primary concerns and that would allow us to better engage with our interviewees.  

Recruit and track participants

To build a list of participants, I first reviewed the customers who were already part of our formal “labs” program (always be recruiting!). Then, I put the word out to our Sales and Professional Services teams to identify other viable customers with whom we could talk. The resulting list was manageable enough for us to track using a shared spreadsheet that included contact information, interview dates, and miscellaneous notes.

Flickr image source: http://tinyurl.com/nomqgdl

Contact Names	Organization	Preferred Phone/Email	Participation	Last Contact Date	Future Participation	Primary Internal Contact	Notes
Nellie Bluth	The Bluth Company	nbluth@thebluths.com	Asked to be included in Fall research initiatives	7/12/15	Also willing to help with...	Maggie Lyes, Acct Mgr	Has been helpful in the past but admits to not being technical

Conduct and record the interviews

When we had accumulated a sufficient list of suitable users, I crafted and sent out an email invite along these lines:

“Hello Vincenzo, I would like to enlist your help for a product research initiative.

I head up the Product team and am leading an active project to learn more about how customers are using our tools. I am reaching out to you because your group is currently using [tool]. I believe you would be able to provide some great feedback on what is and isn’t working well.

If you are interested in participating, the next steps are easy:

1 - Reply to this email with some available interview times over the next 2-3 weeks.
For example, “Tuesday mornings are better”, “tomorrow”, “Wed/Thurs between 3-5pm Pacific”.

2 - Prepare to walk us through how you or your end users use [tool] today

This is a great opportunity to tell us about your successes with [tool] as well as things that would make it easier for you. The conversation should not take more than 45 minutes.”

Not surprisingly, our more fervent users responded immediately. With the others, it sometimes took a little more effort to secure the interviews. We mostly used virtual web meetings to conduct and record the interviews where the customer did most of the driving while we watched.

Our agenda for the interviews followed this path:

1. Show us how you use the product today [50% of the interview time was spent here]
2. As an [administrator, end user], what would you say if anything, are your favorite things about [tool]? [10%]
3. What, if anything causes you or your users problems when using [tool]? [20%]
4. Switching gears, we'd like to show you our ideas for a new design... [20%]

I had an intern transcribe notes from the recordings, putting all the material in a central repository so we could index it properly and share it with the rest of Product and Engineering.

The impact

The customer interviews went smoothly giving me exactly what I had hoped to obtain. There was plenty of validation around our current shortcomings, some surprises about certain attributes that users found valuable, and an abundance of constructive griping. The early prototype seemed to be a hit with users and gave me additional confidence to pursue that path.

I expect to deliver a first round of product improvements in the very next release and will continue to engage these customers to validate our efforts.  Look for more reports from theProductPath around customer research, product validation, and feature prioritization here on PM Decisions.

More articles from our blog PM Decisions

Respond to unhappy customers

Flickr image source: http://tinyurl.com/orb4qg5

To learn more about post-purchase sentiment, I decided to engage with our Customer Success team and become more aware of where our users were struggling.

I will admit that my Product team and I tend to spend more time interviewing and talking with amicable customers. That is not to say that we only approach our happy users; in fact, a fair number of the research-oriented conversations have been tempestuous. Indeed, through our collective research efforts over the past year, we have collected quite a bit of product feedback that would land well on the other side of "needs improvement".

But in this exercise, I wanted to turn my attention specifically to customers who were in peril. And I wanted to go straight to the front lines.

What drove this decision

In addition to what we were learning from the interviews we had been conducting with customers and prospects around new features and enhancements, I also wanted to know more about our product's pain points. And I thought that a great way to do that was to talk with customers who were currently lodging their complaints with our Support team.

The decision: Carve out time to listen to and understand some of the problems customers were having with our product and to help identify ways to address each distinct concern.

In span of a single week, I was able to participate in three separate support cases. I got a chance to hear, in some cases first-hand, from customers who had legitimate complaints about:

1. An unacceptable implementation scenario
2. An (unintentional) product feature gap
3. A grueling Administrator on-boarding experience

These scenarios afforded me with a good cross-section of problems to explore. One was griping about our product's inability to satisfy their complex security problem. Another had caught a change we had made in a recent release that legitimately "broke" one of their use cases. And a third was trudging through our formal training course and was beginning to grasp all that he would ultimately have to learn to support his own company's solution.

Plan of attack

I was intrigued to have three customers with three very different concerns. My process for each was similar however. I spoke with Customer Success person who was interfacing directly with the users, tried my best to assess the particular situation and made sure I understood each customer's own challenge. 

In the end, I helped to resolve each situation but with three very different outcomes.

Scenario 1 - "Fire" a customer for which there was truly no fit

My first conversation revolved around a newer customer who had been planning to go live with their implementation in the next few weeks. In working through their initial project, however, they had determined that they needed a much more sophisticated security scheme and were now struggling with that part of our platform.

It did not take me long to recognize we were at an impasse.

Somewhere along the way, we had clearly misled this particular customer. We had since exhausted all the obvious and even more creative options and I confirmed that we would not be otherwise enhancing the product to support their unique use case.

It did not take me long to recognize we were at an impasse. Looking to quickly resolve the situation, we made the tough decision to rip up the customer's contract and return their money.

Scenario 2 - Escalate a support ticket regarding a product oversight

My next conversation was much different. Here a long-time customer had submitted a complaint with our Support team about a small feature that was removed in a recent product release. It was clear to me that we screwed up.

It was clear to me that we had screwed up.

After hearing their scenario, I agreed and admitted that we had inadvertently taken away a small piece of functionality that they had been relying on. I recommended that we escalate the Support ticket to trigger a feature enhancement request that would bring back comparable functionality to help restore their solution.

Scenario 3 - Meet face-to-face with an overwrought Administrator

And finally, toward the end of the week, I was made aware of another new customer who had been floundering a bit. The designated "administrator" at this company, who was in charge of making it all work was currently enrolled in our (highly recommended) training class to learn the ins and outs of our software but was laboring under the breadth and depth of the course material.

Flickr image source: http://tinyurl.com/qyuf6pa

In truth, the original implementation project was poorly scoped and was running over budget. It was clear that more time would be needed to complete the work. But the Administrator was becoming even more unsure about his ability to ultimately support his own company's users when the solution was finally deployed.

I scheduled an hour meeting with him after the training class (we actually talked for more than 90 minutes) and listened to his concerns. After some healthy venting, he ultimately agreed that none of his immediate doubts were tied to the platform itself. Instead we decided to revisit the original statement of work and work with Sales and Professional Services to properly expand the scope of the project to fit his organization's needs.

The impact

Looking back, I am not sure if my products are better off as a result of these interactions. At the end of the week, I had been engaged in three unique customer scenarios and delivered three different resolutions:

1. Defend the product, even if it meant losing revenue and risking our reputation
2. Admit our product mistakes and try to earn back the customer's trust
3. Reset a (new) customer's product expectations and help them find a successful path forward

Look for more reports from theProductPath around customer research, product validation, and voice of the customer here on PM Decisions.

More articles from our blog PM Decisions

Prep teams for upcoming trade show

Flickr image source: http://tinyurl.com/pynmbgf

Knowing that our most important industry trade show was barely a month away, I wanted to do my part to make sure we were adequately prepared and that our products were positioned to show well at the event.

Trade shows can be ideal opportunities for Product teams. If your company registers as a vendor, you can take advantage of your booth space and deploy an engaging team to help introduce new products and announce exciting new features to people cruising through the exhibition hall maze.

You may also gather lightweight customer data from the dozens of rapid conversations your booth team will have with passers-by. If you have already nailed your target market and are promoting the right product messages ahead of time, some may actually seek you out as they wander the aisles.

Even when you don’t have a booth and are just registered as an attendee, you can still take advantage of the fact that for a few days at least, a highly concentrated population of your potential customer base will be under one roof. With enough planning, you could still set up meetings with key prospects, industry analysts and partners. You should also carve out some cycles in your schedule to gather information to update your intel on the competition.

I wouldn’t exactly call us trade show veterans but over the years, we had steadily improved our collective efforts. This year, I wanted to raise the bar even higher, especially for my team.  

What drove this decision

This annual industry event precipitated our biggest marketing campaign of the year and we intended to pull out all the stops. We started setting up meeting schedules with customers and prospects weeks in advance and of course, were busy making preparations to lure in new contacts and leads. That meant crisp product demos that hit all the right notes. We needed to highlight the latest improvements to our story in a concise but captivating way to increase our chances of engagement.

Most of our close partners would also be in attendance and the show would provide us with an opportunity to reconnect with them as well, exchanging updates and looking for new integration or implementation options to serve our joint customers.

This was going to take some coordination - none of it was going to just happen magically. 

The decision: Kick off early planning for the show to synchronize all the product-related activities across departments.

We needed to highlight the latest improvements to our story in a concise but captivating way to increase our chances of engagement.

I don’t think it’s wise to pin all your product/sales hopes on a single event but I have seen first hand how a large industry event like this can become a natural rallying point for the Product, Marketing, and Sales teams.

I had already coordinated our next major product release to line up with the event dates so that we would have exciting new announcements to share at the show. That was a good start.

To make sure we would get the most out of the show, I would need to circle back with other departments to work on making all the other pieces fall into place.

Plan of attack

With good preparation, we would likely have hundreds of opportunities to tell and sell our story to a mostly captive audience. I wanted to support our company's collective efforts and put our products directly in the spotlight. That meant having persuasive product demos, queuing up notable product announcements, promoting influential partnerships, and more.

I also had to make sure my own team would be taking full advantage of all the available resources at the show.

Review product demos with the Sales Team

I started by spending some time with our Sales Engineering team to review the product features that I thought should be highlighted to create compelling demos for the traffic we would see in both the booth and in our private meeting rooms.

We rely on Sales Engineers to help execute the technical part of the B2B sales process. Together with the Sales Reps, they will tailor and deliver product demonstrations based on a joint understanding of the customer's needs. At a busy trade show we don't often have the time to fully qualify leads and so we have to tweak our methods.

In recognizing that most attendees would not be able to spend more than a few minutes with us, we optimized the familiar customer story to have even more impact in a shortened discussion. 

Confirm launch plans for a new product

Flickr image source: http://tinyurl.com/nghc5jl

Our big theme for this year's show was around the launch of our new mobile application. My partner over in Product Marketing had started orchestrating a full communication plan which would culminate in a product unveiling at the upcoming event. Aside from being able to show off the product itself (see Demos above), I wanted to make sure all our messaging was on target.

We had also been seeking out opportunities to further promote the new product outside of our own booth. For example, we applied for and were awarded a special "innovator" designation in the vendor listings. We also secured a coveted time slot to present at one of the show's more prominent venues where I would have much more time and resources to advertise our newest offering.

Finalize partnership integration announcements

Through discussions with a few of our strategic partners, we had determined that the trade show would also be a great time to create some joint buzz. One integration partner, for example, was going to use the show to significantly step up their own presence and our company had a part to play in that upcoming marketing blitz.

With another partner, we focused on a more tactical story. Rather than each of us independently promoting our own products, we agreed to highlight product integrations and the success our customers had realized by using our respective public APIs. Of course, it always helps to have these customers come forward and attest to our claims.   

Synchronize agendas with the Product team

And finally, I sat down with my own team to review our strategy for consuming as much information as possible during the show. We walked through the complete list of formal sessions, scheduled workshops, and other informal gatherings to build agendas that would provide maximum coverage. It was tempting to completely fill up everyone's calendar but we also wanted to be able to accommodate spontaneous and unplanned events as well.

The impact

At the time of this writing, the trade show is still more than three weeks away. There will likely be many more details to work through before we all hop on airplanes and head out. I feel good about our preparation to date and will continue to push all the teams to band together to deliver a positive outcome for our company.

Look for more reports from theProductPath around managing product teams and getting out of the building here on PM Decisions.

More articles from our blog PM Decisions

Create an in-person event to engage your customers

Image source: https://www.flickr.com/photos/highwaysagency/5997001123/

In realizing I had not spoken directly with our customers in some time, I decided it was time to get out of the building.

With a major software release right around the corner and being short a Product Manager due to a recent exit, the team was justifiably busy. There was plenty to do with making sure the upcoming deployment went smoothly while also firming up our plans for the very next release. And then there were the myriad other tasks that were on our plate, making it easy to rationalize that there simply was no time in the schedule to step away to talk to our customers.

But let's be honest here. By not making the time to engage your customers, all your other product plans are likely to fail.

What drove this decision

I was starting to get that gnaw in my stomach, the one that comes from making too many uninformed decisions. Because, no matter how confident you may be in setting the direction of your product(s), there is nothing more validating than getting direct feedback from your customer base.

Our Product team had been fairly diligent about reaching out to our users on a regular basis but many of those conversations are virtual and there is often a communication gap with phone calls and web-based meetings. I was eager to get some good face-to-face interaction with our target customers, to hear what they liked about our software, to learn what we could do to help them accomplish more, and to share where we were headed with our new roadmap. 

Image source: https://www.flickr.com/photos/mdd/9890924226

The decision: Leave the building (and the city) and spend some quality face time with a group of customers representing a good cross section of our users. 

As hesitant as I was about disappearing from the office for a few days, I knew the in-person discussions would more than compensate. And the plan for this particular trip was to spend a half-day with more than just 1 or 2 customers, so I knew I would return with a good assortment of responses to share with the team.

Plan of attack

Our Customer Success team is in the (good) habit of reaching out to new and existing customers in order to follow up with ongoing implementation initiatives and also to promote the Art of the Possible. We decided to make this event a twofer, extending their standard agenda with a short, product-specific program.

Build Simple Customer Profiles

Before "leaving the building", the first order of business was to gather some intel about the attendees with whom I would be meeting. Together with our Sales and Professional Services teams, I put together some brief customer profiles, paying specific attention to their specific use case and, as a crude gauge of their comfort level with the product, how long they had been users of our system.

This gave me some good, initial context for the level of discussion I would likely be having and helped me refine the scope of the topics I could cover. 

Create an Engaging Presentation

I know that very few people look forward to watching a PowerPoint presentation but, when done well, a good slide deck can be an ideal way to communicate ideas with a large audience.

For this event, I pulled together a series of slides that I thought would keep their attention and, more specifically, prompt some honest discussion among the group. For example, I included in the deck:

• A high-level, vendor's perspective of their business problem, showing how the company views their particular customer scenarios and intended to trigger dialog around exceptions and edge cases;
• Annotated screenshots of recently-introduced product features that they may not have had a chance to play with but would take too much time to demo completely, meant to spark conversation around additional exploration and use of the product;
• And a 1-page Product Roadmap that communicated, in broad strokes and without specific dates, the priority of particular product initiatives related to their use cases, designed to stimulate feedback and confirm we were on the right track. 

Days before the event, I reviewed the material with my team, incorporating some great, internal feedback while also rehearsing the presentation flow.

Demo, Survey, Preview, Recruit

There was a lot to squeeze into the time slot we had carved out for my Product talk. I wanted to promote the current state of the product and explain how and where we were looking to make enhancements. I had brought along some early, high fidelity prototypes in case we hit on a particularly popular or sensitive topic. I also was hoping to conduct a brief survey to get feedback on a number of our product roadmap intentions. And finally, I had hoped to recruit some of the customers into our more formal "labs" program so we could follow up with more intense analysis.

Other than a few weather-related, last minute cancellations, the meeting went off without a hitch. The participants seemed pleased with having direct access to the Product organization and appreciated the brief glimpse into the company's product strategy. Many of them have since enrolled in the more structured labs program and have indicated a strong preference to stay engaged with us.

Share Results With the Product Team

I returned to the office the next week eager to review the notes with the Product team. And while it seems there is never sufficient time to fully digest results of customer interviews and surveys, I was able to share a synopsis of the event and fold much of the feedback into our (still crude) research database.  

The impact

The good news is that the majority of the feedback validated our current product plans, at least for the next few major releases. It is certainly helpful to hear from your own customers that the product is indeed solving their needs. There were some familiar "hot spots" and valid complaints from the group and this has led to some re-prioritization work on our side that might not have been prompted by an interview with a single customer.

I strongly suspect there will be more such events in our future and that, as we continue to strengthen our outreach efforts, we will find it increasingly easier to get out of the building.   

Look for more reports from theProductPath around customer research, product validation, and voice of the customer here on PM Decisions.

More articles from our blog PM Decisions