Build the prototype to aid in new product discovery
In recognizing that we were finally in a position to start gathering real requirements from a prime new customer, I decided to move forward with building a prototype for a new product offering.
The Product Decision: Complete a working prototype that we could use to drive productive conversations with customers around requirements.
Flickr image source: http://tinyurl.com/pfyguao
In recognizing that we were finally in a position to start gathering real requirements from a prime new customer, I decided to move forward with building a prototype for a new product offering.
The Product team had known for some time that we had a legitimate gap in our product portfolio around an MS Office 365 Plug-In but we were not overly anxious as it was clear that the gap was not preventing the company from winning new deals. Even though our main competitor would regularly raise the issue with prospects during the sales cycle, drawing attention to our "deficiency", it had never cost us a sale.
The MS Office 365 Plug-In saga continues to unfold...
You can review the entire story through my past Product Decisions:
HEAD OFF A NEW PRODUCT IDEA BEFORE IT GAINS ANY MORE STEAM
CLARIFY WIN-LOSS SPECULATION WITH POST-MORTEM INTERVIEWS
BUILD A QUICK PROOF OF CONCEPT
From the beginning, when this "urgent need" was first identified, I had been steadily pushing back on internal stakeholders (see sidebar). I knew we didn't have the available resources or bandwidth to tackle this and without a line of customers outside my "product door" clamouring to have the new feature, I was hesitant to even start conversations around the Plug-In.
What drove this decision
In the previous quarter, we had signed our biggest customer in the company's history and, in our lengthy discussions with them around their needs, we had identified the (missing) Plug-In as a key, must-have feature.
This one customer, when fully deployed, could have more than 2,000 people using the new Plug-In. Based on those numbers, it seemed reasonable to me that we now had a decent base of users from which we could start to gather some useful insights.
The decision: Complete a working prototype that we could use to drive productive conversations with customers around requirements
I was eager to start talking with and observing real users in the field but I didn't want to do that using the low fidelity and crudely fashioned proof-of-concept we had completed a few months back. It was time to build and start testing with a real, functioning prototype.
Plan of attack
We were still very early in the product cycle for this Plug-In and there was no pressure to cut corners or skip steps in our product discovery process. I wanted to make sure we used a disciplined approach.
In this first phase, we would create a prototype that would give our Tech team more confidence in what it would ultimately take to develop and support such an app AND that would give the Product Team enough material to have meaningful discussions with the first round of users.
Vet the feasibility of our technical approach with Engineering
Flickr image source: http://tinyurl.com/p65wgqb
I had some solid ideas for how to create an initial version of the Plug-In though it was fair to say that we would be forging into unfamiliar territory. The Tech team had never developed this kind of app but we knew we would be getting support from our friends at Microsoft who is one of our key technology partners.
The first step in building the prototype was making sure our team could nail down some of the technology challenges that we would face. For example, high on our list were questions about user authentication, API access, and deployment of the app to customers through Microsoft's "online store".
So, over the course of a few weeks, the team completed a functioning prototype that addressed these areas of concern. Now, more confident that we had cleared the most immediate technical hurdles, I proceeded with getting some internal validation.
Share the prototype with stakeholders inside the company
I very much wanted to show the prototype to our Sales Engineers. They had always been good product collaborators and certainly understood the broad use case for the Plug-In. I set up a meeting with them and the Product team and with the Engineers to get and record some early usability feedback.
On the way to that meeting, I made a small detour to give a brief demonstration to our CEO as a courtesy “first look". He had taken a keen interest in this initiative and had been stalking me for many weeks. And even though we had only addressed the most simple use case in this iteration, he was very impressed with our rapid progress. My only regret was letting an Engineer run the demo - who uses superheroes for their sample data? Ha!
I should have known not to let an Engineer run the demo - who uses superheroes for their sample data?
We then sat with the Sales Engineers and reviewed our current hypotheses with them. They see much more front-line action with prospects in the sales cycle and helped validate our approach, often tying back to their deals from the past weeks and months.
I then had each of them install the Plug-In on their own machines to test it out. The response was overwhelmingly positive and we could see them already thinking about how they might adjust their standard demo scenarios to incorporate this new component.
One of them suggested they start using the Plug-In immediately to help them with one of their own team activities and I offered to check back in with them in a few weeks to see how it worked for that additional use case.
Show it to the technology vendor on whose platform our prototype was built
We had invited representatives from our partner Microsoft into our office for a high-level meeting to talk about Office 365 integration among other things. The timing of their visit provided a perfect opportunity for a prototype demo and to show what we had achieved in the past few weeks, with some help from their own technical folks.
After setting up the customer scenario with the group, we walked them through the working prototype and talked through some of the challenges we had worked through to get to this point. The Microsoft team provided additional validation around our approach for the Plug-In and gave us some pointers for how we could move forward. Our remaining questions were captured and would be carried back to their teams to help get us the answers we would need before rolling this out, even to beta customers.
In the end, I was pleased to know that our partner was ready to help us with next steps. They also expressed interest in the upcoming deployment with our new customer as it would provide a great case study for both us and them.
The impact
With the Engineering team, I was able to work through a few of the big technical unknowns to build a working version of the Plug-In. By meeting with and showing off the Plug-In to a few internal resources, we were able to get some early, but solid validation. And in reviewing the finished work with the vendor on whose platform we had built the Plug-In, we confirmed that we were on the right track in how we were going to deploy and support the new product.
But none of this would be worth anything if we didn’t immediately start validating the Plug-In with actual users. That work would likely start the very next week and is lining up to be the next chapter in this series.
Look for more reports from theProductPath around product validation, feasibility, and product investments here on PM Decisions.
More articles from our blog PM Decisions
Conduct high-priority product research
To address one of the most critical problem areas for customers using our software platform, I decided to roll up my sleeves and conduct the fieldwork myself.
The Product Decision: Enlist the help of our User Research expert to coordinate formal interviews with a suitable group of administrators and end users from our active customer base.
Flickr image source: http://tinyurl.com/pmvvurb
To address one of the most critical problem areas for customers using our software platform, I decided to roll up my sleeves and conduct the fieldwork myself.
For many months, we had been recording feedback from our customers, some direct, some indirect around one particular area of the product. Everyone agreed that we had delivered really powerful features - the common challenge was in the setup and configuration. We had made it too complex to use this valuable piece of the product!
I couldn’t agree more. I knew how hard it was, even for more technical users to configure the tools. I have a deep programming background but even I would struggle trying to ramp up on proprietary schemas with cryptic configuration syntax and poor documentation.
We had been attempting to compensate and even head off frustrated customers by offering to have our own Professional Services team help get customers up and running. But that only served to delay the inevitable as the complaints would then change to “even if you build it for me the first time, I’m not confident I can take it over and maintain it!"
What drove this decision
I had been anticipating a suitable window of time where we could focus on revisiting and rethinking these problems. The teams had been making good progress on some of the pre-requisite components that would be leveraged to rebuild this key feature of our platform. It was time to initiate a round of interviews with customers about how they use the current product, what (if anything) they liked about it and what (again, if anything) they could point to that causes problems for them or their end users.
I wanted to take full advantage of this opportunity to get out of the building, to speak directly to users and reinforce this important research activity with the members of my Product team. Mostly though, I wanted to improve our product and make our customers happy.
The decision: Enlist the help of our User Research expert to coordinate formal interviews with a suitable group of administrators and end users from our active customer base.
I am fortunate to have a talented resource on our team who has raised the bar around user research for our entire company. She continues to guide us all on the finer points of interviewing users early on around product discovery and later on for product usefulness and usability. With her by my side, I felt very confident that we would get to the crux of the problems and ultimately, be able to prioritize the work to come.
While I was eager to get started and optimistic about what we would learn, I heeded my colleague’s advice and adhered to the plan we made to ensure a favorable outcome.
Plan of attack
It was so tempting to just start writing requirements. But I've been wrong before.
I have used our product internally and I have also demoed it hundreds of times to customers so I felt confident that I knew what was wrong and what would need to be fixed. It was so tempting to just start writing requirements.
But I’ve been wrong before. Let me say that again, I’ve been wrong before. I have felt strongly about how our product should work and went straight to the Engineers with requirements only to find out, after they had spent valuable time implementing the stories that there were no customers lined up to use the new enhancements.
Start with hypotheses and a good prototype
Because I was so familiar with this particular part of our platform, I was able to quickly compose a lengthy list of hypotheses of pain points where I would expect to hear complaints from both heavy and casual users:
- We believe Admins are struggling to initially configure a URL to set up a new instance of [tool] in environments like Salesforce.com.
- We believe end users are frustrated with the number of clicks required to complete the ... process, especially if only one document is necessary.
- We believe Admins are resisting adopting and expanding their use of [tool] because it lacks good error handling, is difficult to test, and has too few options for controlling its behavior for the end user.
- We believe end users struggle with scrolling and otherwise navigating through the different areas of the page, e.g. with long lists of templates or with very long data forms and may find it helpful to track incremental progress as they work through sections of a longer form.
- We believe end users are underwhelmed with the stark layout of the [tool] page, the lack of branding, and the lack of any help text.
- We believe end users struggle with using the [tool] with different browser window sizes, specifically when the buttons move around when the page is resized.
- ...
I then had the UX team mock up a clickable prototype that addressed some of the primary concerns and that would allow us to better engage with our interviewees.
Recruit and track participants
To build a list of participants, I first reviewed the customers who were already part of our formal “labs” program (always be recruiting!). Then, I put the word out to our Sales and Professional Services teams to identify other viable customers with whom we could talk. The resulting list was manageable enough for us to track using a shared spreadsheet that included contact information, interview dates, and miscellaneous notes.
Flickr image source: http://tinyurl.com/nomqgdl
| Contact Names | Organization | Preferred Phone/Email | Participation | Last Contact Date | Future Participation | Primary Internal Contact | Notes |
|---|---|---|---|---|---|---|---|
| Nellie Bluth | The Bluth Company | nbluth@thebluths.com | Asked to be included in Fall research initiatives | 7/12/15 | Also willing to help with... | Maggie Lyes, Acct Mgr | Has been helpful in the past but admits to not being technical |
Conduct and record the interviews
When we had accumulated a sufficient list of suitable users, I crafted and sent out an email invite along these lines:
“Hello Vincenzo, I would like to enlist your help for a product research initiative.
I head up the Product team and am leading an active project to learn more about how customers are using our tools. I am reaching out to you because your group is currently using [tool]. I believe you would be able to provide some great feedback on what is and isn’t working well.
If you are interested in participating, the next steps are easy:
1 - Reply to this email with some available interview times over the next 2-3 weeks.
For example, “Tuesday mornings are better”, “tomorrow”, “Wed/Thurs between 3-5pm Pacific”.
2 - Prepare to walk us through how you or your end users use [tool] today
This is a great opportunity to tell us about your successes with [tool] as well as things that would make it easier for you. The conversation should not take more than 45 minutes.”
Not surprisingly, our more fervent users responded immediately. With the others, it sometimes took a little more effort to secure the interviews. We mostly used virtual web meetings to conduct and record the interviews where the customer did most of the driving while we watched.
Our agenda for the interviews followed this path:
- Show us how you use the product today [50% of the interview time was spent here]
- As an [administrator, end user], what would you say if anything, are your favorite things about [tool]? [10%]
- What, if anything causes you or your users problems when using [tool]? [20%]
- Switching gears, we'd like to show you our ideas for a new design... [20%]
I had an intern transcribe notes from the recordings, putting all the material in a central repository so we could index it properly and share it with the rest of Product and Engineering.
The impact
The customer interviews went smoothly giving me exactly what I had hoped to obtain. There was plenty of validation around our current shortcomings, some surprises about certain attributes that users found valuable, and an abundance of constructive griping. The early prototype seemed to be a hit with users and gave me additional confidence to pursue that path.
I expect to deliver a first round of product improvements in the very next release and will continue to engage these customers to validate our efforts. Look for more reports from theProductPath around customer research, product validation, and feature prioritization here on PM Decisions.
More articles from our blog PM Decisions
Unveil new product at company event
In recognizing the work that all the departmental teams had contributed to create this novel product, I decided to orchestrate a group presentation to roll out the new offering to the entire company.
The Product Decision: Preview the new product in a group setting where each department contributes to the larger story.
Flickr image source: http://tinyurl.com/obvs3hc
In recognizing the work that all the departmental teams had contributed to create this novel product, I decided to orchestrate a group presentation to roll out the new offering to the entire company.
I really enjoy our Release Previews. These are monthly get-togethers I created to have the Product team demonstrate, to the entire company, the latest, shippable product updates and enhancements. These meetings are intentionally distinct from the less formal Sprint Review, typically led by and targeted at Engineers.
That is not to say that Engineers are not part of the Release Preview - actually they are often featured as we are showing off their handiwork in most cases. But the broader messages we deliver to the company at the Release Preview are more value-driven, as in what value are these updates going to provide to our customers and to our own business?
We all know it takes a coordinated effort to build and release new products. During the Release Preview, I try to make sure we recognize everyone who contributes to our incremental progress and that goes well beyond Product and Engineering. But this was not our standard release.
This was the official launch of our new Reference App.
What drove this decision
This was a major achievement for our company and I wanted this particular Release Preview to stand out from the rest. After many months of planning, development, testing, and customer validation, we were ready to unveil a unique new offering that would differentiate our company - and it was important to me and the team that everyone in the company appreciate the collective accomplishment.
The decision: Preview the new product in a group setting where each department contributes to the larger story.
More important than getting everyone up to speed on and pumped up about the new product, I wanted them to all to understand and appreciate the work that all of us had done to make it a reality. We had shown prototypes of the product many months ago and despite small, periodic updates, many in the company had all but forgotten about it.
In the reminder email I sent to the company days before the event, I emphasized how this particular Release Preview was going to be an extravaganza. Now it was time to put together a presentation that lived up to the hype.
Plan of attack
The highlight of our Release Previews are most certainly the product demonstrations. But as I mentioned before, we are very prudent about wrapping those demonstrations with stories about the value that is ultimately delivered to the end customer. This presentation would be no different but the story would be larger and all-inclusive.
Recount How we gOt here
To start things off, I asked our head of Research to provide the back story. This app came out of one of the best customer research initiatives we had ever run and I couldn't think of a better way to set the stage. Straight from our customers' mouths, we heard about the struggles they had been having with our platform. Over and over, we were told the same thing - you have to give us more visibility into what's going on.
We listened, we prototyped and we validated. The reason I felt so confident in this presentation - and in this new product - was because it was practically dictated to us by our end users.
Get representatives from each department to contribute their parts of the story
Then, one by one, I had representatives from each department stand up and speak to other facets of the product.
Professional Services described the lengths they would go to just to provide a makeshift remedy to the customer's visibility problem.
Engineering and Ops took turns talking about the speed at which we would be able to deliver updates to the product and how much our entire deployment had improved as a result of launching this app.
UX gushed about how the new app leveraged a recently developed component library and how it help to launch a new UI paradigm for the company.
Sales and Marketing weighed in with the overwhelmingly positive response they were getting from the early previews we had been sharing with prospects and customers.
And so on. By assigning each of them a dedicated speaking role and by giving all the contributions equal weighting, I was able to relate a shared victory for everyone.
GET THE CEO TO WRAP IT ALL UP WITH A ROUSING FINISH
I would have been happy enough with just the departmental contributions, but to end on an even higher note, I asked our CEO to supply his own thoughts on our newest product. In a brief but stirring, unrehearsed speech, he shared his appreciation for all our hard work and for the company-wide commitment to product innovation.
It was well-received by all.
The impact
To say the Release Preview went well would be an understatement. We had the largest employee turnout ever for an event like this and I received a great deal of positive feedback afterwards. One of the recurring messages was that we might want to reserve additional, spillover conference rooms in the future to accommodate more of our folks in the office!
I thanked all the speakers individually for their participation and gently reminded them that we still had more work to do to make the official launch a success. At the end of the day, with all the solidarity we generated, I couldn't help but feel that success was, if not inevitable, then certainly more attainable.
Look for more reports from theProductPath around product reviews, releasing products, and product culture here on PM Decisions.
More articles from our blog PM Decisions
Make Product Decisions Like Cagan
In what many Product Managers might consider an average week, I made three classic product decisions that would make Marty Cagan smile.
The Three Product Decisions: Determine what was most valuable for our customers, what was most usable by our end users, and what was most feasible according to our Engineering team.
Image sources: https://www.flickr.com/photos/68751915@N05/6848823919, https://www.flickr.com/photos/nataliedowne/6446884983, https://www.flickr.com/photos/norue/9625271288
In what many Product Managers might consider an average week, I made three classic product decisions that would make Marty Cagan smile.
After spending more than 10 years in the Product Manager role, I can't actually recall an average week. Perhaps we are attracted to the role for that reason - because it is unpredictable. If it's not changing requirements at one end, then it might be new technology "insights" at the other. The cast of characters we will work with week to week is variable too as new customer prospects enter the picture and compete for attention with existing customers that have their own unending wish lists.
To be successful, Product Managers must be able to navigate these challenges week to week. Good mental frameworks are essential to making sound decisions and to maintaining credibility over time. When you are feeling overwhelmed, however, you may find it helpful to confront the tough product decisions with this deceptively simple approach.
Inspired - How to Create Products Customers Love by Marty Cagan
What drove these decisions
Most Product Managers have run across Marty Cagan’s book Inspired - How to Create Products Customers Love. I consider it a must read for anyone in the general vicinity of Product Management.
Among the many gems in this book, Cagan identifies three fundamentals ways to validate your product decisions: determine what's valuable for your customers, what is usable by your users (not always the same personas as the actual buyer), and what is feasible given the resources you have at your disposal.
These are not mutually exclusive. Whether you're early in your product discovery or tackling a thorny issue for customer 10,001, you need to land on a resolution that best balances these three forces in order to move forward.
Over time, with practice, this starts to become instinctual and woven into your daily PM routine. You may even be surprised, as I was, in reflecting back on an "average" week that there is a very valid reason for why many of your product decisions had turned out so well.
The Three Product Decisions: Determine what was most valuable for our customers, what was most usable by our end users, and what was most feasible according to our Engineering team.
In this particular week, I made and can highlight three, separate decisions for our product and ultimately, for the future success of our company. What makes these particular decisions stand out from the dozens I made is that each was particularly punctuated by one of Cagan's three principles.
Plan of attack
Decision 1 - What’s MOST Valuable
The team had been exploring how we could redesign an existing product feature and was driving that discussion using input we had gathered from rounds of customer testing. As we unwound the different pieces of the story, we revealed the true underlying complexity and ultimately determined that it was too big to tackle all at once.
I decided to break up the larger story into 3 smaller, discrete pieces and sequence the resulting work to deliver incremental improvements over time - a common problem solving approach. But this decision made it much easier for the team to single out which piece would ultimately provide the most value to our customers and, all other dependencies aside, which we would schedule first.
Decision 2 - What’s MOST Usable
In a separate thread, the team was reviewing feedback we received after showing a proposed new design to some of our end users. We had been planning to introduce a completely new design to address some known usability problems with a particularly popular feature.
Few things are more influential for a Product Manager than hearing end users describe how they use, or would use your product to do their job. It was precisely this feedback, straight from our users' mouths, that prompted me to scale back the new design. It turns out that we had been overthinking the problem and could actually deliver a more usable solution with fewer changes (i.e. less work on our end).
Decision 3 - What’s MOST Feasible
Software Engineers like to explore new technology. I know this from having spent the first 10 years of my career as an enterprise software developer. In this current company, I had been having conversations with the Engineering team about how we would tackle a large, scale-related challenge deep in the bowels of the platform (not customer facing). The team had fully researched and had achieved some early success with an alternative technology that seemed applicable for this particular problem.
So, in this very same week, we had come to a point where I needed to make the call on whether to schedule the scale-related work for an upcoming release. And while the Engineering team was convinced the technology would ultimately solve our challenge, we were not jointly confident in our collective ability to proceed safely. Because of this uncertainty around technical feasibility, I swapped out the work from the release, postponing it for a later date after more research and experimentation could be done.
The impact
As I mentioned at the start, these types of decisions are commonplace in a Product Manager's average week: problem decomposition to help reveal customer value, user testing to help reveal usability concerns, and ongoing engineering R&D to help reveal feasibility limitations. Still, it never hurts to return to some of the core product principles to remind us all why we are here. And thank you Marty Cagan for all your contributions, and for inspiring me.
Look for more reports from theProductPath around Product Feasibility, Usability, and Value here on PM Decisions.
More articles from our blog PM Decisions
To address one of the most critical problem areas for customers using our software platform, I decided to roll up my sleeves and conduct the fieldwork myself.
The Product Decision: Enlist the help of our User Research expert to coordinate formal interviews with a suitable group of administrators and end users from our active customer base.