post

Footer About

Footer Get Involved

Footer Links

Mind the Product

General

Product Unplugged Episode 3 with Pierce Healy

Product Unplugged Episode 2 with Lorilyn McCue

Product Strategy

Building Product Teams

Diversity and Inclusion

Digital Transformation

Artificial Intelligence (AI) & Machine Learning (ML)

Product Leadership

Product Management Career

Product Ethics

Minimum Viable Product

Enterprise Product Management

OKRs

Customer Research

Hypothesis and Testing

Business Strategy

User Research

👩‍🚀🚀   Live now for MTP members: <a href="https://mtp2017stg.wpengine.com/membership/#events" style="color:#FFFFFF;text-decoration:underline;">Driving Innovation AMA with Tendayi Viki</a>

Roadmaps: Break the rules without breaking the principles

Got questions about your product career?

Product roadmaps vs data – Jeremy Levy on The Product Experience

From engineering to product management at Google

Mind the Product joins Pendo

George Fletcher is a Senior Product Manager that has been working in product for over a decade.  A passionate product manager promoting lean product management, rapid prototyping and the adoption of design sprints throughout the businesses he's worked for.  Having built successful products in a variety of different industries George has found a home in the acceleration and incubation hubs in larger corporate environments helping transform mindsets and processes towards lean, digital thinking.

George Fletcher

<p>In this ProductTank London talk, Ben Fields, formerly Lead Data Scientist at FutureLearn, looks at getting the most from the relationship between product managers and data people, working out people&#8217;s background understanding, the benefits of leading with a story, and how best to spread knowledge.</p>
<p>There are three key aspects of how data science can effectively sit within a product and software organisation:</p>
<ol>
<li>Gear and orient to your audience</li>
<li>Optimise for diversity</li>
<li>Embed with product</li>
</ol>
<h2>1. Gear and Orient to Your Audience</h2>
<p>How can you show your audience content that they would not otherwise have found? Whether a site is selling ecommerce products or promoting editorial content, delivering an accurate relationship between what the user is presently viewing and then where you direct them next is more likely to lead to conversion.</p>
<h4>A Story About Popularity Bias</h4>
<p>Ben says that a recommender system and widget built at FutureLearn attempted to orientate the user to the course they were viewing and then gear and direct them to other possible, related options &#8211; options that they may not have encountered without this widget.</p>
<p>A recommender system should make effective use of data that an application has available. However, this can push a model to decisions that might not be the best for the viewing user. Popularity bias can creep into data models.</p>
<p>The FutureLearn recommender system would promote courses based upon similar enrolment patterns and behaviour from the wider user base.  o, at its simplest if Person A enrolled in Course 1 and Course 2, then when Person B views Course 1 they may also be recommended Course 2.</p>
<p>However, FutureLearn found that its recommender system kept promoting a similar set of courses that didn&#8217;t necessarily have a fair relation to the course being viewed. This was because the system took popularity into account and one provider, the British Council, provided a number of courses that were popular among FutureLearn users. As a result, a lot of British Council courses appeared in the widget. This meant that the widget was not meeting the aim of showing courses that users would not have found otherwise.</p>
<blockquote><p>If we&#8217;re just repeating what we already saw then what work are we doing?</p></blockquote>
<p>Ben has another story about popularity bias. A popular book retailer had with a “people who bought this also bought…” widget. When browsing the product page for a Java RMI (remote method invocation) book the widget recommended two other software engineering related books and then finally &#8230; Harry Potter.</p>
<p>It this wasn&#8217;t incorrect, it just isn’t what was wanted and likely had little value. The system picked out Harry Potter because “everyone” has bought Harry Potter.</p>
<p>It is very difficult to move from popular courses, books, content, artists, products, deep into the long-tail, less popular and less visible items. However, that longer tail can hold a lot of dormant value which can be unlocked if it is seen by users at relevant opportunities.</p>
<h2>2. Optimise for Diversity</h2>
<p>One way to combat popularity bias is to optimise for diversity. When designing a complicated system such as a behavioural recommender, it is key to spend time working with the people who will find the edges of the requirements.</p>
<h4>Telling and Supporting Your Stories With Data</h4>
<p>FutureLearn wanted to build a prediction engine that would predict course enrolment and revenue. The system would take the known information about the course due to run and model it against behaviour seen on the site to get enrolment and revenue figures. Start date, course length, how many times has it been run, how much learning there is, and course density, are some of those elements that can be coupled with other factors such as seasonality, peaks, and troughs.</p>
<p>Using a machine learning tool called TPOT, the team at FutureLearn was able to cut short system setup and prove it worked.  It took a week to show that the system was able to predict revenue when given a set of data attributes.</p>
<p>This lean approach gave FutureLearn an immediate output and evidence base, supported by data. While not pristine, the system gave better predictions about which courses might be better than others than the previous rule-based, business process methods.</p>
<h2>3. Embed With Product</h2>
<p>Finally, the FutureLearn team found success by keeping their data scientists close to product. The product team outnumbered the data scientists in the business but this was offset by using an embedded rotation system. Data scientists moved around and embedded into the various product teams based upon priority and focus.</p>
<h4>Priority</h4>
<p>Here the data team would consider need and ask when/where are the data scientists going to be most effective?  Which product team needs them, when and for what scale of project?</p>
<h4>Focus</h4>
<p>The battle here is to ensure that the decision aligns with the strategic aims of the company. By serving and orienting to the business focus the team could better put the resources where they could be most effective.</p>
<p>Ben’s final parting thought about scarce resources was that effective internal communication and an understanding of the value that data science will allow companies to be far more effective.</p>
<div class="g g-21"><div class="g-single a-77"><a class="gofollow" data-track="NzcsMjEsMSw2MA==" href="https://mtp2017stg.wpengine.com/product-management-training/public-workshops/"><img decoding="async" src="https://mtp2017stg.wpengine.com/wp-content/uploads/2021/10/General-Training-Next-Level-FOOTER-scaled.jpg" /></a></div></div>

core/freeform

Solving Ethical AI Problems, When Algorithms Go Wrong, and the Skills Needs To Work on AI Solutions: Insights From Kriti Sharma

How AI Is Changing The Product Management Job Description by Mayukh Bhaowal

Designing Meaningful Human Experiences by Kate O’Neill

Can AI Create a Fairer World? By Kriti Sharma

They’re Just Fancy Averages by Ben Fields

Data Science and Digital Businesses by Ali Gulez

Talking About Data Science by Jenessa Lancaster

AI at an Early Stage Startup by Zara Tam

Building Search Products With Machine Learning – Shakhina Pulatova

Using Deep Learning to Find Beautiful Photos by Alex Miller

ProductTank Columbus

ProductTank Tauranga

ProductTank Edinburgh

Prioritised Member AMA - Perfecting product roadmaps

ProductTank San Diego

Navigating difficult relationships panel

ProductTank Philadelphia

ProductTank Bucharest

ProductTank NoVa

Remote Training Workshops

ProductTank London

ProductTank Birmingham

ProductTank Oslo

ProductTank Astana

ProductTank NoVA

ProductTank Belgrade

ProductTank Madrid

ProductTank Leeds

ProductTank Zürich

ProductTank Perth

ProductTank Tokyo

ProductTank Chicago

ProductTank Auckland

ProductTank Richmond

ProductTank Charlotte

ProductTank Reston

ProductTank Washington

ProductTank Sofia

ProductTank Raleigh

ProductTank Ljubljana

ProductTank Kraków

ProductTank Athens

ProductTank Barcelona

ProductTank Dublin

ProductTank Montréal

ProductTank Bogotá

ProductTank Seattle

ProductTank Madison

ProductTank Singapore

ProductTank Portland

ProductTank Budapest

ProductTank Valencia

ProductTank Porto

ProductTank Los Angeles

ProductTank Austin

ProductTank Norwich

ProductTank Des Moines

ProductTank San Francisco

ProductTank Helsinki

ProductTank Reading

ProductTank Brighton

ProductTank Bielefeld

ProductTank Malmö

ProductTank Canberra

ProductTank Paris

ProductTank Manchester

#mtpcon SF+Americas 2022 Conference

#mtpEngage Hamburg 2022 Leadership Forum

#mtpEngage Hamburg 2022 Workshops

#mtpEngage Hamburg 2022 Conference

#mtpcon London+EMEA 2022 Conference