Product Managing (Somewhat) Intelligent Products That Work With Humans
Product Managing (Somewhat) Intelligent Products That Work With Humans


Building data science and machine learning into a product does not mean automating all human work away. Instead of aiming for a flawless artificial intelligence, we can accept that the technology is only somewhat intelligent and make it work together with human domain experts to fill in the gaps in the system’s capabilities and train it to improve.
This combination allows to to build full stack vertical applications in a particular domain.
We use as an example a system that learns from expert users how to answer frequently asked questions sent to customer support automatically. Dealing with natural language and question answering provides a good illustration of challenges of defining which parts should be done by machines and which are best left to humans.
We describe the algorithms and the the data pipelines, but go deeper into the considerations beyond algorithms that are required to make the product successful:
* How to choose evaluation metrics that capture the business objectives
* How to make the users feel good about interacting with the system through managing perceptions and expectations
* How to anticipate the AI to make mistakes and design the system to minimize the impact of these errors on user experience
* How to plan the handoff between the machine and the human workers
* How to build an integrated workflow where work is divided between the machine and the human workers
* How to align the interests of the human workers with the machine
* How to design the machine/human interface to maximize clear signals for machine learning
* How the probabilistic part of the system works with the deterministic one

This session should be of interest to engineers, product managers and designers who want to know more about building machine learning into their products."


Eugene leads data science at Directly, a startup in San Francisco that helps companies scale excellent customer service by letting expert users help other users on demand. He builds augmented intelligence systems that allow humans and machines work together to make customer support better. Previously Eugene was creating data stories and data products in the fitness wearables space at Jawbone. Prior to Jawbone, he was CTO/Co-founder of survey startup Qualaroo.

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google