Booking.com is are running more than a 1000 concurrent experiments on the guest facing website. Less known is that Booking.com is also using large scale experimentation to improve the property owner side of the experience. This brings a set a different challenges and complexities. So what are the 5 unique challenges and advantages when running experiments behind a login? In this session you learn about:

– Creating new metrics for experimentation
– The tale of two hypothesis
– Using soft-stops 
– How to use non-inferiority experiments to make sure you are not impacting 
– What about the user? If you login everyday, do you not see all changes?