Finding product market fit with hypothesis driven development
Envato—one of the most successful technology startups in Australia—run eight digital marketplaces aimed at creatives such as designers and developers. These marketplaces cover everything from videos, photos and audio assets to website themes and code plugins. In fact, everything needed to complete a creative project.
One marketplace focused on the massive stock photography market. PhotoDune offered low cost royalty-free photography created by Envato’s community of photographers (authors). The stock photography market was the largest that Envato operated in. Even a small growth in the market share would result in a significant increase in revenue and author income.
PhotoDune – Envato’s existing stock photography library
At the time a number of new contenders had entered the market with a new disruptive model. Libraries like UnSplash offered high quality high resolution photography for free. Envato needed to disrupt themselves before competition like UnSplash disrupted them. A challenge was set to re-invent and differentiate Envato’s stock photography product in the market and find a way to compete against these new models.
Workshopping a new model
Together with the CEO, product and strategy teams, the design team helped shape a new proposition. We looked deep into the problem analysing what we knew about the competition and the gap between the best in the market and Envato. We looked at the revenue per photo per month averaged across all photos against the same measure for the best authors. We looked closely at the new competitors such as UnSplash. Identifying what made them different and also where opportunities existed. For example because UnSplash offered photos for free they were not unique, with the same photos appearing all over the internet on many websites and blogs.
UnSplash – A new disruptive model
Pain points within the existing stock photography product were also explored. Identifying areas such as the number of choices that were offered for image size and licence options as an area of confusion for the user. Also the size of preview images were very small compared to other stock photography libraries.
A hypothesis was created:
We believe that offering a higher quality photo library (better, curated photography presented as a premium and exclusive collection) would result in an increase in earnings for Envato authors. We know this will be true when revenue per photo per month is higher than our average for the best authors on PhotoDune.
There was also a stretch goal to beat market leader shutterstock’s revenue per photo per month.
shutterstock – The market leader
A hypothesis driven experiment
To test the hypothesis an experiment would be created with just enough of a new product. A premium collection of photographs would be curated from both the current collection and by inviting new submissions from authors. A new brand would be created to differentiate it from the existing photo library and allow it to be tested as a stand alone product. Price and licence options would be limited to make the purchase decision much simpler.
To create just enough of a product to test the hypothesis and reduce the time and cost of running the experiment a number of decision were made. The new product would include the minimal browsing experience. Search was out, there were simply not enough photos in the collection so search was more likely to lead to no results. Checkout was also avoided. Envato had a solid checkout experience with all the data to give standard drop out rates and implementation estimates. To avoid checkout, the experiment would simply take the user into PhotoDune to complete payment. Not an ideal experience, but enough to give the team the answers needed.
A number of early concepts were created to explore different options. Such as big verses small images and different approaches to grid layouts. The idea of curated collections within the bigger collection was also tested to make it easier to find photos in the absence of search.
Developing the brand
A new brand—UnStock—was created as a play on the fact it was disrupting the stock photography market and taking on the new comers. A number of logo options were iterated on before reaching the right option. The logo had to balance the new premium look and feel with the association of the master Envato brand. It also need to make it clear that Envato was not the author of the photos as it was community driven (a key differentiator in all Envato products). The chosen logo represented the curated collection of photography as framed by Envato.
Iterating on the UnStock logo
The initial concepts were narrowed down to what was believed to be the winner. The design was prototyped in code using an API to bring in real content to test the experience. The results of user testing were very disappointing.
Although users loved the large image previews the main problem observed was the browsing experience. A user could only browse one image at a time. To change the image in the collection the user had to scroll down and then back up to preview the new image. This up and down experience felt very unnatural and limited the chance that the user would discover the photo they wanted.
Failure during user testing due to up and down scrolling
The final experiment
From the user testing on the initial version the team realised that the size of the previews had to be balanced with the chance of discovery. A grid layout was introduced from revisiting some of the earlier designs. The final design was iterated in code (to ensure the experience was tested with real content) to reach the end result. The final product was shipped and then promoted to test the hypothesis.
In early results the data collected showed 16x the target metric. As promotion reduced this levelled out at approximately 4x the target. The new stock photography library was a massive success. The team had identified a new product and discovered market fit in just a few weeks. A lot had been learned about the browsing experience (from observing users browse for photography). However, the experiment was very small in comparison to the main stock photography library. The challenge would be to scale up the product and break even.
Fully responsive to all device sizes
The real challenge
The biggest learning from the experiment was not the results observed or the product created, it was the process itself. Aside from learning how to form cross-functional teams from within the wider product development team and run experiments like this, the team had discovered the limitations of the approach.
The real challenge was ‘what next’. The experiment needed a follow up experiment or a pipeline of experiments to follow. A lot was learned from running a single experiment but a series of experiments would be needed to create a new product in this way. Because a series of experiments had not been planned the initial experiment had tried too many ideas at the same time. This made the results less clear. Running a series of experiments was more difficult. Prioritising experiments—who’s outcome and definition of done is largely unknown beforehand—against other well defined and scoped initiatives in the roadmap was the real challenge ahead for the team.