Build, measure, learn for the win

One of the first projects I worked on at Automattic was a small A/B test for the first screen of the WordPress.com sign-up process. As the first screen in our funnel, it served a critical role, but it hadn’t been tested in quite some time. I tweaked the visual design and completely changed the copy for my first attempt at improving conversions on that screen. Setting it up and tracking the results felt really easy compared anything I had done before. I felt empowered and simultaneously discovered how projects work at Automattic.

That first test was a success and was implemented as the default for all our visitors. It remained that way for a little under a year as our priorities shifted to higher impact areas of our funnel. Little bits of feedback trickled in over time that kept me thinking about many possible improvements but it wasn’t until recently that I decided to take action.

The problem

One of the most common complaints I heard about our screen was that the terminology was confusing. We asked people what type of site they wanted and presented them with four options: a blog, a website, a portfolio, and an online store. These were some of the questions that kept coming up: what’s the difference between a website and a blog? can I change my mind later? two of these options sound good, what if I want both?

Our mistake was assuming people understood what to us seemed like simple concepts but for them were actually foreign. This made a lot of sense when I look back at all the polls I ran and conversations I had with our customers over the last year. I learned that our customers are not like us and that the majority of people signing up at WordPress.com had little to no experience building sites or using WordPress.

screen-shot-2018-01-02-at-10-38-08-am.jpg
Graphic courtesy John Maeda and Rebrand Cities.

A new hope

Armed with this awareness about our customers, I reworked this screen to simulate a first meeting between a design agency and their client. The screen would ask them a series of questions to understand their needs, and then our tools would build a site that addressed their goals. My biggest concerns with this approach were 1) not scaring people with too many questions so early in the signup flow, and 2) knowing which questions to ask.

I quickly developed high level strategies to tackle both these problems. Designing with a mobile context in mind would ensure that I made it quick and easy for people to answer our questions and using a tool like Hotjar, I could quickly run a series of polls to validate and modify my questions.

Learning our customer’s vocabulary

One of my questions could be answered in a number of ways and therefore required an open text field to collect people’s responses. Of course, typing on a phone sucks, so I wanted to make the field as painless as possible. An autocomplete feature made the most sense as it offered people suggestions while they typed their answer. We could not only save people time from writing out the full word but we would also get cleaner data too. To pull this off, I needed a collection of commonly used terms to power the suggestions.

I created a new poll that included the question as it would appear in my design and quickly noticed the answers weren’t what I expected. Thanks to HotJar I was able to turn off the poll, rephrase the question, and launch the poll again within a matter of minutes — all without writing a single line of code. This time around I was happy with the data coming in and let the poll run until we had enough responses. After a couple days we had a lot of raw data to comb through so I used a spreadsheet and a PHP script to clean it up and help us make sense of it all.

php2.jpg

Build, measure, learn

Another question asked people to pick their answer(s) from a handful of choices. I drafted a list of options based on data we had collected in the past and ran them in a poll along with an “other” field that allowed people to type their answer if couldn’t find what they were looking for in my list.

The first iteration of this poll was dominated by responses entered in the “other” field. Some of them were variations of what I had in my list but there were also lots of common responses that were not represented in my list at all. I used this data to update my list and then published another version of the poll. There were some decent improvements in the results but it was still not quite there yet so I rinsed and repeated. This time I got it right because the majority of the responses were selected from my list rather than entered into the “other” field. In addition to that, most of the people that did fill out the “other” field either flipped me the bird or just said hi.

Progress on multiple fronts

I wasted no time while these polls were running and started building the form right after the design was sketched out. The code was built in small chunks which made it easy to review and kept the momentum of the project going with visible progress. As the data came in, I added it to my work in progress until I had enough of a prototype to run some usability tests. The tests revealed some really minor tweaks that needed to happen but mostly confirmed that we were on the right path. Then, shortly after I completed my last poll, I had everything I needed to run my A/B test at scale and see how this new design stacked up against the old one.

The results

It didn’t take long for us to reach statistical significance but I still let the test run for at least seven days to account for any seasonality during the week. The results indicated that this design profoundly influenced our business metrics. Like it says in NEA’s Future of Design report:

Ultimately everything a design team does has consequences (both good and bad) for a business. The more you know about the business and its goals/intentions, the better a design team is positioned to deliver good experiences and results for the business.

— Tim Riley: Senior Director of Digital Experience at Warby Parker

This post also appeared on Automattic’s Design blog.

Building better app layouts

I’ve been writing CSS for over ten years and I’m glad to finally see there’s a way to create a fixed navigation that doesn’t feel like a hack. Among other issues, normally you would need to add ghost padding or margins to account for the fixed positioning. Thanks to the adoption of Flexbox that should be a thing of the past.

I’m working on a design for an App-ified PWA version of my blog and wanted to try something different with the navigation. My design has a navigation bar fixed to the bottom of small screens for easy and comfortable access while maintaining a conventional navigation at the top of the screen for large ones. I thought that would be perfect for Flexbox and there’s no better way to see how it feels than to mock it up.

With my trusted CSS Tricks guide at hand I made my way to Codepen and created a prototype to try it out. I used Flexbox to create two panes that take up all the space on the screen. One with a fixed height for the navigation and another big stretchy one for the content. It was easy for me to change the position of the navigation on different screen sizes thanks to Flexbox’s ability to change the direction columns flow.

The code

This is the code I used to make it work:

HTML

<div class="navigation">
  ... navigation goes here
</div>

<div class="content">
  ... content goes here
</div>

CSS

html {
  height: 100%
}

body {
  flex-grow: 1;
  height: 100%;
  display: flex;
  flex-direction: column-reverse;
}

@media (max-width: 850px) {
  body {
    flex-direction: column;
  }
}

.content {
  flex-grow: 1;
  overflow-y: auto;
}

The prototype

Head on over to Codepen, check it out for yourself, and let me know what you think in the comments below.
View the prototype

Designing with words

Reading Storytelling for User Experience by Whitney Quesenbery and Kevin Brooks got me thinking about different ways of exploring ideas in my design process. It inspired me to try something new on a recent project. I found that both the end result and the journey were very rewarding.

Starting with words

For most projects I start with sketches, flow diagrams, or wireframes. The end result is always some sort of visual output. I have my graphic design background to thank for that. With this project, I changed things up a little and described my user journey in writing rather than illustrating it. The end result was like a story about our customer’s journey through our signup process.

I broke each step into a series of bullets that were short and concise. It was important for me to make them easy to read. My self imposed constraint forced me to keep things at a high level. This made it better for collecting feedback and moving onto the next stage of the project because people didn’t get caught up in small details.

Words are cheap

I don’t even need to think when I type because of how long I’ve been using a computer. It’s like I look at a screen and words appear right out my brain. Working on this project in this way made me realize that I can type quicker than I can sketch. I was able to cover lots of ground in less time and generated more ideas than I normally would.

I worked in waves by documenting my thoughts and then editing them. The ideas got better with each edit and sometimes they’d even spark new ones. I had to be diligent to capture them before they escaped. My keyboard skills came in hand as I copy and pasted new phrases, finished them off, and then got back to the original thought. After a while, the content began to mature. and I started to think about the implementation.

Structure brings Freedom

I remember my visual brain going wild while writing these flows. The descriptions painted such vivid pictures. I couldn’t wait to get to the visual part of the project. Once I did, I found it really easy to get going. Working within the constraints of the written text was very fun for me. With the high level details all figured out, I was able to focus on the details and get creative.

Conclusion

With this experience behind me, I can easily say I’d do it again. Heck, I’d like to see if there are other ways I can incorporate words into my design process. I’m going to continue exploring and writing about it here so check back soon. If you have any suggestions let me know, I’d love to hear about them.

Product and marketing: two sides of the same coin

Colin Bentley and Brian Donohue are product managers at Intercom — a product first company. They were interviewed on the company’s podcast about their experiences since they joined the company of 30. Today, there are 300 people working at Intercom and their product services over 100,000 active customers.

As a product first company it was normal for them to think if you built a great product, everything else would follow. People were cynical when Intercom started making their first marketing hires.

Over time, the product folks began to see they weren’t very different than the marketers. They realized they both focus on the needs of their customers. While the product managers build features, the marketers position them to their customer’s so they’re desirable and useful. As an example, Twitter’s tweet button could have been easily labeled submit. Instead, a lot of work went into positioning it as a Tweet button so that Tweeting could become a thing.

Today, Colin and Brian explain how important it is for their team to collaborate. It’s essential for them to fuse both the marketing and product perspectives. This allows them not only to build the most important things for their customers but also gives their product a unique place in the market.

Listen to the full podcast on Intercom’s site.

Apprehensive feet waiting to take their first steps

How to get started on your next design project

Getting started on a new project can be intimidating. Thankfully as designers we have these tools to help us get the ol’ juices flowing.

User flows

Experience your product as the people using it would. Take screenshots, write notes, and press every single button you find. Document your journey in steps as you go and take notes.

Stakeholder interviews

Learn from your peers and customers. Find anyone that’s worked on your project before you, speak with your peers, and interview the decision makers on the project. Most importantly, get in touch with the people using your final product.

Competitor analysis

See how your product measures up and find some inspiration along the way. Start by identifying your direct competitors. Then expand search to similar products. Finally look for companies in completely different fields. Make a list of the defining features of your product. As you review each competitor: document the features they have, take notes, and add new features to your list that you find interesting.

Sales safari

Gain a unique perspective about your customer. Find people using your product without influencing them in any way. The best way to do this is to look up things they’ve said in the past by checking your forums, customer support requests, old surveys, and social media accounts. Look out for how they speak, what they think about your product, and what challenges they’re running into.

Inspiration boards

Capture your favourite pieces to inspire you throughout your project. Visit your favourite inspiration sites or use screenshots from your competitor analysis. Save images and arrange them in a way that makes sense to you. Make it really visual and physical. Print it out if you have to and stick it somewhere where you can see it from time to time.

Data analysis

Find out how your product is performing. Get familiar with tools like Google Analytics, mySQL data bases, and what ever reporting software your company uses. Understand your business performance metrics so you can figure out how you can make a real difference both for your customers and your business.

Communication

Keep everyone up to date with your progress. Share your findings as frequently as you can to spur discussion and gather ideas. It easier to get buy in when people are involved in the process. Get people excited by posting your diagrams, notes, and sketches for everyone to see.

Conclusion

What does your design process look like? Is there anything on my list that’s missing? Want to learn more about one of these in more depth. Reach out on Twitter and let me know.

The key to success for your product

I just listened to Scott Bellsky explain the importance of a product’s first mile on Intercom’s podcast. The first mile comes right after someone signs up for your product. It is often referred to as an onboarding experience. This is where you orient your users within your product and give them a sense of what they’re going to do next. Careful attention is paid to the type of copy you write, how your tour looks, and what defaults you set for your users.

According to Scott, the key is to have someone do a few key things that shows them the value of your product over the long term.

Here are some of the notes I took from the podcast:

  • The first mile is usually the last mile a team focuses on. Teams often spend lots of time building their products and only think about the onboarding experience as an after thought or when they see their users aren’t sticking around.
  • Scott believes that in the first 15-30 seconds of every new customers experience, they are lazy, vein, and selfish. Lazy because they don’t want to read or do any work. Vein because they want to know how the product will make them better. Selfish because they want the product to make them look good. These are important characteristics to keep in mind when crafting this experience.
  • He suggests that maybe we shouldn’t tell them or show them, but rather we should just do it for them with the defaults completed.
  • It’s important to remember we can’t believe people will have a relationship with us if they can’t get through those first interactions. They can’t buy into our vision or to recommend us to other people. It just needs to serve them now!
  • The first mile is successful when users reach the “zone”. When they experience that sense of accomplishment and see how the product will serve them in the future.
  • Messaging apps like Telegram and Slack have really smooth onboarding. They even think about the small things like the magic link to log you in because when users come back the second time, they don’t usually remember their passwords.
  • The first mile isn’t something that can be done once and forgotten about. It’s important to continually optimize the first mile by understanding your users and being empathetic to their needs and circumstances.

You can listen to the full podcast on Intercom’s blog or your can learn more about the first mile by reading Scott’s blog post on Medium.