When Should I Do Usability Testing?

By | product, strategy, user testing | No Comments

In the last post I explained what a usability study is. If you haven’t read that, check it out here.  

Today I wanted to focus on when to do a usability study, but before we dive into that let me first address what types of products benefit from a usability study.

When I talk about usability studies I’m typically focused on tech products which I group into websites, web applications, native applications or mobile apps. I do believe there is benefit in bringing in the user for other types of products during the entire lifecycle and getting feedback along the way as you design and build, whether that be something physical, like the next great widget or something like a book, which Ash Maurya lays out in great detail in his book Running Lean (for that very book).

Bringing it back to tech products my answer would be that they all benefit from usability studies. I like to focus on the low and high end of complexity as use cases and letting the middle sort itself out, so in that spirit let’s break it down:

Low complexity
A portfolio website to show off your artwork and increase consulting work. Surely you don’t need to do any usability testing. It’s straightforward, here’s some beautiful design work and here’s how to get in touch with you. Boom, done. My counter argument to that line of thinking is, what if you could increase the amount of consulting work you get by 10% with a few hours of work? How about 50%? By putting the site in front of people with the task of hiring a design consultant you could discover that they want to fill out a form rather than call or email you. That small change could increase your rate of return.

High complexity
An advertising platform that allows advertisers and publishers to manage their accounts in real time. This would be a web application with a mobile component that allows  both external and internal users different levels of access. With this type of product you would have many levels of usability testing. From a persona level you have advertisers and publishers, all of which are probably not the same and would need to be broken down into sub groups. Even within those personas it’s often the case that users will need different levels of access, such as the Sales Executive and the Ad Buyer. That’s without yet considering how the internal account managers will need to impersonate users and have their own switches and levers.  

Putting these products in front of users is crucial to saving time and money as you’re building and maximizing profits after it hits the market.  

But when do you put it in front of users? When is the right time to do usability testing?

The short answer is, throughout your entire process, from discovery to design, through development and even after launch.

In Tom Chi’s talk on Rapid Iteration (which I posted back in December) he says, “For everyone who builds products, the real medium is human behaviour in the real world, and your purpose should be…to see that human behaviour modified by what you build.”

The longer answer is that the more usability testing you can achieve during your product creation, the better. In Tom Chi’s talk he mentions a feedback loop of only a few days, constantly giving feedback to the design and engineering teams. That is a high bar to reach for indeed.

Instead, breaking the project down into Discovery, Design, Development and Launch we can start to identify the points at which usability tests can be effective, even if we aren’t on a bi-weekly cycle with our feedback loop.

Product discovery starts with understanding what you’re hoping to achieve. See my post on setting your product vision. Involving users at this stage typically means conducting user interviews or nothing at all. One thing you could do as you’re qualifying your intuition is put users in front of similar products and gain insight from how they interact.

During the design phase I encourage iterative design and adding in parallel design if you have the time. Iterative design is the process of taking user feedback on a design and updating it with marked usability improvement with each iteration that you do.  Parallel design starts with multiple concepts and takes the best from all three to end up at 1 design to start the iterative process from.

You want to work through as many cycles as you can, continually gaining insight from users, before a line of code is ever written. In this great article on the Nielsen Norman Group website it goes through the details of each of these projects and state “there’s no one perfect user interface design, and you can’t get good usability by simply shipping your one best idea. You have to try (and test) multiple design ideas.”

During this phase UX, design and engineering should be talking every day about progress of the project and UX should be thinking about what they’re going to test on a weekly basis. In the video from Tom Chi above his team does it twice a week. Do what you can, if you can only do it once every two weeks it’s better than not at all. By taking what engineering is building and putting it in front of users and finding the flaws (and really awesome stuff) as you’re building it will save you from huge rewrites and updates after launch which saves your team huge amounts of time and money.

As I wrote about a few days ago, when you launch you’re not finished, keep pushing towards your goal that you set up at the beginning of the project and while now you will have quantitative data coming in through the many users you have hitting your app or site on a daily basis, it’s also good to sit with your customers and understand how they’re using it in the real world and continue the loop of improving so you can more quickly hit your goals.

In the next post I’ll discuss How to Conduct a Usability Test on the cheap and if you’re interested in having someone take care of it for you, contact me and maybe we can work together.

Your Future Life

By | strategy, vision | No Comments

I had the idea to treat my life as I would a product that I own. I have set my vision and started breaking down the steps I think I need to take to make that vision a reality, a little bit at a time, experimenting, testing and iterating through my learnings. A part of the grand vision includes working on this blog and putting these posts out into the world.

Just as you wouldn’t go on a grand expedition without a destination, you shouldn’t allow yourself to move through your life journey without setting a true north. I wrote about setting your true north in my post about how to set your product vision. Make it big, make it so big that it’s a little bit scary. It’s your life, so why not shoot for something that gets your heart racing? Do you want a big house or to retire in 10 years or to visit all 7 continents? What’s that life you’ve always wanted for yourself?

Now write it down at the top of a piece of paper (or Google doc). This is your personal vision statement. This is where you start. Make it specific and add depth. If you’re more of a visual person, consider making a vision board. Jen Sincero is a big fan of this and suggests making it as precise as possible. The exact house you want, how you want to dress, the job you want to have, gather images and stick them all up there. A Pinterest board is fine, but having an actual physical version that you have on your wall and you can look at every day is better.

My future life includes making a certain amount of money per year, taking 2 “big” vacations every year, competing in 3 physical events per year (like the Savage Race I have coming up in April), living more sustainably with a small farm (we already have the chickens), and continuing work on building this blog with the vision I shared in my previous post. There are several others which I am keeping to myself, focused on my wife and family.  The key is there is a balance between work and life, in fact it’s tipping more towards the life side of things.  As I wrote in my Christmas day post, that balance is very important.

It all has to start with a vision of your future life, just like building a product. Take all the ideas about how you can get there and start to make assumptions or hypotheses and write those down to. This blog has been an experiment and I’ve learned a lot in a short period of time. I had certain goals set out, some I achieved, and some which I didn’t. At the end of the month I’ll share a retrospective of 2016 and at the beginning of the New Year I’ll share a look back at this experiment and all the learnings that came along with it.  I also have some cool posts lined up for January and I’ll lay out my plan forward.

Want more clarity on how to take those visions and turn them into actionable strategies. Michael Hyatt, Ruth Soukup, and Seanwes all offer different perspectives. In future posts I’ll be sharing the experiments, iterations and pivots that I believe will help guide me as I move through the unknown, just like I do when I’m building a product.

What’s your vision? Share it with me in the comments.

Product Lessons from Mark Zukerberg’s Home AI Challenge

By | product, starting, strategy | No Comments

A few days ago Mark Zukerberg posted to his Facebook page about his experience working on building a simple AI to run his home, which he aptly named Jarvis after Tony Stark’s AI from Iron Man. And the same day Fast Company wrote an article on it. You may have heard about it.  

I also discovered that Mark Zukerberg sets a yearly challenge for himself. So, in addition to being CEO of Facebook and having a family he makes it a priority to continue to learn and grow. 

Throughout his update, there were lessons on the right way to build products. He used creative problem solving, iteration and existing software rather than building from scratch to get to his vision as quickly as possible.

From the very onset of the project he faced challenges because there is currently no common API for the home appliances. As home automation AI becomes more prevalent (and it will happen quickly) this is gap that will need to be filled.

He had particular trouble finding a toaster that would allow you to push the bread down without having it turned on. “I ended up finding an old toaster from the 1950s and rigging it up with a connected switch.”

This is an example of taking a problem and turning it on it’s head. There were probably other more complicated solutions, like reengineering a present day toaster, but he took the problem and broke it down to its smallest parts and attacked it from a new angle. This type of creative problem solving is necessary as we navigate building products.

Another hurdle was allowing Jarvis to understand voice commands. As we should when building any new product he took an iterative approach by first communicating with Jarvis via text message and later using that as a springboard to allow for voice commands.

I was particularly interested by his discovery that he often prefers to communicate with Jarvis via text rather than using voice commands. And based on this observation he determined that as we build out AI, we need to recognize that they will also need messaging interfaces in addition to voice.

Building iteratively and real world experimentation and discovery at work.  

In order to create the messaging interface he could have built an app from scratch, but chose instead to build a Messenger bot.  Why? Because it was easier. There are open source APIs to build these bots, which allowed for a strong base to start with. When building towards MVP it’s important to move as quickly as possible and often that can be done through retrofitting your ideas into existing solutions, even if it’s not as sexy as you imagine it to be when you’re dreaming of the final version.  

Beyond his iterative approach, creative problem solving, and building on top of existing work it’s important to acknowledge the very fact that he challenged himself to do this. If he had never put pen to paper and challenged himself at the beginning of the year there’s little chance that he would now have a kick ass home AI, not to mention the knowledge gained, including learnings he wasn’t expecting at the beginning, which is common in taking on big challenges. You just don’t know what you don’t know yet.

As I’ve been saying from my very first post. Start. 

Then, make small iterative progress towards whatever it is you’re looking to do. Set a goal and break it down into bite sized chunks and then break it down some more.  Make it so easy that you’d be hard pressed not to make a little progress every day.  

From beginning to end it took Zuckerberg a little over 100 hours to get to the current version of Jarvis.  Think about what you could do in 100 hours over the course of this coming year.  

What challenge do you want to tackle in 2017? Let me know in the comments below.

Know Your Vision

By | strategy, vision | No Comments

I recently put a deck together to help explain my philosophy and execution around building great products. (Want a copy? Send me a note.) The first section is entitled “Clear Vision”.

You can not build great products without a clear product vision.  Beyond products, the same goes for anything else you’re hoping to achieve, whether it’s losing 10 lbs or getting up the nerve to finally get that promotion you know deep down that you deserve.

In Start With Why, Simon Sinek, very concisely states that people don’t buy what you do, they buy why you do it, and then he goes on to explain that concept in glorious detail. I highly recommend picking up a copy. I even made a nice link for you, but if you don’t feel like reading a whole book, at least check out his Ted Talk.  (Then come back and tell me how inspiring it was in the comments.)

You need to understand your why and use this as the backbone for your vision.

This vision (your why) is what you will use to inspire your team and guide them. This is what you will use to pitch your company to investors or yourself to your boss.  This is what you can hold on to when you’re not quite sure what you’re building is the right thing, and it’s your first filter for whether or not what you’re building is the right thing. Does it align with your vision? Is it getting you closer to the future you envision?

As I mentioned in yesterday’s post, this is a build from the ground up situation, and in that spirit it’s only fitting to share my vision of what I’d like to see this become.  My reason behind spending any time on this.  My why.

I could be tempted to list out all the things that I intend to do here, like starting an email list, and creating a course or a book or both.  I could go on about how I’d like to build a community and launch a podcast, but none of that is why.  That’s all what and how.  That’s all strategy and that’s what tomorrow’s post will be all about.

My vision for this site is to help people build better things, specifically creative based products, but I feel these lessons can be applied more broadly.

Oh, and one more thing…

That Start With Why link up there is an affiliate link. Basically if you click on it and buy that book, I get some pennies…I’m not even sure how many pennies…but it’s part of the way I intend to monetize this blog, so if you read my last post, then you know this is another step towards making things a little bit better than they were yesterday.  Included in that are setting up the email address ( and starting (but not quit finishing) the about us page.  Maybe tomorrow.  If you want to set up your own affiliate links on your own blog or site, sign up over here.

31 Posts in December

By | starting, strategy | One Comment

This is my first post.

This is the first post of 31 posts in December.  One a day.


Because this is the start, today is the start. And starting is the part a lot of people don’t do.

But the part that even more people fail to do is to keep going. Over the past few years I’ve learned a lesson that I wish I would have learned when I was much younger. More than anything, consistency delivers results. Diet, exercise, starting a business, following your dreams, or colonizing Mars. Do a little bit at a time, change your habits, create new ones, and those little things change your life.

I’m starting today and posting every day in December, because I know that will start the boulder rolling down the hill. Consistency turns into momentum and with momentum the things that were once hard just become what they are…the things you do.

What does all this have to do with building products or creating solid designs (oh is that what this blog is all about?), honestly it has everything to do with that and I hope over the next 30 days to tie it all together, but today is just the start.

I know (as of this writing anyway) that the site is a WordPress template, with no personal style or design, and I have no logo and there’s not even an About page up yet. I have no tracking, I have no traffic, and I have no intention of sending out an email and driving traffic today.  But tomorrow I hope to.  And that’s how I can start, right now.  That’s the tiny step I can make and the promise that I keep to myself (which I happen to be very good at, following through when I decide to do something), is that for the next 30 days I’ll keep posting, and sharing and building and we’ll see where I’m at come the new year. The only thing for sure is that it will be more than it is today.