Mind Tools -  Essential skills for an excellent career
About Mind Tools Membership Corporate Services Tool Explorer Contact Us
Newsletter 249
July 31, 2012

In This Issue...
Business Experiments
Gap Analysis
How Good Are Your Project Management Skills?
Uncommon Service
Managing Overconfident People
Facebook Find us on
Twitter Follow us
on Twitter
  Deliver On Your Ideas!

Last week, we looked at how you can win support for your new ideas. This week, we're focusing on how you can implement your ideas and initiatives successfully.

Our featured article looks at how you can test your ideas, without taking excessive risks, with business experiments.

Then, find out how you can use Gap Analysis to identify what needs to be done to implement your ideas.

Successful implementation also relies on good project management skills - find out how to improve yours with our self-test.

Enjoy the newsletter!

  James & Rachel

  James Manktelow and Rachel Thompson
MindTools.com - Essential skills for an excellent career!
Featured Resources at Mind Tools
Business Experiments
Taking Intelligent Risks

Learn how to test ideas reliably, and grow your business without taking unnecessary risks. All Readers' Skill-Builder
Business Experiments
Gap Analysis
Identifying What Needs to be Done in a Project

Use this simple tool to identify what needs to be done to meet your project's objectives. All Readers' Skill-Builder
Gap Analysis
How Good Are Your Project Management Skills?

You need a number of key skills to manage a complex project successfully. Take our quiz to identify areas where you can develop your skills. All Readers' Skill-Builder
How Good Are Your Project Management Skills?
... And From the Mind Tools Club
Uncommon Service, by Frances Frei and Anne Morriss Speaker

This book looks at how your organization can become more successful, by prioritizing service over other areas of your business. Learn more about it here. Premium Members' Book Insight
Uncommon Service
Managing Overconfident People
Controlling the Effects of Overconfidence

Learn how to manage overconfident team members.
All Members' Skill-Builder
Managing Overconfident People
Win-Win Negotiation

Find out about the basics of good negotiation with this short video. All Readers' Video
Learning Strategies

Too Busy for Training?

Join the Mind Tools Club!

Learn the skills you need - when you need them, and in easy, bite-sized chunks.

Click here to boost your career

Sticky Notes
Editors' Choice Article
Business Experiments
Taking Intelligent Risks
(Also known as Pilot Projects and Pilot Experiments)

Your boss has asked you to develop a new service that will boost customer retention, and you've come up with several ideas that you think might work.

The problem is, you're not sure which ones will have the biggest impact. And you don't have the time and resources to implement each idea fully. So, which idea should you move forward with?

This is one situation where it's useful to set up business experiments. By running these, you can test ideas on a small scale, before you take big risks or commit significant resources to a larger project.
Learn how to test your ideas.
© iStockphoto/Pgiam
In this article, we'll discuss how to conduct business experiments, and we'll look at how you can learn from them.

Why Experiment?

Many businesses don't grow unless they innovate, and innovation comes from developing and implementing new ideas. However, innovation usually carries a risk of some kind: fundamental assumptions may be wrong, new products may not work sufficiently well, customers may not accept your ideas, and so on.

This is why it's so useful to run business experiments. By testing ideas, you can understand how they might perform on a larger scale. You can also learn from your failures and mistakes, and refine your ideas to make them better.

Business experiments can involve anything from very basic tests (for example, changing the welcome message on your website or testing a new process for dealing with telephone enquiries) to more-complex projects (for example, testing a new product or service with a small number of customers).

Business Experiments and Risk

Business experiments can sometimes seem fraught with uncertainty and risk. After all, there's often no way to know whether they will succeed.

However, that's kind of the point. When you do a business experiment, you're opening yourself up to risk in a very controlled way. Although you clearly want to avoid failure, you're actually willing to fail, so that you can avoid more costly mistakes in the future, and so that you can develop and exploit high risk opportunities.

In the short term, business experiments may seem too small and too risky to be worth the effort. However, when you look at the benefits of these over the long term, you'll see that it's often more costly not do to them. By being willing to learn - both from the ideas that succeed and from the ones that fail - you'll uncover the successful ideas that will help your business or organization be the very best that it can be.

Tip 1:
Depending on your role, you may need to get permission from your boss or senior management team before you conduct a business experiment, even if the risks are minimal. Use your best judgment here.

Tip 2:
If you need to plan how to get support for your experiments, see our article on Sales Skills for Non-Salespeople, and our Bite-Sized Training session, Sell Your Idea!

Tip 3:
Business experiments won't be suitable in all situations - for example, where you're dealing with a very small market, or where success can be copied very quickly. Here, you'll have to make a decision based on your instincts and on the information that you have available.

Tip 4:
Before you conduct a business experiment in a "live" environment, you may also want to test your ideas with users and customers through interviews, focus groups, or surveys, where appropriate.

How to Organize an Effective Experiment

To organize an effective business experiment, consider using the approach below. This is based on a process developed by innovation expert, Thomas H. Davenport.

1. Create a Hypothesis

As with scientific experiments, the first part of the process is to create a hypothesis - this is your prediction about what will happen if your test is a success. For example, "Sending out two extra mailers a month will increase overall revenue" or "Moving the navigation to the left-hand side of the website will keep users on the website for longer."

As you create your hypothesis, think about the following questions:
  • Can you measure what happens when you make the change? (You need to be able to "pass or "fail" the test, based on your hypothesis.)
  • Does it fit with your team and organization's overall strategy, goals and values?
  • Does it add value to what you do?
If the answer to any of these questions is "no," then you may want to amend your hypothesis or consider not doing the experiment at all.

Finally in this step, identify how you'll measure the success of your experiment. Ask yourself, "What does 'success' look like?"

What you measure will depend on your industry and on the type of experiment you're conducting, and it can include metrics such as sales and revenues, new enquiries, visits to key web pages, or more general factors such as ease of use, or feedback from customers or team members.

Again, also think about how these measurements will affect your overall goals and strategy. For instance, you might define success as "Increasing website page views." However, this won't be viewed as a success if it causes a drop in overall revenue. (This might happen because people have stopped going to your online store.)

2. Design Your Experiment

Now you need to think about how you'll conduct your business experiment. Outline what you'll be testing and how long you'll be testing it for.

Testing a new product or process is relatively simple - you just need to run the experiment for a specific time, and then use the metrics you identified in the previous step to evaluate its success.

However, if you're testing a modification to an existing product or process, you need to compare "like for like" scenarios, so that you can measure the success of your hypothesis. You can do this by using a "control group" and a "treatment group." (This is sometimes called A/B testing or split testing.)

Start by controlling as many variables as you can - that is, try to minimize change wherever possible.

Your control group is what you use for your baseline measure. Try to avoid making any changes with this group during your experiment. Do the same with your treatment group, except for the change that you want to test. You can then compare your control group with your treatment group to test your hypothesis.

For instance, imagine that your hypothesis is, "Opening until 10 p.m. will increase profits in retail stores." You choose two stores as the treatment group; these will open until 10 p.m. Your other stores are your control group; these won't change their opening hours. Then, after two weeks, you'll compare each store to see if opening until 10 p.m. increased profits during the test period.

In some cases, it will be impractical to use a control group and treatment group simultaneously. (You wouldn't be able to run the test above if you only had one store, for example.)

When this happens, you can use carefully-selected past data as your control group, and then use the data that you get when you make the change as your treatment group. In the example of the store with only one site, you could compare sales this year with sales for the same period last year.

Overall, keep your experiment as simple as you can. The more complex the experiment, the more it will cost to do, the more risks it will involve, and the more time it will take to analyze the results. However, you need to make sure that your experiment will provide meaningful data.

Tip 1:
When you use a control group and a treatment group, test one factor at a time. For example, if you tested extended opening hours and brighter lighting, you wouldn't know how which of these affected profits. (You can test other factors in further experiments.)

Tip 2:
To get a better understanding of the risks associated with a business experiment, use tools like Risk Analysis and the Risk Impact/Probability Chart.

Tip 3:
If you need to design and build a prototype service or product to test, don't underestimate the amount of effort that will need to go into this. If you produce something that is sub-standard, your experiment is more likely to fail.

3. Run the Experiment

Once you've designed your business experiment, it's time to take action and run it.
Inform everyone in your organization who is affected by the test, before it goes live. They should know how you'll be conducting it, and why it's taking place.

Once the test is live, monitor it to make sure that nothing happens that could distort your findings. For instance, factors such as bad weather could affect sales in the example we highlighted above. If events like this do happen, consider re-running the experiment or removing some of the data from your final analysis.

If you're testing new products and services with customers, you may want to let them know that they're using a test product. This is a useful way to protect your reputation if things go wrong. It also encourages people to give feedback.

You may also want to make sure that people who are affected negatively by the experiment are compensated appropriately.

4. Analyze Results and Follow Up if Necessary

Once you've conducted your experiment, take some time to analyze the results thoroughly.
First, compare actual performance against your hypothesis:
  • What was supposed to happen and what did happen?
  • Why was there a difference?
  • Was the experiment a success? (Don't be too disappointed if it wasn't!)
  • What unexpected consequences happened as a result of your experiment? How could you manage or take advantage of these?
Then think about how you can use what you've learned. Would conducting further experiments help you get even more value from your idea?

For instance, would opening stores until 9 p.m. instead of 10 p.m. make them more profitable? Or would modifying a product further result in even better feedback from customers?

In some cases, you can take action on what you've learned at the same time that you continue to conduct further experiments. Other times, you may have to conduct follow-up experiments before you can understand the best way forward.

There is a whole range of statistical tests that you can use to analyze the results of business experiments.

When you use these, bear in mind that it's often very difficult to control external factors in business experiments. This means that significance/confidence levels are often much lower in business experiments than they can be in laboratory experiments.

Key Points

Business experiments help you test your ideas and gather further information, before you commit significant resources to a large project. They can be anything from very basic tests to complex projects involving prototype products or services.

To conduct an effective business experiment, do the following:
  1. Create a hypothesis.
  2. Design your experiment.
  3. Carry the experiment out.
  4. Analyze results, and follow up if necessary.
Sometimes, you can implement what you've learned while you continue to conduct further experiments. Other times, you may have to conduct follow-on experiments before you can understand the best way forward.
Share this article:   LinkedIn  Facebook  Twitter  Google+1
A Final Note

Next week, in our 250th newsletter, we're exploring how we and our people can best cope with change.

Until then, have a great week!

James Manktelow

Email us
Mind Tools
Essential Skills for an Excellent Career!
Mind Tools will treat your email address with complete respect and will not circulate it to any third party. (Click here to view our Privacy Policy.)

If you have enjoyed this issue, please do email it on to your friends and co-workers.

To find out about new resources on the Mind Tools site as soon as they're uploaded, click here to subscribe to the Mind Tools RSS feed (you'll need an RSS newsreader installed).

You can also use Mind Tools in your organization. Our corporate division provides solutions from 25 to 10,000+ users and we connect seamlessly with leading LMS/Portals, so you can put our unique toolkit directly into the hands of your workforce. Find out more here.

We welcome appropriate reprinting and reuse of Mind Tools material, but you must get our permission first! To do this, please visit our Permissions Center.

© Mind Tools Ltd, 2012.

This newsletter is published by Mind Tools Ltd of 2nd Floor, 145-157 St John Street, London, EC1V 4PY, UK.

Mind Tools Ltd (Company Number 04829074, VAT Number: GB 840 1273 62) and Mind Tools (North America) Ltd (Company Number 05610859) are companies registered in England and Wales. Registered office: Hardwick House, Prospect Place, Swindon, Wiltshire, SN1 3LJ, United Kingdom.