Microsoft Surface Case Study

Microsoft Corporation is a multinational corporation headquartered in Redmond, Washington, that develops, manufactures, licences, supports and sells computer software, consumer electronics and personal computers and services.

Its best known software products are the Microsoft Windows line of operating systems, Microsoft Office suite and Internet Explorer web browser. Its flagship hardware products are Xbox game console and the Microsoft Surface series of tablets.

It is the world’s largest software maker measured by revenues.

Microsoft has dozens of digital properties, attracting millions of visitors on a daily basis. Microsoft.com, the company’s network homepage, is one of the most visited sites in the world and serves a host of purposes from brand building to store sales and product support to news.

And the mission statement for the site is clear: To inspire people worldwide to love Microsoft through compelling online experiences, ongoing engagement and direct conversions.

Traditionally, Microsoft manages multiple digital properties with numerous stakeholders embracing different goals and strategies. With dozens of business units and an expansive partner ecosystem, efforts to connect those stakeholders and priorities to drive business goals are front and centre as Microsoft transitions to a One Microsoft.

This requires collaboration and best-in-class practices. “One Microsoft is a continuous process to share best practices among our business units and boost synergies,” said Michael Sheppard, Senior Programme Manager of Experience Optimisation at Microsoft. “And we know it’s imperative to make data-driven decisions to advance those efforts. We search out partners and programs that are built on a solid data strategy.

Microsoft embarked on a rigorous request for proposal (RFP) process in early 2014 to review testing and targeting vendors. After months of research and evaluation, Microsoft determined that Webtrends Optimize was the best fit to meet the broad business requirements of its complex culture and organisational structure.

“We considered a wide range of factors including security, privacy, interoperability, features, functionality and support,” explained Sheppard. “Webtrends Optimize checked all those boxes and we knew they could help us pull together an enterprise-wide strategy while providing individualised support to the various business units across the company.”

Sheppard, who drove the RFP process from start to finish, explained that security and interoperability were remarkable differentiators of the Webtrends Optimize platform.

Also of value to Microsoft is the holistic and rigorous approach Webtrends Optimize applies to testing. Their consultants visit the Microsoft campus regularly for meetings and presentations and Sheppard says it’s grown from a vendor-client relationship into a natural extension of his team.

Today, Sheppard is focused on growing and maturing the testing program throughout the company. His team directly manages testing for the Microsoft.com homepage and leads the use of Optimize in more than 20 other business units. Some of those groups utilise Webtrends Optimize to manage all areas of optimisation, including strategy, design, copywriting and execution.

Others use the Webtrends Optimize team for support and training while creating their own strategy and running tests using the visual editor – an intuitive interface within Optimize for building content and experience optimisation tests. Sheppard is working to grow self-sufficiency among the business units so that digital marketers can run their own experiments with easy-to-use tools.

“Even though we promote the concept of ‘One Site’ at Microsoft, each of our websites and business units has unique goals that need to be supported,” said Sheppard.

“With differing levels of maturity and varying features and functionality on our different properties, Webtrends Optimize is able to meet distinct needs with custom integrations while maintaining oversight that everyone’s efforts are coordinated to meet company-wide goals.”

Joe Worthey, one of the optimisation consultants at Webtrends Optimize who works exclusively on the Microsoft account, says Microsoft understands exactly how to get the most out of testing and optimisation on a large scale.

“Enterprise-scale optimisation is more than just testing to find the right button colour,” said Worthey. “It takes a well-planned and programmatic approach that tests hypotheses and iterates as a part of a larger optimisation strategy. Some tests are simple and some are very complex. But everything is thoughtful and requires rigour, process and documentation. Microsoft is successful because they understand the importance of developing a comprehensive experience optimisation programme that is designed with their specific goals in mind.”

Microsoft builds hundreds of tests a year using Webtrends Optimize. From simple, to highly complex, topics for testing have included site re-designs, usability, purchase funnels and validating new features and functionality.

In this particular test Microsoft wanted to increase online purchases of its Surface product line. The team created a challenger page with new offer copy and implemented a test through the visual editor to determine if it could improve the buy button click-through-rate.

After running two AB tests in an eight-day period, the challenger page was named the winner.

“Our consulting team works closely with the Surface group to understand their needs so that we test to maximise ROI,” said Worthey. “In this case, there was a very limited time offer that wasn’t performing as well as expected. AB testing using the visual editor enabled the team to brainstorm and execute the test in one day, see results quickly and increase revenue along the way.”