Google Search: A paradigm shift for internet discovery
Google Search: A paradigm shift for internet discovery
How a better algorithm led to a better user experience... and hundreds of billions of dollars in revenue
Hello! I’m Pat and Better by Design is my newsletter where I decode the DNA of hit products to inspire tomorrow’s classics. If you’re new here, join 1600+ product creators as we uncover new insights each week.
Intro
One text field, two buttons, and hundreds of billions of dollars in revenue.
A quarter-century after its release, Google Search is as relevant as ever. And despite all of the other products Google now offers, Search still drives the tech titan’s business.
What made the product so powerful?
Let’s break it down.
Product context
Cultural trends
The web creates new challenges for information retrieval. The amount of information on the web is growing rapidly, as well as the number of new users inexperienced in the art of web research. - Page & Brin
In the late 90s and early 2000s, the internet was morphing from a niche network into a global phenomenon. The dot-com boom fueled a rush to digitize, as organizations recognized the potential of this growing medium.
The early attempt to organize the web’s information into web directories (reflected most popularly by Yahoo) was approaching its limits as the amount of content grew by orders of magnitude. Society was transitioning from the information scarcity of the pre-internet era to the information overload of the digital age.
A tool that helped users find the most relevant information was no longer just a convenience; it was a necessity. People craved a way to navigate the explosive growth of online content, creating a perfect storm for Google's search-focused solution.
Competition
Google wasn't the first search engine.
Early pioneers like AltaVista, Lycos, and Yahoo! were already established players and had set the stage. However, their technical approaches had limitations, particularly when it came to providing relevant results at the new scale of the internet.
As Page and Brin state in their original academic paper, the approaches of the competition had inherent problems:
Human maintained lists cover popular topics effectively but are subjective, expensive to build and maintain, slow to improve, and cannot cover all esoteric topics. Automated search engines that rely on keyword matching usually return too many low quality matches. - Page & Brin
Google's new algorithmic approach and the simplicity of its interface set it apart. Additionally, over time Google remained focused on search while others, like Yahoo!, split their attention in multiple directions. This clear differentiation and focus gave Google an edge and helped it become synonymous with online search.
Audience
Google Search had a universal audience – anyone with internet access.
But its appeal was particularly potent for students, researchers, and businesses that relied heavily on information retrieval. Google’s association with Stanford University helped expand its reach within the academic and tech communities of Silicon Valley, providing a fertile incubator for the nascent tech.
These initial users’ positive experiences catalyzed a word-of-mouth effect, contributing to Google’s early adoption.
Monetization
Google Search was, and is, a free service for consumers. Its initial focus was on capturing market share by building a superior search engine, not on monetization.
In fact, in the early days, the founders were hesitant about placing ads on Google, as they were concerned it could compromise the integrity and objectivity of the results.
In time, they achieved both massive market share (estimated ~85% in 2023) and fantastic profitability (estimated ~75% operating margins for its core business).
The introduction of Google AdWords (now Google Ads) in 2000 offered a powerful lever for monetization. By delivering context-sensitive advertisements, Google could leverage its search dominance while providing advertisers with unprecedented reach and precision.
I think it’s fair to say that Google’s success in advertising fundamentally changed the landscape for digital ads, making it a go-to example of how to monetize for an entire generation of internet companies.
Levers of design impact
Innovation
Google Search's main innovation was its algorithm for ranking results.
The PageRank algorithm redefined how search results were generated, turning the conventional wisdom on its head.
Unlike the major players of the time, PageRank determined a page’s importance by evaluating backlinks from other websites. This system proved much more useful, reliable, and scalable than the previous keyword based systems.
Interestingly, the 1998 patent for PageRank references a site-scoring algorithm called Rankdex developed by Robin Li in 1996. Li went on to create Baidu, China’s leading search engine.
This innovative, new approach to search clearly shifted the landscape for discovery on the web as we know it today.
Functionality
Google was intensely focused on providing a useful, functional, search experience.
The speed and accuracy of search results drove home the tool’s usefulness. While other search engines at the time struggled to provide relevant results that kept up with the internet’s growth, Google’s method factored that in from the start and achieved superior relevance that set it apart.
The clutter-free homepage, featuring a solitary search bar, simplified the user interface and emphasized its primary function: to help users find what they were looking for. It also helped improve the performance of the website in a time when internet bandwidth was limited.
For users, these factors combined to deliver a big leap forward in search experience.
For advertisers, they offered a new, more useful way to deliver effective ads in a targeted fashion.
This one-two punch of functional excellence returned incredible results.
Aesthetics
While subtle, aesthetics played a crucial role in Google Search's success.
The minimalist design, characterized by the use of white space was a noticeable departure from the design of its predecessors who crammed more and more information onto their landing pages with each passing year.
This aesthetic choice made Google visually distinct in a crowded category, while also defining a clear design constraint for new search feature development.
In contrast to the minimalist backdrop, the vibrant Google logo added a playful, human touch without cluttering or distracting from the core experience. This injection of personality provided a good jumping-off point for the development of Google as a brand.
Sustainability
Google Search's longevity in tech’s ever-changing landscape is a testament to its design, both at the algorithmic and interface levels.
PageRank was designed specifically with long-term scale in mind. This enabled Google to out-compete its early competition who struggled with quality at scale and gave the company a better foundation for growth as the market continued to shift.
Google’s simple UI also sidestepped the UI scalability problems of existing web directories that inevitably reached an upper limit on the amount of content they could reasonably organize onto a single page (or pages).
How to replicate Google Search’s success
“PageRank can be thought of as a model of user behavior.” - Page & Brin
Gut check with “intuitive justification”
From the outset, Google’s creators were thinking about aligning its functionality to human behavior.
As described in the original paper, PageRank served as “an objective measure of [a page’s] citation importance that corresponds well with people's subjective idea of importance.”
It wasn’t enough that the algorithm be objectively better than others; it needed to also be subjectively better. It needed to align with people’s expectations.
By designing specifically to match objective improvement with subjective desirability, Google was able to deliver a technical achievement that resonated with users.
Particularly in our current data-obsessed culture, it’s important for product creators to seek this match. Do a gut check for what Page and Brin call “intuitive justification” in their paper: does this quantitatively good solution hold up qualitatively?
If so, great. But if not, keep iterating until it does.
“Basically, our goal is to organize the world’s information and to make it universally accessible and useful.” - Larry Page
Experiment, but retain focus on the core
Google’s sustained focus on its core value proposition - helping users find the most relevant information quickly and accurately - contributed significantly to its success. But the company also cast a wide net with experiments.
While it’s easy to look at sites like Killed by Google and think “hmm, this doesn’t look very focused…”, I think it actually shows the company’s ongoing commitment to its core. It’s good to try a bunch of stuff. That’s how you uncover new successes! But from a business perspective, cutting off ideas that don’t add enough value is just as important as amplifying the ones that click. That culling practice is an essential part of maintaining the quality of the core business over time.
It’s a good reminder that product creators should keep their core value proposition at the heart of their design process. In the early days it helps to ensure that you nail the product’s foundation and in later stages it serves as a useful filter for deciding what expansion opportunities make sense.
“Google is designed to scale well to extremely large data sets… Further, we expect that the cost to index and store text or HTML will eventually decline relative to the amount that will be available. This will result in favorable scaling properties for centralized systems like Google.” - Page & Brin
Prioritize scalability
Google's ability to scale its search engine, handling billions of queries daily without compromising speed or accuracy, is an essential factor in its success.
This can be a tricky balance in the early days of a product as it forces some trade-offs with what you can execute and ship in the short term. The way you get both scalability and fast execution is to be diligent about our last point: focus. Spreading yourself too thin inevitably comes at the expense of a more robust, scalable core.
From the outset, product creators should be mindful of scalability. This doesn’t mean you’re designing for Apple or Google’s current scale, that’s way overkill. But rather it means that you’re thinking a couple of steps ahead for what you might need to support. You’re not over engineering to support scale you might never achieve. You’re designing to give yourself some runway and leverage when things take off.
You may decide there are worthy scalability trade-offs to facilitate shipping early and often, but you won’t regret taking the time to design architectures that can handle growth with grace.
“We know that Google Earth and Google Maps have had a tremendous impact on Google traffic, users, brand, adoption, and advertisers. We also know Google News, for example, which we don't monetize, has had a tremendous impact on searches and on query quality. We know those people search more.” - Eric Schmidt
Leverage ecosystem network effects
Google successfully harnessed network effects within its ecosystem to amplify its success in a number of ways.
As more people used Google Search, the quality of search results improved, attracting more users and creating a virtuous cycle of growth.
As advertisers saw the user base expand, they were incentivized to advertise with Google, thereby bolstering its revenue stream.
As Google expanded its product suite over time, it built an ecosystem of apps that all served a role in strengthening the company’s position in search (Gmail, Maps, Chrome, YouTube, etc…).
These didn’t all come at once, but rather built on top of each other over time, gradually but persistently adding nodes to a network that reinforced Google’s search prowess.
Product creators should look for strategic ways to leverage network effects wherever possible. Unlocking a virtuous cycle on either the marketing side or product side can have an outsized impact well worth the cost of discovery.
Next time on Design DNA
"Don't aim to be the best. Be the only." — Kevin Kelly
While Kevin Kelly was talking about carving your personal niche, I think the advice works just as well for products.
In a video game market often marked by trying to be the “best” in a processing power arms race, Nintendo chose to take another path.
It stopped chasing “the best”, embraced being “the only”, and developed a hit product with a fanbase that’s like no other.
We’ll unpack what makes the Nintendo Switch better by design, next time.
Until then,
Pat 💚
If you got a little value from this post, consider subscribing, sharing, or following me on Twitter. If you got a lot of value, consider pledging to support my work with a paid subscription in the future.
