People love Google for its ability to find and rank content on the web, but most people have no idea how everything works. The internet is changing as everything gets linked up and becomes ‘cross-device compatible’ and Google has to keep up. More and more searches, especially mobile searches, are for consumable media, as cord-cutters around the world ditch their cable subscriptions and physical media for on-demand access to TV, movies, games and music.
The rapid growth of all these things are forcing Google to index the world differently. They are now relying much more heavily on complex speech, language and translation APIs, as well as individual personalisation and machine learning to generate results. Where Google used to be able to index and surface web pages, now they can show results for concepts, ideas and media without websites, or tell you about these things using voice. Understanding how all these new technologies come together, will change how you think about a simple search query forever!
1 / 94 Use ← → to navigate
Auto-generated transcript - may contain errors.
Tap a timestamp to jump the video.
Like you said, my name is Cindy Crum. I am super excited to be here talking to you today about the new way that Google understands the world. Just a little bit more background about me. I love cats, but I also love dogs. This is my dog, Barkley.
He's adorable. He wears a bow tie every day, he's a gentleman. So I have been in mobile marketing and specifically mobile SEO for a very very long time and in that time, I've gotten to work with some of the biggest companies in the world and doing that has made me really good at predicting the future because the big big companies aren't as nimble as small start ups.
They need to know what the future's gonna look like so that they can put plans in place early because they know they're gonna be slow getting through the plans. So they have to anticipate the future better than other companies and so that's one of the biggest strengths I bring to my clients and it's also started to be useful in conferences because people are like curious what I think now.
So that's why I'm here. So what I'm here to tell you is fundamentally Google is changing the way they work, and most marketers are missing the point. So we're gonna start with this diagram of how Google understands the world, or well, this diagram of how the world just works in general, right?
We have new innovation and then we have it starts out low and slow, and then there are technology improvements where it ramps up and up and up and adoption is really really fast, and then after that a product will kind of level out because everyone's gotten to that product.
They know what's going on. They don't need to buy it. And this is the s curve of technology. It's in any kind of business one zero one class, But within these changes you see smaller changes. Sorry you guys, the slides down here aren't matching up.
So within that growth are massive small component changes where you have little things that get better. So you have large system improvements like going from an iPhone one to an iPhone five, but within that you have things like we have better camera, more storage, little s curves within the big s curve, but there's also this subversive thing that's always happening where there's an incumbent technology and a replacement technology.
So the incumbent technology is what we're all using, what we're all paying attention to, but the replacement technology is sneakily happening in the background and it's following the same path, right? A slow start where it's gaining speed and then all of a sudden it takes off.
And we've seen this time and time again with our technology. Think about music going from live to records to A tracks to tapes to CDs, right? You had all this stuff, you had all the CDs and then it went to MP3s and you're like, but I don't want to buy it again, but you have to because the replacement technology comes and forces you to.
Same thing with film, then Betamax, then VCR, then CD or DVD, and now we're streaming everything. The replacement technology sneaks in and changes how everything is done. And I think that's what's happening with Google right now. And so you can see, this is Rand, did you guys see Rand talk yesterday and he'll be doing a keynote later today?
So Rand published a study of data that came out of a company called JumpTap, no, JumpShot, and what the study showed was that the behavior on mobile and desktop search is different. So we already know from Google that mobile search represents more than fifty percent of the searches that people are doing.
More than half. But the problem is most marketers are still focusing on what's happening on desktop and they're ignoring mobile. But on mobile it's just different. The results can look different from phone to phone, They change based on location. They change based on a lot of different factors that aren't put in to the regular web results.
And those changes are causing people not to click. Sixty one percent of mobile searches don't result in a click and it's because instead of showing websites and stuff like that, Google is showing a bunch of different stuff at the top of the results that are stealing the clicks from the website.
People are getting the information that they want directly in the web result. So it's things like map packs, businesses, things like videos, things like lists, this is a random curated list of kids song about dogs, random. Knowledge graph. Knowledge graph carousels, where you get a whole list of knowledge graph options and then you have to click on one and then you get to the knowledge graph.
Answers, where you search for, you try and answer a question and it shows you the answer directly in the result. Why would you ever click through to a result when it's lifted it for you and put it into the search results? You just wouldn't do it.
And a company called Stone Temple out of the US did a study that showed that in mobile and voice search, the number of answers and knowledge graph style results is growing. The purple lines are voice search and the red lines are mobile search, but Google is getting more and more dense or showing more and more of these kinds of results that are not websites.
So if you're marketing your website really hard but you're not paying attention to what Google's showing at the top that's much richer than just a blue link, you're missing the point. And Google has doubled down on all of the technology that is not just web based search or it's not just screen based search.
Where Google's been investing their time and money recently is a bunch of stuff that is eyes free, as they call it. Meaning there's not necessarily a screen. You can't even assume that there's a keyboard. Things like Google Home, Google Assistant, their VR stuff, all of the casting stuff.
This is what Google's really been focusing on, investing on, and if that's where they're putting their money, that's where you should be focusing too, because you have to figure out why are they doing this and why would they do this if they didn't have a business model to suit it.
But the answer is they do have a business model to suit it. What Google says in their data shows that the number of voice searches are growing dramatically and it's hitting that hockey stick in the s curve. Voice search is about to go like this.
Google's own data shows that by twenty twenty they predict we're gonna be having more conversations with our digital assistant than we are with our spouse. Now that doesn't say a good thing about marriages, but it does say a lot about what Google is planning for.
If they think that's the reality, then that's what they're gonna invest in, that's what they're gonna double down on. So to predict Google's plans, you have to think about what is their motivation? What are they trying to achieve as a business so that you can meet that?
And from Google's perspective, they know that more than ninety percent of the world's data was created in the past two years. That's a lot of data. As humans with all of our technology, we're generating more and more numeric measurable data about ourselves and what we do and where we go, and now it's quantifiable.
And you have to remember that Google's mission is to organize the information of the world, not just to organize the websites of the world. And so we're creating more and more data or information and Google wants to organize it. The problem is, spoiler alert, websites don't work without screens.
If you're only doubling down on your website, you're gonna miss the boat because Google says everything's going to voice and they're often right about these things. So you have to start thinking, how will my brand, how will my company, how will my product work in a non website environment?
How will we market it? What would we do if we had to market this without a screen? Because Google wants very much to index things that are not just websites. They wanna index music and books and videos and businesses that don't have websites because all of those things can be great and are valuable even if no one bothered to build a website.
Just because a book doesn't have a website doesn't mean it's not a fantastic book. And so Google wants you to be able to surface this non website content in a search. Why wouldn't they? Their job is to organize the information of the world.
The problem is unless you built a website historically Google couldn't find it. Google couldn't know about it or unless someone put your content, was nice enough to put your content on their website or you paid them to. Google couldn't find it. It didn't exist to them because the way Google developed was they sent out these basically blind crawlers to crawl the web and click on links and capture keywords as they moved along, right?
Capture the text in the page and in the anchor links, the anchor text. So they just kind of did it blindly, and we call this the link graph. Google calls it the link graph. This is how this website is linked to this website, so they're probably related in some way, maybe by the anchor text and the keyword and so forth.
So that's the link graph and that's how they were organizing their understanding of the world. But Google's core functionality had to change because it's missing so much content and because the content that it has won't work in all the context that Google wants it to.
Because the link graph is kind of like this. It's just what we found and we piled together. It's not super organized and it's not super scalable. So what they did, this change, is that they reorganized the content that they had. They just took what they had and put it in a different order to make it easier to surface.
So this is like a group of books organized by color. You can organize on any principle you want with it still being the same content. So I'm not saying that websites are going away, I'm just saying that they've been reorganized in something that's more scalable and usable in the future search conditions that Google's thinking about.
And we call this knowledge graph. And I'll tell you right now, this is my theory, Google hasn't said this, this is just what I see and think is happening. So knowledge graph is what we're used to seeing when we get an answer, like a really rich answer at the top of a search result where it has pictures and information and tabs about some topic.
And those topics are related to each other in the same or in a similar way that a website might be linked to another website. A topic is linked to another topic or subtopics. Google understands those relationships, but the thing about these is they're a bit more static and predictable than links and they're harder to game because so many people in the SEO industry were buying links and trying to manipulate how Google understood the world.
Knowledge graph is harder to manipulate. So at the center of this change, the core is the idea of entities and entities are people, places, or things that have relationship to other things like knowledge graphs. So a knowledge graph entry you can consider an entity.
So if we think about this thing as a bucket and buckets are related to mops and brooms and dust pans, right? These are just lateral relationships, but buckets can also be aligned with things vertically to relationships where this is a bucket and this is a bucket handle.
Or this is a bucket and this is the water that goes in the bucket that it carries. So there are all these different kinds of relationships that Google has cataloged but it's not just things like this. It can be ideas like capitalism is related to socialism in a way.
They're not the same but they have a relationship in that they describe similar topics but treat them differently. So you can even look if you've ever onboarded with a new Android phone, it tries to get you to give them information about your interests.
So the first screen shows like all these categories of interests that you might have. Things like sports and news and politics and entertainment, public figures, business, and when you click into one like sports, you're given to a page that's specialized for you based on your location.
It shows you the local sports teams based on your geography. We assume that because you're in this geography you might actually care about these but you might not, so we also do it laterally and we say or you can drill down and just look at all the different kinds of sports that we have information on, all the different leagues.
And then when you click into one, you get all the teams within the league, and then when you click into that, you get all the players within that team, right? So this is a static hierarchy that Google can understand, and so they've started indexing on it and they've even started giving these topics links of their own as if they were websites, but they don't actually have a website.
It's just a link to a knowledge graph result in a Google search result. So these are static things that Google has categorized as entities that people might want to look up and care about. So entities are interesting because they make Google's job easier because they're language agnostic.
They happen before keywords. The bucket exists whether you describe it in English, French, or German it's still a bucket. Or in this example, blue chair, if we search for blue chair, it doesn't matter, especially for an images, it doesn't matter what the text content of the page is, like what language it's in, as long as it's a blue chair.
So a picture of a blue chair on a website that's written in Japanese for me as an image result is just as good because I'm looking for images, not text. And so they can translate the query and show you all of the blue chairs, not just the blue chairs that are written about in English.
Now that may seem nuanced and boring, but Google cares more about language than you think. And it's not just because Google is trying to listen to us in a voice query, it's because they're trying to understand us. You could just type a keyword into a search box, Google could just return website results that had those keywords in it and had lots of other good signals.
When Google can't return websites, they have to do a better job of understanding the query so that they can help you find what you're actually looking for, so that they can help you surface the answer without just giving you a list of websites, because no one wants a long list of links to websites in a voice search.
Do you want Google Home or Google Assistant to read you like, www dot. That's a horrible experience. No one wants that. And if it listed like all ten of the websites in the result, how would you click? Would you be like, the third one.
That's a horrible experience, no one wants that. So Google had to be smarter. They had to use the intelligence that they had and build on it to not just listen but to understand. The problem in voice and regular search is that Google loves English, it's best in English, and if you've lived anywhere where they don't speak English, you've seen that the search results aren't as good when you're not searching in English.
And this is of course true as well in voice search. The problem was because Google was crawling and indexing the text and language of the web, they were putting language before intelligence. It was in the wrong order. So they couldn't build on anything.
They couldn't really understand. They were basically just scraping words and matching the words. And that's not real intelligence. That's fake intelligence. So what happened when Google spread outside of the English speaking world and started doing translation or, well not doing translation but indexing content in other languages is they started with a blank slate like as if they didn't know anything.
And so they were indexing French as if they had never learned English and so they had to relearn all the concepts and connections of the words and the relationships they're in. And that's a really really bad way to go about understanding the world.
It's not real intelligence. It's fake intelligence and it's not how we would do it as a human. If you're teaching a toddler to speak and understanding the world, you say this is a table and this is a chair and they often occur in life together and we call it table and chair.
But then if you try and teach the toddler the words in Spanish, you don't have to recap that this is a table and this is a chair and they often occur together. And that you sit in the chair and eat off of the table.
That you don't have to relearn those concepts, you just add on to the concept that you know with new keywords, new language. And that's how real learning actually happens is with translation. So you reverse the order and you put the intelligence first and the language second, and then you can take the core intelligence and build off of it and iterate off of it with more and more languages.
And that's real intelligence and that's what allows Google to cut the line, right, and to not have to relearn everything because the problem with learning languages, especially languages that aren't widely spoken, is a lot of Google's learning happens from machine learning, which requires very large data sets and feedback in the algorithm.
And so English is great because there are lot of people that speak English, search in English, click or don't click in Google results in English, but if you're talking about a smaller language like Bulgarian, they don't have enough data or feedback in the response loop to have the machine learning and the AI move quickly.
So what they do is they say, we're gonna use English and then translate English to Bulgarian, and then you understand the world based on our core understanding of relationships within English, and usually entities have the same relationships regardless of the language that you speak, so it's a pretty close approximation.
And so that's what gets us from that old kind of search where Google doesn't really understand or it's not so confident in its understanding where when you search for what time it is, it shows you the time but it also is like, we might not be getting this right, so here's some websites that might have a better answer.
When Google understands it, you get to this where they're like, we gotcha, we're just gonna tell you what time it is. This is the real answer, we know when you ask what time it is that you weren't looking for a website because we understand the context of what this question is.
We're not just blindly showing you website results that match with keywords. And so we can see that Google's doubling down in some of this stuff because they're building a bunch of tools that show you entity classifications. So a domain is a kind of entity and Google has classified all the domains on the internet and you can use the market finder tool that they've created to see how your domain has been classified.
They've also got language APIs. The best one is the natural, cloud natural language API, which you can take text from your website and put it in on their web tool of the API and it'll break down the language and show you how the words are being parsed for the top ten languages in the world.
And it'll even break down the sentence and the relationships because this is how it understands when you have multi word queries. It says this describes this, this is the noun and this is the adjective, so that's how it knows that blue and chair, blue is describing the chair.
And so they need to look for chairs first and then look for the categorization of blue. And in voice queries, right now Google Home and Google Assistant work in ten languages that just so happen to be the ten languages that are covered in that cloud natural language API, but Google has said that they're gonna roll out thirty two total languages in the next year.
So the cloud natural language API is gonna get much smarter, much more capable of this translation because it's building on the understanding that it has from these core languages. So when you search in one of the ten languages in a Google Home or Google Now search, Google Now understands that language natively without having to translate it, it's got it, right?
Google is now fluent in French. What happens is it's not fluent in the other languages, so it has to do workarounds. But Google Maps has already announced that they're gonna launch in thirty nine new languages. What we see, at least right now, is that Google is using Google Maps as the Trojan horse, the testing for a lot of this new entity understanding that they're doing because it's especially important for searches where you might not speak a language.
So let's say I'm in Germany and I wanna go to a supermarket because I wanna buy some food for my hotel room. I don't know how to say supermarket in German and it's an extra step for me to go to Google Translate to search maps for supermarket, but if German supermarkets didn't put the English word they might not come up.
That's a horrible experience. So now Google has consolidated the language understanding, they now have pictures where you can click on groceries. I don't have to speak German to get the right keyword because Google understands that groceries has a German translation and it knows what it is, so it does that work for me and shows me grocery stores even without that are in Germany near where I am even without me speaking German.
That's entity understanding. That's not relying on me to know the language or the right keyword. And Google's really doubled down in terms of allowing businesses to use maps as a marketing platform. They've launched Google Posts as part of Google My Business, which lets you put ads in your map listing, and that's gonna be really powerful.
They've also started translating all the reviews of your local business into whatever language your phone setting has. So if your phone is set to German, then when you search for a restaurant, even if the original review is written in English it's gonna translate it to German for you because it knows that English is not worth you reading.
All you get is the stars unless you have put it into another translation software. So Google's doing that for you. A lot of the new stuff is happening in maps. The other thing that's happening is Google's enabling people to claim their personal brand identity or their personal entity knowledge graph entry.
You can see when I search for my name it says do you want Cindy Crum the author or Cindy Crum the business owner? Because it thinks we might be two different people. We're the same people. So it's trying to disambiguate because it's still learning about who I am, but on the right, emergency vets, that's a TV show, you can see when you click in the triple dot you can get verified to manage this entity understanding, right, because they're like this one doesn't have a website, we need someone to help us understand who this is, what it is.
So with language, Google when it understands the language, this is really complex, don't worry about it, but what it says essentially is that if Google speaks the language natively then it's gonna do entity understanding natively, but if it doesn't speak the language natively it still does entity understanding by translating the query to English first and then looking for a match in the knowledge graph that way.
So it's super smart. So why is this important? Entities are the presentation layer of the internet or that's what Google wants. Actually Google wants entities to be the transaction layer of the internet because they're better for a voice query. If you think about a knowledge graph entry, one of the first things that'll happen is disambiguation.
Disambiguation is the process of Google saying, hey when you searched for Monty Python did you mean Monty Python the group or Monty Python the movie? Because they're related, but they're slightly different. They're two different entities, and so in voice it can disambiguate verbally by giving you multiple choice options.
Did you want a or b? And so you see this coming out visually in search, but it's something that translates really well to an audio voice interaction. You also have the link to the knowledge graph. You have the entity understanding and the classification, so this is the comedy group.
So within the comedy group they have songs and albums and overview information, and then they link over to the top entity domain, the domain that's related to this query, the most official domain. So we're used to Google linking to websites, right? We see it in knowledge graph in business, that happens all the time.
What's new, the big change, is that they're starting to link to deep pages on websites within knowledge graph and within the entity understanding. So this is from the World Cup. When you use Knowledge Graph to find out about a particular match, they would link you to the match preview on the FIFA site, and it's not the homepage, it's a deep link into the site.
That's new. That's Google getting in front of your website clicks to lift the information and say if you still want to drill down and go more, here's where we found this. It's an additional layer so that Google can be the presentation layer. So this is a map result where Google has included, has injected a search result related to my query into the map result.
Again, is getting in front of the web results to be the presentation layer. So why would they do this? Like, it's hard to come to terms with, but Google's a business and they have to make money. And so they see this as a way to one, enable voice search in a very broad way, but two, a way to monetize.
So this is a branded map pack for Target. Now if you search for certain brands they have a brand logo and it only shows you entries for that brand. It used to be if you searched for Target you might get other businesses that had the word Target in it, like on Target jobs.
Now it's like when someone searches for Target they probably mean like the main Target, not like these other things that include the word Target, it's much smarter. But have you seen this? Now they have a tab on Target that's like Target sells products, we've listed them here for you.
And when someone buys a product from this screen, Google takes a cut. This is one of Rand's examples, Lego ship in a bottle. This goes down even more granularly to a product, a weird Lego ship in a bottle. Who thought of that? And we have the product ID and we have the official Lego dot com link right there, but it's a knowledge graph for a specific product.
And on the right, what we see is that they're letting people buy this specific product from a lot of different people, not just the canonical original manufacturer. They're cutting us out at the knees if we're lego dot com because who wouldn't buy the new Lego ship in a bottle from eBay?
That's cheaper, why not? So cross device user profiles take entities and they allow them to be more customizable. Not only can they use the language on your phone, but Google is tracking a lot of our information, and this is the creepy slide, but like are all of the places you can search and find out about what you've done on your phone or your computer because all of it, all of your activity when you're logged in is being saved to one account.
And so in this I can search my historical activity on all my devices, it'll tell me what I did on what device. I can search my cloud, if I have a business account it'll search my inbox, my documents, my calendar, all at the same time with one cloud search.
And then I can find my historical locations, and there's a page where you can see all of the places you've been in the past whatever time period, and that's what this looks like in a very detailed way. This is Cindy's day, Cindy's average day when I'm not working.
It even puts the pictures that I took of Berkeley in the place where I took them. So I'm at home and I took two pictures of my sleepy dog, and then I drove somewhere, then I went to ink for coffee, and I went to T.
K. Maxx, which we call T. J. Maxx, and then I went around. And so we've got lots of different things that it's categorizing about me and what it knows about me, but also look at the circular icons to the left of my activities, the coffee cup and the shopping bag and the house.
It's classified the places I went into a group saying this is a place for shopping, this is a place for coffee, this is a personal residence. They're entity classifying the world around me to know what I do and what I like. And then they put the pictures in.
And they're now allowing people to label personal things in maps, so I can label not just home and work, but I can label kids school or something else. And so that way when you call it up on maps in a voice only search you can say how much traffic is there on the way to school?
And it'll get it because you've labeled it. And Google's getting better at understanding different voices within a family, within a context with Google Home, so that the kids can have their own profile or so that Google can correspond their results to what they know that person likes and they can take the queries from one person and associate them with a related Google account so that they can personalize better.
And so when you think about entity understanding and where Google is going, have to understand that non website content exists beside the website content. There are things that are entity related. When you have a top level concept you can have answers, videos, maps, apps, and you can have a website, but they don't have to live underneath each other.
Now they can live laterally. That lateral integration allows Google to say, hey, we know that you searched for Monty Python but we're in your car so we're guessing that you don't want the Monty Python website. Maybe you want to play Monty Python in the speakers, you want to play a song.
Or if you're searching on a web connected TV, then you might want the video. And so they can show things that will work. They're not gonna try and show a Monty Python video on your Google watch. Or they can't legally show the Monty Python video on your Android Auto when you're driving.
So they can now use the disambiguation and the entity understanding to show you the right thing for the right device context and based on who you are historically and what you've liked. And so Google's using this kind of understanding to get in to the entity of IoT, Internet of Things, right, to connect these devices but still have a good intelligent experience.
And remember, personalized experiences are great for Google because they're easier to monetize, they can market to you something that they know you want and they can usually do it with higher margins in all of these different channels that they've developed to monetize it.
So walk away with this understanding. Not everything needs a website. Businesses, people, events, songs, all these different things existed and are great on their own and we've been forcing companies to wrap websites around them but they didn't need it. It was extra, it was just extra work.
It's not helpful. So Google's now trying to do a better job. So expect to see more and more filters so that people can disambiguate by voice only. Expect to see focus on brand and brand entities, and expect to see more focus on non website content like video, podcast, anything that can be done in a voice only context.
Make sure you're using the tools that I showed you, the Market Finder tool and the Natural Language API to understand how your text is being understood by Google, and optimize your non domain entity content, especially the voice stuff. And whenever you hear Google use buzzwords like mobile first, AI first, or voice first, remember, these are all basically the same thing, they're just trying to enable smarter understanding of the world around you.
If you want to learn more about this, I've got a five part article series. There are only four that are published, the fifth one's going out soon. That's a link to it. Also, if you're curious to understand like what my company does, we have a whole tool set where you can see mobile results from anywhere in the world, from a hundred locations at a time on one query because they change, and that's me, that's it.
