Here is a recap of what happened in the search forums today, through the eyes of the Search Engine Roundtable and other search forums on the web.
Search Engine Roundtable Stories:
Other Great Search Forum Threads:
Google noted in their project Owl announcement that they also updated their How Google Search Works site. The site was first introduced in March 2013 and had some content and feature revisions over the years.
It now has been totally redesigned and a lot of the content has been removed or hidden in other places.
Here is the new design:
Here is what it looked like a year ago:
As Gary Illyes said on Twitter, they refreshed the site, but the site itself is not new.
@iqseo That's a few years old, it was just refreshed recently— Gary Illyes á( á )á- (@methode) April 25, 2017
When they first introduced it, they had a way to see examples of pages removed from the search results because of spam. So I built an index of those pages and then got slapped from Google for doing so. Can you blame me? Anyway - the good old days.
Forum discussion at Twitter.
Back in 2013, Matt Cutts of Google was clear that having a patent does not mean they use the patent material in their live search ranking algorithm. In fact, I believe most or many are not used and the research goes into a library somewhere. That doesn't mean you can't learn from patents - you can.
But to say Google's ranking algorithm does X because X it mentioned in a specific patent or research paper is simply not accurate.
Gary Illyes from Google reminded us of that fact on Twitter the other day. He said "we do lots of research and some stuff never gets used for reasons." Some stuff in theory sound good, but in practice, simply doesn't work. I'd love to know the percentage, but I suspect it is high.
Forum discussion at Twitter.
Did you know that Google is willing to review random SEO agencies to see if their SEO practices are fair and within their guidelines? I didn't think Google spent the time doing that. Year, they have both algorithmic penalties and manual actions but to review an SEO agency specifically?
There are less than a handful of times I can remember Google ever going after a specific agency and their clients. Google has gone after link networks but specific SEO agencies - not that often from what I can tell. But who knows, Google doesn't tell me everything - far from it.
Here is a Twitter exchange with someone who wants to out an SEO for shaddy SEO techniques and Gary Illyes of Google:
In short, here is Gary Illyes asking for the information so someone at Google can dig deeper. How is that for competitive sabotage?
Now, Gary is going to kill me because his inbox is going to be flooded with spam reports.
Forum discussion at Twitter.
In Google Search Console, there is a feature under the Fetch as Google tool to request indexing. When you click that, it let's you select either "crawl only this URL" or "crawl this URL and its direct links." Someone on Twitter asked John Mueller how many direct links would Google crawl from that page that you are requesting indexing on? John responded on Twitter saying to keep the links on that page to a minimum because the tool is not a replacement for the Sitemap file.
Here is the tweet:
@_sebastiansimon I'd limit the number of links on a page to a reasonable amount. That tool isn't meant to replace a sitemap file.— John â.o(â§â-½â¦)o.â (@JohnMu) April 25, 2017
What is a "reasonable amount" of links for a page? Who knows - it depends on the page. Sites like Techmeme, which I find super useful, have thousands of links on one page.
Forum discussion at Twitter.
Recently rank checker started hanging on some search queries & the button on the SEO Toolbar which launched rank checker stopped working. Both of these issues should now be fixed if you update your Firefox extensions.
If ever the toolbar button doesn't work one can enable the Menu bar in Firefox, then go under the tools menu to the rank checker section to open it.
Years ago we created a new logo for rank checker which we finally got around to changing it today. :)
Last August I wrote a blog post about how attention merchants were sucking the value out of online publishing. In it I noted how the Yahoo! Directory disappeared & how even DMOZ saw a sharp drop in traffic & rankings over the past few years.
The concept of a neutral web is dead. In its place is agenda-driven media.
As the tech oligarchs broadly defund publishing, the publishers still need to eat. Aggregate information quality declines to make the numbers work. Companies which see their ad revenues slide 20%, 30% or 40% year after year can't justify maintaining the labor-intensive yet unmonetized side projects.
There is Wikipedia, but it is not without bias & beyond the value expressed in the hidden bias most of the remaining value from it flows on through to the attention merchant / audience aggregation / content scraper platforms.
Last month DMOZ announced they were closing on March 14th without much fanfare. And on March 17th the directory went offline.
A number of people have pushed to preserve & archive the DMOZ data. Some existing DMOZ editors are planning on launching a new directory under a different name but as of the 17th DMOZ editors put up a copy at dmoztools.net. Jim Boykin scraped DMOZ & uploaded a copy here. A couple other versions of DMOZ have been published at OpenDirectoryProject.org & Freemoz.org.
Although site policies suggest that an individual site should be submitted to only one category, as of October 2007, Topix.com, a news aggregation site operated by DMOZ founder Rich Skrenta, has more than 17,000 listings.
Early in the history of DMOZ, its staff gave representatives of selected companies, such as Rolling Stone or CNN, editing access in order to list individual pages from their websites. Links to individual CNN articles were added until 2004, but were entirely removed from the directory in January 2008 due to the content being outdated and not considered worth the effort to maintain.
but by-and-large it added value to the structure of the web.
As search has advanced (algorithmic evolution, economic power, influence over publishers, enhanced bundling of distribution & user tracking) general web directories haven't been able to keep pace. Ultimately the web is a web of links & pages rather than a web of sites. Many great sites span multiple categories. Every large quality site has some misinformation on it. Every well-known interactive site has some great user contributions & user generated spam on it. Search engines have better signals about what pages are important & which pages have maintained importance over time. As search engines have improved link filtering algorithms & better incorporated user tracking in rankings, broad-based manual web directories had no chance.
The web of pages vs web of sites concept can be easily observed in how some of the early successful content platforms have broken down their broad-based content portals into a variety of niche sites.
When links were (roughly) all that mattered, leveraging a website's link authority meant it was far more profitable for a large entity to keep publishing more content on the one main site. That is how eHow became the core of a multi-billion Dollar company.
Demand Media showed other publishers the way. And if the other existing sites were to stay competitive, they also had to water down content quality to make the numbers back out. The problem with this was the glut of content was lower ad rates. And the decline in ad rates was coupled with a shift away from a links-only view of search relevancy to a model based on weighting link profiles against user engagement metrics.
Websites with lots of links, lots of thin content & terrible engagement metrics were hit.
Kristen Moore, vp of marketing for Demand Media, explained what drove the most egregious aspects of eHow's editorial strategy: “There’s some not very bright people out there.”
eHow improved their site design, drastically reduced their ad density, removed millions of articles from their site, and waited. However nothing they did on that domain name was ever going to work. They dug too deep of a hole selling the growth story to pump a multi-billion Dollar valuation. And they generated so much animosity from journalists who felt overwork & underpaid that even when they did rank journalists would typically prefer to link to anything but them.
The flip side of that story is the newspaper chains, which rushed to partner with Demand Media to build eHow-inspired sections on their sites.
Brands which enjoy the Google brand subsidy are also quite hip to work with Demand Media, which breathes new life into once retired content: "Sometimes Demand will even dust off old content that’s been published but is no longer live and repurpose it for a brand."
As Facebook & Google grew more dominant in the online ad ecosystem they aggressively moved to suck in publisher content & shift advertiser spend onto their core properties. The rise of time spent on social sites only made it harder for websites to be sought out destination. Google also effectively cut off direct distribution by consolidating & de-monetizing the RSS reader space then shutting down a project they easily could have left run.
As the web got more competitive, bloggers & niche publications which were deeply specialized were able to steal marketshare in key verticals by leveraging a differentiated editorial opinion.
Even if they couldn't necessarily afford to build strong brands via advertising, they were worthy of a follow on some social media channels & perhaps an email subscription. And the best niche editorial remains worthy of a direct visit:
Everything about Techmeme and its lingering success seems to defy the contemporary wisdom of building a popular website. It publishes zero original reporting and is not a social network. It doesn’t have a mobile app or a newsletter or even much of a social presence beyond its Twitter account, which posts dry commodity news with zero flair for clickability.
As a work around to the Panda hits, sites like eHow are now becoming collections of niche-focused sites (Cuteness.com, Techwalla.com, Sapling.com, Leaf.tv, etc will join Livestrong.com & eHow.com). It appears to be working so far...
...but they may only be 1 Panda update away from finding out the new model isn't sustainable either.
About.com has done the same thing (TheSpruce.com, Verywell.com, Lifewire.com, TheBalance.com). Hundreds of millions of Dollars are riding on the hope that as the algorithms keep getting more granular they won't discover moving the content to niche brands wasn't enough.
As content moves around search engines with billions of Dollars in revenue can recalibrate rankings for each page & adjust rankings based on user experience. Did an influential "how to" guide become irrelevant after a software or hardware update? If so, they can see it didn't solve the user's problem and rank a more recent document which reflects the current software or hardware. Is a problem easy to solve with a short snippet of content? If so, that can get scraped into the search results.
Web directories which are built around sites rather than pages have no chance of competing against the billions of Dollars of monthly search ads & the full cycle user tracking search companies like Google & Bing can do with their integrated search engines, ad networks, web browsers & operating systems.
Arguably in most cases the idea of neutral-based publishing no longer works on the modern web. The shill gets exclusive stories. The political polemic gets automatic retweets from those who identify. The content which lacks agenda probably lacks the economics to pay for ads & buy distribution unless people can tell the creator loves what they do so much it influences them enough to repeatedly visit & perhaps pay for access.
There may be a couple exceptions which prove the rule, but new TLDs are generally an awful investment for everyone except the registry operator.
Here is the short version...
Imagine registering a domain for $10, building a business on it, and then learning the renewal fee will increase to hundreds of $ a year.— Elliot Silver (@DInvesting) March 7, 2017
And the long version...
About a half-decade ago I wrote about how Google devalued domain names from an SEO perspective & there have been a number of leading "category killer" domains which have repeatedly been recycled from startup to acquisition to shut down to PPC park page to buy now for this once in a lifetime opportunity in an endless water cycle.
The central web platforms are becoming ad heavy, which in turn decreases the reach of anything which is not an advertisement. For the most valuable concepts / markets / keywords ads eat up the entire interface for the first screen full of results. Key markets like hotels might get a second round of vertical ads to further displace the concept of organic results.
It isn't just gTLD's that are stalled. ALL extensions are stalling. The demand by END USERS in 2017 is not what it was years ago. #Domains— Rick Schwartz (@DomainKing) March 1, 2017
The tech monopolies can only make so much money by stuffing ads onto their own platform. To keep increasing their take they need to increase the types, varieties & formats of media they host and control & keep the attention on their platform.
Both Google & Facebook are promoting scams where they feed on desperate publishers & suck a copy of the publisher's content into being hosted by the tech monopoly platform de jour & sprinkle a share of the revenues back to the content sources.
They may even pay a bit upfront for new content formats, but then after the market is primed the deal shifts to where (once again) almost nobody other than the tech monopoly platform wins.
The attempt to "own" the web & never let users go is so extreme both companies will make up bogus statistics to promote their proprietary / fake open / actually closed standards.
If you ignore how Google's AMP double, triple, or quadruple counts visitors in Google Analytics the visit numbers look appealing.
But the flip side of those fake metrics is actual revenues do not flow.
Facebook has the same sort of issues, with frequently needing to restate various metrics while partners fly blind.
Have you tried Angry Birds lately? It’s a swamp of dark patterns. All extractive logic meant to trick you into another in-app payment. It’s the perfect example of what happens when product managers have to squeeze ever-more-growth out of ever-less-fertile lands to hit their targets year after year. ... back to the incentives. It’s not just those infused by venture capital timelines and return requirements, but also the likes of tax incentives favoring capital gains over income. ... that’s the truly insidious part of the tech lords solution to everything. This fantasy that they will be greeted as liberators. When the new boss is really a lot like the old boss, except the big stick is replaced with the big algorithm. Depersonalizing all punishment but doling it out just the same. ... this new world order is being driven by a tiny cabal of monopolies. So commercial dissent is near impossible. ... competition is for the little people. Pitting one individual contractor against another in a race to the bottom. Hoarding all the bargaining power at the top. Disparaging any attempts against those at the bottom to organize with unions or otherwise.
To be a success on the attention platforms you have to push toward the edges. But as you become successful you become a target.
And the dehumanized "algorithm" is not above politics & public relations.
Pewdiepie is the biggest success story on the YouTube platform. When he made a video showing some of the absurd aspects of Fiverr it led to a WSJ investigation which "uncovered" a pattern of anti-semitism. And yet one of the reporters who worked on that story wrote far more offensive and anti-semetic tweets. The hypocrisy of the hit job didn't matter. They still were able to go after Pewdiepie's ad relationships to cut him off from Disney's Maker Studios & the premium tier of YouTube ads.
The fact that he is an individual with broad reach means he'll still be fine economically, but many other publishers would quickly end up in a death spiral from the above sequence.
If it can happen to a leading player in a closed ecosystem then the risk to smaller players is even greater.
In some emerging markets Facebook effectively *is* the Internet.
Domains have been so devalued (from an SEO perspective) that some names like PaydayLoans.net sell for about $3,000 at auction.
$3,000 can sound like a lot to someone with no money, but names like that were going for 6 figures at their peak.
Professional domain sellers participate in the domain auctions on sites like NameJet & SnapNames. Big keywords like [payday loans] in core trusted extensions are not missed. So if the 98% decline in price were an anomaly, at least one of them would have bid more in that auction.
Why did exact match domains fall so hard? In part because Google shifted from scoring the web based on links to considering things like brand awareness in rankings. And it is very hard to run a large brand-oriented ad campaign promoting a generically descriptive domain name. Sure there are a few exceptions like Cars.com & Hotels.com, but if you watch much TV you'll see a lot more ads associated with businesses that are not built on generically descriptive domain names.
Not all domains have fallen quite that hard in price, but the more into the tail you go the less the domain acts as a memorable differentiator. If the barrier to entry increases, then the justification for spending a lot on a domain name as part of a go to market strategy makes less sense.
Arguably EMDs have lost more value than brandable domain names, but even brandable names have sharply slid.
If you go back a decade or two tech startups would secure their name (say Snap.com or Monster.com or such) & then try to build a business on it.
But in the current marketplace with there being many paths to market, some startups don't even have a domain name at launch, but begin as iPhone or Android apps.
Now people try to create success on a good enough, but cheap domain name & then as success comes they buy a better domain name.
As long as domain redirects work, there's no reason to spend heavily on a domain name for a highly speculative new project.
Rather then spending 6 figures on a domain name & then seeing if there is market fit, it is far more common to launch a site on something like getapp.com, joinapp.com, app.io, app.co, businessnameapp.com, etc.
This in turn means that rather than 10,000s of startups all chasing their core .com domain name off the start, people test whatever is good enough & priced close to $10. Then only after they are successful do they try to upgrade to better, more memorable & far more expensive domain names.
Money isn't spent on the domain names until the project has already shown market fit.
One in a thousand startups spending $1 million is less than one in three startups spending $100,000.
Some of the companies which are registries for new TLDs talk up investing in marketing & differentiation for the new TLDs, but very few of them are doing much on the marketing front.
You may see their banner ads on domainer blogs & they may even pay for placement with some of the registries, but there isn't much going on in terms of cultivating a stable ecosystem.
When Google or Facebook try to enter & dominate a new vertical, the end destination may be extractive rent seeking by a monopoly BUT off the start they are at least willing to shoulder some of the risk & cost upfront to try to build awareness.
Where are the domain registries who have built successful new businesses on some of their new TLDs? Where are the subsidies offered to key talent to help drive awareness & promote the new strings?
As far as I know, none of that stuff exists.
In fact, what is prevalent is the exact opposite.
So many of them are short sighted greed-based plays that they do the exact opposite of building an ecosystem ... they hold back any domain which potentially might not be complete garbage so they can juice it for a premium ask price in the 10s of thousands of dollars.
While searching on GoDaddy Auctions for a client project I have seen new TLDs like .link listed for sale for MORE THAN the asking price of similar .org names.
If those prices had any sort of legitimate foundation then the person asking $30,000 for a .link would have bulk bought all the equivalent .net and .org names which are listed for cheaper prices.
But the prices are based on fantasy & almost nobody is dumb enough to pay those sorts of prices.
Anyone dumb enough to pay that would be better off buying their own registry rather than a single name.
The holding back of names is the exact opposite of savvy marketing investment. It means there's no reason to use the new TLD if you either have to pay through the nose or use a really crappy name nobody will remember.
I didn’t buy more than 15 of Uniregistry’s domains because all names were reserved in the first place and I didn’t feel like buying 2nd tier domains ... Domainers were angry when the first 2 Uniregistry’s New gTLDs (.sexy and .tattoo) came out and all remotely good names were reserved despite Frank saying that Uniregistry would not reserve any domains.
Who defeats the race to the bottom aspects of the web by starting off from a "we only sell shit" standpoint?
And that's why these new TLDs are a zero.
Many online verticals are driven by winner take most monopoly economics. There's a clear dominant leader in each of these core markets: social, search, short-form video, long-form video, retail, auctions, real estate, job search, classifieds, etc. Some other core markets have consolidated down to 3 or 4 core players who among them own about 50 different brands that attack different parts of the market.
Almost all the category leading businesses which dominate aggregate usage are on .com domains.
Contrast the lack of marketing for new TLDs with all the marketing one sees for the .com domain name.
Local country code domain names & .com are not going anywhere. And both .org and .net are widely used & unlikely to face extreme price increases.
A decade ago domainers were frustrated Verisign increased the price of .com domains in ~ 5% increments:
Every mom, every pop, every company that holds a domain name had no say in the matter. ICANN basically said to Verisign: "We agree to let you hose the masses if you stop suing us".
I don't necessarily mind paying more for domains so much as I mind the money going to a monopolistic regulator which has historically had little regard for the registrants/registrars it should be serving
Those 5% or 10% shifts were considered "hosing the masses."
Imagine what sort of blowback PIR would get from influential charities if they tried to increase the price of .org domains 30-fold overnight. It would be such a public relations disaster it would never be considered.
Domain registries are not particularly expensive to run. A person who has a number of them can run each of them for less than the cost of a full time employee - say $25,000 to $50,00 per year.
And yet, the very people who complained about Verisign's benign price increases, monopolistic abuses & rent extraction are now pushing massive price hikes:
.Hosting and .juegos are going up from about $10-$20 retail to about $300. Other domains will also see price increases.
Here's the thing with new TLD pricing: registry operators can increase prices as much as they want with just six months' notice.
in its applications, Uniregistry said it planned to enter into a contractual agreement to not increase its prices for five years.
Why would anyone want to build a commercial enterprise (or anything they care about) on such a shoddy foundation?
If a person promises...
That's 3 strikes and the batter is out.
The claim the new TLDs need more revenues to exist are untrue. Running an extension costs maybe $50,000 per year. If a registry operator wanted to build a vibrant & stable ecosystem the first step would be dumping the concept of premium domains to encourage wide usage & adoption.
There are hundreds of these new TLD extensions and almost none of them can be trusted to be a wise investment when compared against similar names in established extensions like .com, .net, .org & CCTLDs like .co.uk or .fr.
There's no renewal price protection & there's no need, especially as prices on the core TLDs have sharply come down.
Aggregate stats are somewhat hard to come by as many deals are not reported publicly & many sites which aggregate sales data also list minimum prices.
However domains have lost value for many reasons
All the above are the mechanics of "why" prices have been dropping, but it is also worth noting many of the leading portfolios have been sold.
If the domain aftermarket is as vibrant as some people claim, there's no way the Marchex portfolio of 200,000+ domains would have sold for only $28.1 million a couple years ago.
RegistrarStats shows .com registrations have stopped growing & other extensions like .net, .org, .biz & .info are now shrinking.
Both aftermarket domain prices & the pool of registered domains on established gTLDs are dropping.
I know I've dropped hundreds & hundreds of domains over the past year. That might be due to my cynical views of the market, but I did hold many names for a decade or more.
As barrier to entry increases, many of the legacy domains which could have one day been worth developing have lost much of their value.
And the picked over new TLDs are an even worse investment due to the near infinite downside potential of price hikes, registries outright folding, etc.
Most of the registration graphs for new TLDs are far uglier than the one posted above. China will not save the new gTLDs.
Into this face of declining value there is a rush of oversupply WITH irrational above-market pricing. And then the registries which spend next to nothing on marketing can't understand why their great new namespaces went nowhere.
Any baggage they may carry is less than the risk of going with an unproven new extension without any protections whatsoever.
Who really loses is anyone who read what these domain registry operators wrote & trusted them.
Uniregistry does not believe that registry fees should rise when the costs of other technology services have uniformly trended downward, simply because a registry operator believes it can extract higher profit from its base of registrants.
How does one justify a 3000% price hike after stating "Our prices are fixed and only indexed to inflation after 5 years."
Are they pricing these names in Zimbabwe Dollars? Or did they just change their minds in a way that hurt anyone who trusted them & invested in their ecosystem?
Frank Schilling warned about the dangers of lifting price controls
The combination of "presumptive renewal" and the "lifting of price controls on registry services" is incredibly dangerous.
Imagine buying a home, taking on a large mortgage, remodeling, moving in, only to be informed 6 months later that your property taxes will go up 10,000% with no better services offered by local government. The government doesn't care if you can't pay your tax/mortgage because they don't really want you to pay your tax… they want you to abandon your home so they can take your property and resell it to a higher payer for more money, pocketing the difference themselves, leaving you with nothing.
This agreement as written leaves the door open to exactly that type of scenario
He didn't believe the practice to be poor.
Rather he felt he would have been made poorer, unless he was the person doing it:
It would be the mother of all Internet tragedies and a crippling blow to ICANN’s relevance if millions of pioneering registrants were taxed out of their internet homes as a result of the greed of one registry and the benign neglect, apathy or tacit support of its master.
It is a highly nuanced position.
Imagine registering a domain for $10, building a business on it, and then learning the renewal fee will increase to hundreds of $ a year.— Elliot Silver (@DInvesting) March 7, 2017
Update: Shortly after the sharp pricing increases were announced GoDaddy dropped Uniregistry domain names.
When Compete.com launched with credits-based pricing well over a decade ago I felt like a kid in a candy store using their competitive research tool. Recently Compete.com announced they were shutting down, but many of the link analysis & competitive research tools which leverage scraping have also started licensing clickstream data from sources like Clickstre.am & JumpShot.
These sorts of features add a lot of value to traditional keyword tools, as they can highlight the CTR on ads vs organic results & show if people click on anything after they search for a particular term.
When I read Ahref's recent blog post about integrating clickstream data I got that same kid in a candy store feeling I got when I first used Compete. Some highlights...
As an example of how the searches with clicks feature is helpful, consider Google's recently announced RGB conversion feature
In that image you can see how the feature displaces the result set.
What's cool about the Ahrefs feature is you can also see what sort of impact that feature has on click volumes.
After 1 month, 20% of the searches for [RGB to HEX] no longer had any clicks to an external website.
On the second month it looks like the "no click" rate was closer to 7%, so perhaps some of the initial additional search volume was driven by people searching for the related keywords after blogs covered the new feature.
But the nice thing about the feature is you can see how the click rate changes over time as the feature evolves.
In some areas like weather Google ends up dominating most the user behavior with their in-SERP feature.
About half of all weather keyword searches do not click on any listings. And then of those which do click, about 20% of people click on an ad.
That means the potential organic click volume for that keyword is only about 40% of the initial search volume estimates.
Search results keep getting more interactive features & some of them appear to be click black holes. Literally...
You guys, I've discovered a SERP black hole! I'm on #200 suggested PAA for this SERP?! Has anyone else seen an infinite PAA SERP before? pic.twitter.com/YgZDVWdWJ9— Britney Muller (@BritneyMuller) November 23, 2016
Here is a new item comparison feature table.
As more of the value chain appears in the search results, more of the value chain which formerly appeared on websites disappears. This is true from a wide range of aspects including ad sales, content hosting, ad blocking & brand value.
No click into the publisher's site means no ad revenue for the publisher. Voice search will only accelerate the declines seen from mobile, which shifted user attention away from large screens with many listings to smaller screens with fewer listings & a far higher ad ratio in the search results.
Google has already pushed hard to make hotel searches a pay-to-play vertical & yet some publishers are adopting AMP formatting in that vertical. Google is also forcing AMP down publisher's throats in other verticals like recipes.
If central ad networks host your content then they get better user data for your content than you do as a publisher.
Increased user tracking depresses premium ad sales & moves value from niche players to broad networks "Whether it’s a third party like Facebook or Google tracking across the web or an ISP leveraging its distribution arm, this is outside of consumer expectations. Importantly to the digital media industry, it also devalues the context and relationship of consumer trust which drives the businesses of premium publishers."
Some large sites like Google or Facebook either pay ad blockers or technically work around them within their apps. By funding ad blockers exempting the search result page from having their ads blocked, Google is ultimately defunding competing ad networks.
As search results get noisier & more ad heavy, Google is trying to coerce brands into re-buying their pre-existing brand equity. These efforts are effective, as on some branded & navigational searches over half the click volume goes to the ads. Here are a few examples from Ahrefs. The orange bar shows what percent of the SERP clicks are on ads.
And the above doesn't even account for...
So Google is eating the generic terms, the brand terms, and the search query pool more broadly.
There's a reason Google's online travel business is over twice the size of anyone else & has their biggest advertisers seeking more sustainable & more legitimate alternatives.
The biggest travel players are accustomed to Google’s moves and trying their best to adjust and work around them. Missing from this story is the fact that Google’s latest moves are making it nearly impossible for all but the smallest number of consumer travel startups to succeed. — Dennis Schaal
And some of the aggressive stuff carries over into other lines of business outside of travel. Google is also testing large image extensions on AdWords ads on cell phones that don't leave room for even a second AdWords listing on the screen. When one invests in brand they have to start thinking about how much they are willing to pay Google as an ongoing tithing for their success. Look at the following ads where a competitor bidding on a competing brand drives the brand owner's official site below the fold.
Google is willing to make their results worse (to the point they would consider something that looked like their search result page as an ad-heavy doorway redirect page of spam if hosted by anyone other than themselves) in order to monetize navigational searches.
What's more, you can't just opt out & ignore. When brands make agreements to not cross-bid Google has the FTC sue them.
On some high end fashion brands Google lists shopping ads which lead to third party sellers who sell used goods. Quite often counterfeits will also be in the mix. When the counterfeits are destroyed in the first wash, it is the brand owner who was took to the cleaners.
But there's a solution to that... they can pay Google ever-increasing protection.
Remember the whole shtick about good, legitimate, high-quality content being created for readers without concern for search engines - even as though search engines do not exist?
Whatever happened to that?
We quickly shifted from the above "ideology" to this:
The red triangle/exclamation point icon was arrived at after the Chrome team commissioned research around the world to figure out which symbols alarmed users the most.
Google is explicitly spreading the message that they are doing testing on how to create maximum fear to try to manipulate & coerce the ecosystem to suit their needs & wants.
At the same time, the Google AMP project is being used as the foundation of effective phishing campaigns.
Scare users off of using HTTP sites AND host phishing campaigns.
Killer job Google.
Someone deserves a raise & some stock options. Unfortunately that person is in the PR team, not the product team.
I'd like to tell you that I was preparing the launch of https://amp.secured.mobile.seobook.com but awareness of past ecosystem shifts makes me unwilling to make that move.
I see it as arbitrary hoop jumping not worth the pain.
If you are an undifferentiated publisher without much in the way of original thought, then jumping through the hoops make sense. But if you deeply care about a topic and put a lot of effort into knowing it well, there's no reason to do the arbitrary hoop jumping.
Remember how mobilegeddon was going to be the biggest thing ever? Well I never updated our site layout here & we still outrank a company which raised & spent 10s of millions of dollars for core industry terms like [seo tools].
Though it is also worth noting that after factoring in increased ad load with small screen sizes & the scrape graph featured answer stuff, a #1 ranking no longer gets it done, as we are well below the fold on mobile.
In the above example I am not complaining about ranking #5 and wishing I ranked #2, but rather stating that ranking #1 organically has little to no actual value when it is a couple screens down the page.
Google indicated their interstitial penalty might apply to pop ups that appear on scroll, yet Google welcomes itself to installing a toxic enhanced version of the Diggbar at the top of AMP pages, which persistently eats 15% of the screen & can't be dismissed. An attempt to dismiss the bar leads the person back to Google to click on another listing other than your site.
As bad as I may have made mobile search results appear earlier, I was perhaps being a little too kind. Google doesn't even have mass adoption of AMP yet & they already have 4 AdWords ads in their mobile search results AND when you scroll down the page they are testing an ugly "back to top" button which outright blocks a user's view of the organic search results.
What happens when Google suggests what people should read next as an overlay on your content & sells that as an ad unit where if you're lucky you get a tiny taste of the revenues?
Is it worth doing anything that makes your desktop website worse in an attempt to try to rank a little higher on mobile devices?
Given the small screen size of phones & the heavy ad load, the answer is no.
I realize that optimizing a site design for mobile or desktop is not mutually exclusive. But it is an issue we will revisit later on in this post.
Many people new to SEO likely don't remember the importance of using Google Checkout integration to lower AdWords ad pricing.
You either supported Google Checkout & got about a 10% CTR lift (& thus 10% reduction in click cost) or you failed to adopt it and got priced out of the market on the margin difference.
And if you chose to adopt it, the bad news was you were then spending yet again to undo it when the service was no longer worth running for Google.
How about when Google first started hyping HTTPS & publishers using AdSense saw their ad revenue crash because the ads were no longer anywhere near as relevant.
Not like Google cared much, as it is their goal to shift as much of the ad spend as they can onto Google.com & YouTube.
Google is now testing product ads on YouTube.
It is not an accident that Google funds an ad blocker which allows ads to stream through on Google.com while leaving ads blocked across the rest of the web.
Android Pay might be worth integrating. But then it also might go away.
It could be like Google's authorship. Hugely important & yet utterly trivial.
Faces help people trust the content.
Then they are distracting visual clutter that need expunged.
Then they once again re-appear but ONLY on the Google Home Service ad units.
They were once again good for users!!!
Neat how that works.
Or it could be like Google Reader. A free service which defunded all competing products & then was shut down because it didn't have a legitimate business model due to it being built explicitly to prevent competition. With the death of Google reader many blogs also slid into irrelevancy.
Their FeedBurner acquisition was icing on the cake.
Techdirt is known for generally being pro-Google & they recently summed up FeedBurner nicely:
Thanks, Google, For Fucking Over A Bunch Of Media Websites - Mike Masnick
Ultimately Google is a horrible business partner.
And they are an even worse one if there is no formal contract.
When Google routinely acts so anti-competitive & abusive it is no surprise that some of the "standards" they propose go nowhere.
You can only get screwed so many times before you adopt a spirit of ambivalence to the avarice.
Google is the type of "partner" that conducts security opposition research on their leading distribution partner, while conveniently ignoring nearly a billion OTHER Android phones with existing security issues that Google can't be bothered with patching.
Deliberately screwing direct business partners is far worse than coding algorithms which belligerently penalize some competing services all the while ignoring that the payday loan shop funded by Google leverages doorway pages.
BackChannel recently published an article foaming at the mouth promoting the excitement of Google's AI:
This 2016-to-2017 Transition is going to move us from systems that are explicitly taught to ones that implicitly learn." ... the engineers might make up a rule to test against—for instance, that “usual” might mean a place within a 10-minute drive that you visited three times in the last six months. “It almost doesn’t matter what it is — just make up some rule,” says Huffman. “The machine learning starts after that.
The part of the article I found most interesting was the following bit:
After three years, Google had a sufficient supply of phonemes that it could begin doing things like voice dictation. So it discontinued the [phone information] service.
Google launches "free" services with an ulterior data motive & then when it suits their needs, they'll shut it off and leave users in the cold.
As Google keeps advancing their AI, what do you think happens to your AMP content they are hosting? How much do they squeeze down on your payout percentage on those pages? How long until the AI is used to recap / rewrite content? What ad revenue do you get when Google offers voice answers pulled from your content but sends you no visitor?
A recent Wall Street Journal article highlighting the fast ad revenue growth at Google & Facebook also mentioned how the broader online advertising ecosystem was doing:
Facebook and Google together garnered 68% of spending on U.S. online advertising in the second quarter—accounting for all the growth, Mr. Wieser said. When excluding those two companies, revenue generated by other players in the U.S. digital ad market shrank 5%
The issue is NOT that online advertising has stalled, but rather that Google & Facebook have choked off their partners from tasting any of the revenue growth. This problem will only get worse as mobile grows to a larger share of total online advertising:
By 2018, nearly three-quarters of Google’s net ad revenues worldwide will come from mobile internet ad placements. - eMarketer
Media companies keep trusting these platforms with greater influence over their business & these platforms keep screwing those same businesses repeatedly.
You pay to get likes, but that is no longer enough as edgerank declines. Thanks for adopting Instant Articles, but users would rather see live videos & read posts from their friends. You are welcome to pay once again to advertise to the following you already built. The bigger your audience, the more we will charge you! Oh, and your direct competitors can use people liking your business as an ad targeting group.
Worse yet, Facebook & Google are even partnering on core Internet infrastructure.
Any hope of AMP turning the corner on the revenue front is a "no go":
“We want to drive the ecosystem forward, but obviously these things don’t happen overnight,” Mr. Gingras said. “The objective of AMP is to have it drive more revenue for publishers than non-AMP pages. We’re not there yet”.
Publishers who are critical of AMP were reluctant to speak publicly about their frustrations, or to remove their AMP content. One executive said he would not comment on the record for fear that Google might “turn some knob that hurts the company.”
Look at that.
Leadership through fear once again.
At least they are consistent.
As more publishers adopt AMP, each publisher in the program will get a smaller share of the overall pie.
Just look at Google's quarterly results for their current partners. They keep showing Google growing their ad clicks at 20% to 40% while partners oscillate between -15% and +5% quarter after quarter, year after year.
In the past quarter Google grew their ad clicks 42% YoY by pushing a bunch of YouTube auto play video ads, faster search growth in third world markets with cheaper ad prices, driving a bunch of lower quality mobile search ad clicks (with 3 then 4 ads on mobile) & increasing the percent of ad clicks on "own brand" terms (while sending the FTC after anyone who agrees to not cross bid on competitor's brands).
The lower quality video ads & mobile ads in turn drove their average CPC on their sites down 13% YoY.
The partner network is relatively squeezed out on mobile, which makes it shocking to see the partner CPC off more than core Google, with a 14% YoY decline.
What ends up happening is eventually the media outlets get sufficiently defunded to where they are sold for a song to a tech company or an executive at a tech company. Alibaba buying SCMP is akin to Jeff Bezos buying The Washington Post.
The Wall Street Journal recently laid off reporters. The New York Times announced they were cutting back local cultural & crime coverage.
If news organizations of that caliber can't get the numbers to work then the system has failed.
The Tribune Company, already through bankruptcy & perhaps the dumbest of the lot, plans to publish thousands of AI assisted auto-play videos in their articles every day. That will guarantee their user experience on their owned & operated sites is worse than just about anywhere else their content gets distributed to, which in turn means they are not only competing against themselves but they are making their own site absolutely redundant & a chore to use.
That the Denver Guardian (an utterly fake paper running fearmongering false stories) goes viral is just icing on the cake.
many Facebook users wish to connect with people and things that confirm their pre-existing opinions, whether or not they are true. ... Giving people what they want to see will always draw more attention than making them work for it, in rather the same way that making up news is cheaper and more profitable than actually reporting the truth. - Ben Thompson
These tech companies are literally reshaping society & are sucking the life out of the economy, destroying adjacent markets & bulldozing regulatory concerns, all while offloading costs onto everyone else around them.
The crumbling of the American dream is a purple problem, obscured by solely red or solely blue lenses. Its economic and cultural roots are entangled, a mixture of government, private sector, community and personal failings. But the deepest root is our radically shriveled sense of “we.” ... Until we treat the millions of kids across America as our own kids, we will pay a major economic price, and talk of the American dream will increasingly seem cynical historical fiction.
And the solution to killing the middle class, is, of course, to kill the middle class:
An FTC report recommended suing Google for their anti-competitive practices, but no suit was brought. The US Copyright Office Register was relieved of her job after she went against Google's views on set top boxes. Years ago many people saw where this was headed:
"This is a major affront to copyright," said songwriter and music publisher Dean Kay. "Google seems to be taking over the world - and politics ... Their major position is to allow themselves to use copyright material without remuneration. If the Copyright Office head is towing the Google line, creators are going to get hurt."
Singer Don Henley said Pallante's ouster was "an enormous blow" to artists. "She was a champion of copyright and stood up for the creative community, which is one of the things that got her fired," he said. ... [Pallante's replacement] Hayden "has a long track record of being an activist librarian who is anti-copyright and a librarian who worked at places funded by Google."
And in spite of the growing importance of tech media coverage of the industry is a trainwreck:
This is what it’s like to be a technology reporter in 2016. Freebies are everywhere, but real access is scant. Powerful companies like Facebook and Google are major distributors of journalistic work, meaning newsrooms increasingly rely on tech giants to reach readers, a relationship that’s awkward at best and potentially disastrous at worst.
Being a conduit breeds exclusives. Challenging the grand narrative gets one blackballed.
Google announced they are releasing a mobile first search index:
Although our search index will continue to be a single index of websites and apps, our algorithms will eventually primarily use the mobile version of a site’s content to rank pages from that site, to understand structured data, and to show snippets from those pages in our results. Of course, while our index will be built from mobile documents, we're going to continue to build a great search experience for all users, whether they come from mobile or desktop devices.
There are some forms of content that simply don't work well on a 350 pixel wide screen, unless they use a pinch to zoom format. But using that format is seen as not being mobile friendly.
Imagine you have an auto part database which lists alternate part numbers, price, stock status, nearest store with part in stock, time to delivery, etc. ... it is exceptionally hard to get that information to look good on a mobile device. And good luck if you want to add sorting features on such a table.
The theory that using the desktop version of a page to rank mobile results is flawed because users might find something which is only available on the desktop version of a site is a valid point. BUT, at the same time, a publisher may need to simplify the mobile site & hide data to improve usability on small screens & then only allow certain data to become visible through user interactions. Not showing those automotive part databases to desktop users would ultimately make desktop search results worse for users by leaving huge gaps in the search results. And a search engine choosing to not index the desktop version of a site because there is a mobile version is equally short sighted. Desktop users would no longer be able to find & compare information from those automotive parts databases.
Once again money drives search "relevancy" signals.
Since Google will soon make 3/4 of their ad revenues on mobile that should be the primary view of the web for everyone else & alternate versions of sites which are not mobile friendly should be disappeared from the search index if a crappier lite mobile-friendly version of the page is available.
Amazon converts well on mobile in part because people already trust Amazon & already have an account registered with them. Most other merchants won't be able to convert at anywhere near as well of a rate on mobile as they do on desktop, so if you have to choose between having a mobile friendly version that leaves differentiated aspects hidden or a destkop friendly version that is differentiated & establishes a relationship with the consumer, the deeper & more engaging desktop version is the way to go.
The heavy ad load on mobile search results only further combine with the low conversion rates on mobile to make building a relationship on desktop that much more important.
Even TripAdvisor is struggling to monetize mobile traffic, monetizing it at only about 30% to 33% the rate they monetize desktop & tablet traffic. Google already owns most the profits from that market.
Webmasters are better off NOT going mobile friendly than going mobile friendly in a way that compromises the ability of their desktop site.
Mobile-first: with ONLY a desktop site you'll still be in the results & be findable. Recall how mobilegeddon didn't send anyone to oblivion?— Gary Illyes (@methode) November 6, 2016
I am not the only one suggesting an over-simplified mobile design that carries over to a desktop site is a losing proposition. Consider Nielsen Norman Group's take:
in the current world of responsive design, we’ve seen a trend towards insufficient information density and simplifying sites so that they work well on small screens but suboptimally on big screens.
Publishers are getting squeezed to subsidize the primary web ad networks. But the narrative is that as cross-device tracking improves some of those benefits will eventually spill back out into the partner network.
I am rather skeptical of that theory.
Facebook already makes 84% of their ad revenue from mobile devices where they have great user data.
They are paying to bring new types of content onto their platform, but they are only just now beginning to get around to test pricing their Audience Network traffic based on quality.
Priorities are based on business goals and objectives.
When these ad networks are strong & growing quickly they may be able to take a stand, but when growth slows the stock prices crumble, data security becomes less important during downsizing when morale is shattered & talent flees. Further, creating alternative revenue streams becomes vital "to save the company" even if it means selling user data to dangerous dictators.
The other big risk of such tracking is how data can be used by other parties.
Data is being used in all sorts of crazy ways the central ad networks are utterly unaware of. These crazy policies are not limited to other countries. Buying dog food with your credit card can lead to pet licensing fees. Even cheerful "wellness" programs may come with surprises.
Want to see what the future looks like?
About 2 months ago I saw a Facebook post done on behalf of a friend of mine. Gofundme was the plea. Her insurance wouldn’t cover her treatment for a recurring breast cancer and doctors wouldn’t start the treatment unless the full payment was secured in a advance. Really? Really. She was gainfully employed, had a full time, well paying job. But guess what? It wasn’t enough although hundreds of people donated.
This last week she died. She was 38 years old. She died not getting access to a treatment that may or may not have saved her life. She died having to hustle folks for funds to just have a chance to get access to another treatment option and she died while worrying about being financially ruined by her illness. Just horrid.
Is this the society we want? People forced to beg friends on gofundme for help so they can get access to medical treatment? Is this the society we are? Is this truly the best we can do?
Click here to read more.
In my opinion one of the more powerful and underutilized tools of a blog or website is the ability to tag your pages and posts. That said, effectively using tags isn’t easy or straightforward. In this post I’ll take you through some examples of how to use tags and get the most out of them, the pitfalls to watch out ... [Continue reading How to Use Tags on Your Blog or Website]
If you are an run an online ecommerce program, it’s very likely that you have come up against the coupon code dilemma. You offer a deal that allows customers to use a coupon code for a free item, free shipping, or to get a discount. However, what do you do when the promotion ends? If you leave the box open, ... [Continue reading How to Use Coupon Codes to Promote your business]
Many local businesses struggle to find ways to promote themselves, get links, rank in search engines, and ultimately drive traffic to their websites. Most small businesses also have limited budgets making the task more difficult. For those businesses here are 13 ways you can promote your business online for free. Google Local: If you don’t already have one Create an ... [Continue reading 13 Ways to Promote your Local Business for Free]
One of the recurring themes I’ve been seeing in the past few months, comes from web publishers trying to confirm when and where Google updates actually happen. This type of thing has been going on for years and it’s not a new topic, what is new is that Google is now confirming that there is a lot of contradictory information ... [Continue reading Why Google Doesn’t Like to Talk About Updates]
Last week Danny Sullivan twittered he was looking for a creative commons image to use. Christine Churchill poked a little fun at me wondering how I was able to get that so quickly. The trick is using search shortcuts, and I’m going to show you how to do it. For my first example I’m going to set up a creative ... [Continue reading How to Add Search Shortcuts to Google Chrome.]
As I sit here at Stockholm Airport drinking an americano venti at Starbucks, about to board a plane back to Amsterdam, I write on my mobile a story I hope will help you become the person you want to be. One day, it will be your turn to give your first keynote presentation in front […]
Het bericht Dancing with fear – How to give your first Keynote verscheen eerst op Traffic4u.
Het zoeklandschap van Google wordt steeds competitiever. De consument wordt daarnaast veeleisender. Het is daarom belangrijker dan ooit dat jouw advertentie de zoeker in één oogopslag aanspreekt en verder helpt. Doe je dit niet, dan loop je de kans relevante bezoekers mis te lopen. Zoekers zullen sneller klikken op relevante advertenties die aansluiten op hun […]
Het bericht Hoe jij nu nog relevanter kunt adverteren in Google Adwords verscheen eerst op Traffic4u.
Steeds sneller drijft Search Advertising weg van keywords. We leggen het lot van de advertenties in de handen van algoritmes die bepalen wanneer en voor wie deze relevant zijn en dit blijkt succesvol. Zo halen veel retailers al 20-30% van hun online search advertising omzet uit Google Shopping. Dat zien we weer terug in de […]
Dat de consumenten steeds meer tijd besteden op hun smartphone mag duidelijk zijn. In 2015 alleen al zijn er meer dan 65 miljard apps geïnstalleerd vanuit de Play store. Niet alleen in gebruik maar ook qua online winkelen wordt het mobiele apparaat steeds belangrijker. Op dit moment is namelijk al 30% van alle online aankopen […]
Net als in de offline marketing, is een holistische aanpak in de online marketing cruciaal om het maximale resultaat uit de hele online mix te halen. Binnen de Google omgeving wordt echter zelden effectief aangestuurd op doelstellingen die prestaties van het paid (SEA) en organische (SEO ) kanaal als geheel bekijken. Inzicht verkrijgen in dergelijke […]
Het bericht bGenius geeft inzicht in Search: alle SEA en SEO centraal geïntegreerd! verscheen eerst op Traffic4u.