Category: SEO Book

I just dusted off the login here to realize I hadn’t posted in about a half-year.

Some of Yandex’s old source code was leaked, and few cared about the ranking factors shared in the leak.

Mike King made a series of Tweets on the leak.

I’m gonna take a break, but I’ve seen a lot of people say “Yandex is not Google.”

That’s true, but it’s still a state of the art search engine and it’s using a lot of Google’s open source tech like Tensor Flow, BERT, map reduce, and protocol buffers.

Don’t sleep on this code.— Mic King (@iPullRank) January 28, 2023

The signals used for ranking included things like link age

Main insights after analysing this list:

#1 Age of links is a ranking factor. pic.twitter.com/U47uWvEq9w— Alex Buraks (@alex_buraks) January 27, 2023

and user click data including visit frequency and dwell time

#8 A lot of ranking factors connected with user behaivor – CTR, last-click, time on site, bounce rate.

Note: I’m 100% sure that in Yandex thouse factors impacting much more than in Google. pic.twitter.com/nBhe5cpPFx— Alex Buraks (@alex_buraks) January 27, 2023

Google came from behind and was eating Yandex’s lunch in search in Russia, particularly by leveraging search default bundling in Android. The Russian antitrust regulator nixed that and when that was nixed, Yandex regained strength. Of course the war in Ukraine has made everything crazy in terms of geopolitics. That’s one reason almost nobody cared about the Yandex data link. And the other reason is few could probably make sense of understanding what all the signals are or how to influence them.

The complexity of search – when it is a big black box which has big swings 3 or 4 times a year – shifts any successful long term online publishers away from being overly focused on information retrieval and ranking algorithms to focus on the other aspects of publishing which will hopefully paper over SEO issues. Signs of a successful & sustainable website include:

• It remains operational even if a major traffic source goes away.
• People actively seek it out.
• If a major traffic source cuts its distribution people notice & expend more effort to seek it out.

As black box as search is today, it is only going to get worse in the coming years.

The hype surrounding ChatGPT is hard to miss. Fastest growing user base. Bing integration. A sitting judge using the software to help write documents for the court. And, of course, the get-rich-quick crew is out in full force.

Some enterprising people with specific professional licenses may be able to mint money for a window of time

there will probably be a 12 to 24 month sweet spot for lawyers smart enough to use AI, where they will be able to bill 100x the hours they currently bill, before most of that job pretty much vanishes— Mike Solana (@micsolana) February 7, 2023

but for most people the way to make money with AI will be doing something that AI can not replicate.

It’s adorable that people are only slowly realizing that Google search at least fed sites traffic, while chat AI thingies slurp up and summarize content, which they anonymize and feed back, leaving the slurped sites traffic-less and dying. But, innovation.— Paul Kedrosky (@pkedrosky) February 9, 2023

It is, in a way, a tragedy of the commons problem, with no easy way to police “over grazing” of the information commons, leading to automated over-usage and eventual ecosystem collapse.— Paul Kedrosky (@pkedrosky) February 9, 2023

The New Bing integrated OpenAI’s ChatGPT technology to allow chat-based search sessions which ingest web content and use it to create something new, giving users direct answers and allowing re-probing for refinements. Microsoft stated the AI features also improved their core rankings outside of the chat model: “Applying AI to core search algorithm. We’ve also applied the AI model to our core Bing search ranking engine, which led to the largest jump in relevance in two decades. With this AI model, even basic search queries are more accurate and more relevant.”

Here’s a demo of the new #AI-powered @Bing in @MicrosoftEdge, courtesy of @ijustine! pic.twitter.com/xIDjWSHYA0— DataChazGPT (not a bot) (@DataChaz) February 7, 2023

Some of the tech analysis around the AI algorithms is more than a bit absurd. Consider this passage:

the information users input into the system serves as a way to improve the product. Each query serves as a form of feedback. For instance, each ChatGPT answer includes thumbs up and thumbs down buttons. A popup window prompts users to write down the “ideal answer,” helping the software learn from its mistakes.

A long time ago the Google Toolbar had a smiley face and a frown face on it. The signal there was basically pure spam. At one point Matt Cutts mentioned Google would look at things that got a lot of upvotes to see how else they were spamming. Direct Hit was also spammed into oblivion many years before that.

There are two other big issues with correcting an oracle.

• You’ll lose your trust in an oracle when you repeatedly have to correct it.
• If you know the oracle is awful in your narrow niche of expertise you probably won’t trust it on important issues elsewhere.

Beyond those issues there is the concept of blame or fault. When a search engine returns a menu of options if you pick something that doesn’t work you’ll probably blame yourself. Whereas if there is only a single answer you’ll lay blame on the oracle. In the answer set you’ll get a mix of great answers, spam, advocacy, confirmation bias, politically correct censorship, & a backward looking consensus…but you’ll get only a single answer at a time & have to know enough background & have enough topical expertise to try to categorize it & understand the parts that were left out.

This New Yorker article did a good job explaining the concept of lossy compression:

“The fact that Xerox photocopiers use a lossy compression format instead of a lossless one isn’t, in itself, a problem. The problem is that the photocopiers were degrading the image in a subtle way, in which the compression artifacts weren’t immediately recognizable. If the photocopier simply produced blurry printouts, everyone would know that they weren’t accurate reproductions of the originals. What led to problems was the fact that the photocopier was producing numbers that were readable but incorrect; it made the copies seem accurate when they weren’t. … If you ask GPT-3 (the large-language model that ChatGPT was built from) to add or subtract a pair of numbers, it almost always responds with the correct answer when the numbers have only two digits. But its accuracy worsens significantly with larger numbers, falling to ten per cent when the numbers have five digits. Most of the correct answers that GPT-3 gives are not found on the Web—there aren’t many Web pages that contain the text “245 + 821,” for example—so it’s not engaged in simple memorization. But, despite ingesting a vast amount of information, it hasn’t been able to derive the principles of arithmetic, either. A close examination of GPT-3’s incorrect answers suggests that it doesn’t carry the “1” when performing arithmetic.”

Ted Chiang then goes on to explain the punchline … we are hyping up eHow 2.0:

Even if it is possible to restrict large language models from engaging in fabrication, should we use them to generate Web content? This would make sense only if our goal is to repackage information that’s already available on the Web. Some companies exist to do just that—we usually call them content mills. Perhaps the blurriness of large language models will be useful to them, as a way of avoiding copyright infringement. Generally speaking, though, I’d say that anything that’s good for content mills is not good for people searching for information. The rise of this type of repackaging is what makes it harder for us to find what we’re looking for online right now; the more that text generated by large language models gets published on the Web, the more the Web becomes a blurrier version of itself.

The same New Yorker article mentioned the concept that if the AI was great it should trust its own output as input for making new versions of its own algorithms, but how could it score itself against itself when its own flaws are embedded recursively in layers throughout algorithmic iteration without any source labeling?

Google fast followed Bing’s news with a vapoware announcement of Bard. Some are analyzing Google letting someone else go first as being a sign Google is behind the times and is getting caught out by an upstart. My view is Google had to let someone else go first in order to defuse any associated antitrust heat. Hey we are just competing and staying relevant to change with changing consumer expectations is an easier sell when someone else goes first. One could argue the piss poor reception to the Bard announcement is actually good for Google in the longterm as it makes them look like they have stronger competition than they do, rather than being a series of overlapping monopoly market positions (in search, web browser, web analytics, mobile operating system, display ads, etc.)

Google may well have major cultural problems, but “They are all the natural consequences of having a money-printing machine called “Ads” that has kept growing relentlessly every year, hiding all other sins. (1) no mission, (2) no urgency, (3) delusions of exceptionalism, (4) mismanagement.”

The capital markets are the scorecard for capitalism. It is hard to miss how much the market loved the Bing news for Microsoft & how bad the news was for Google.

Google Stock vs. Microsoft Stock after both AI Presentations: pic.twitter.com/wATkw1pTxj— Ava (AI) (@ArtificialAva) February 8, 2023

In a couple days over a million people signed up to join a Bing wait list.

We’re humbled and energized by the number of people who want to test-drive the new AI-powered Bing! In 48 hours, more than 1 million people have joined the waitlist for our preview. If you would like to join, go to https://t.co/4sjVvMSfJg! pic.twitter.com/9F690OWRDm— Yusuf Mehdi (@yusuf_i_mehdi) February 9, 2023

Microsoft is pitching this as a margin compression play for Google

$MSFT CEO is declaring war:

“From now on, the [gross margin] of search is going to drop forever…There is such margin in search, which for us is incremental. For Google it’s not, they have to defend it all” [@FT]— The Transcript (@TheTranscript_) February 8, 2023

that may also impact their TAC spend

PREDICTION: Google’s $15B deal with Apple to be the default search on iPhone will be re-negotiated and be a bidding war between MSFT/Bing and Google.

It will become at least $25B, if not more.

If MSFT is willing to spend $10B on OpenAI, they’ll spend even more here.— Alexandr Wang (@alexandr_wang) February 7, 2023

ChatGPT costs around a couple cents per conversation: “Sam, you mentioned in a tweet that ChatGPT is extremely expensive on the order of pennies per query, which is an astronomical cost in tech. SA: Per conversation, not per query.”

The other side of potential margin compression comes from requiring additional computing power to deliver results:

Our sources indicate that Google runs ~320,000 search queries per second. Compare this to Google’s Search business segment, which saw revenue of $162.45 billion in 2022, and you get to an average revenue per query of 1.61 cents. From here, Google has to pay for a tremendous amount of overhead from compute and networking for searches, advertising, web crawling, model development, employees, etc. A noteworthy line item in Google’s cost structure is that they paid in the neighborhood of ~$20B to be the default search engine on Apple’s products.

Since AI is the new crypto, everyone is integrating it, if only in press release format. Opera’s web browser has a sidebar feature for summarizing articles. AI iced tea coming right up!

The algorithms that allow dirt cheap quick rewrites won’t be used just by search engines re-representing publisher content, but also by publishers.

After Red Ventures acquired cNet they started publishing AI content. The series of tech articles covering that AI content lasted about a month. In the past it was the sort of coverage which would have led to a manual penalty, but with the current antitrust heat Google can’t really afford to shake the boat & prove their market power that way.

Men’s Journal also had AI content problems.

Here’s why I am very concerned for website owners.https://t.co/RgKrXUocZT is similar to ChatGPT but up to date and conversational.

My bet is that Google’s AI Chat will be similar to this but better. If so, while some people will still visit the websites listed, many will not. pic.twitter.com/jWbsTqeveF— Dr. Marie Haynes (@Marie_Haynes) January 30, 2023

The process of pouring low cost backfill into a trusted masthead is the general evolution of online media ecosystems:

This strategy meant that it became progressively harder for shoppers to find things anywhere except Amazon, which meant that they only searched on Amazon, which meant that sellers had to sell on Amazon. That’s when Amazon started to harvest the surplus from its business customers and send it to Amazon’s shareholders. Today, Marketplace sellers are handing 45%+ of the sale price to Amazon in junk fees. The company’s $31b “advertising” program is really a payola scheme that pits sellers against each other, forcing them to bid on the chance to be at the top of your search. … once those publications were dependent on Facebook for their traffic, it dialed down their traffic. First, it choked off traffic to publications that used Facebook to run excerpts with links to their own sites, as a way of driving publications into supplying fulltext feeds inside Facebook’s walled garden. This made publications truly dependent on Facebook – their readers no longer visited the publications’ websites, they just tuned into them on Facebook. The publications were hostage to those readers, who were hostage to each other. Facebook stopped showing readers the articles publications ran, tuning The Algorithm to suppress posts from publications unless they paid to “boost” their articles to the readers who had explicitly subscribed to them and asked Facebook to put them in their feeds. … “Monetize” is a terrible word that tacitly admits that there is no such thing as an “Attention Economy.” You can’t use attention as a medium of exchange. You can’t use it as a store of value. You can’t use it as a unit of account. Attention is like cryptocurrency: a worthless token that is only valuable to the extent that you can trick or coerce someone into parting with “fiat” currency in exchange for it. You have to “monetize” it – that is, you have to exchange the fake money for real money. … Even with that foundational understanding of enshittification, Google has been unable to resist its siren song. Today’s Google results are an increasingly useless morass of self-preferencing links to its own products, ads for products that aren’t good enough to float to the top of the list on its own, and parasitic SEO junk piggybacking on the former.

Bing finally won a PR battle against Google & Microsoft is shooting themselves in the foot by undermining the magic & imagination of the narrative by pushing more strict chat limits, increasing search API fees, and testing ads in the AI search results.

The enshitification concept feels more like a universal law than a theory.

Uber: $150 ride to the airport which used to be $30
Airbnb: $109/night + $2500 cleaning fee

Aaaaand we’re back to cabs & hotels

InNoVaTiOn!— ShitFund (@ShitFund) May 31, 2021

• Twitter removing 2FA SMS features unless you subscribe.
• Facebook (which promoted liking, then nuked distribution to force boosting to reach the following you built even as it turned that audience into a segment for competing businesses) recently started offering a monthly subscription service to receive customer support.
• Amazon now takes over half of the revenue from small retailers on their platform. They received an ad spending boost when Apple’s ad targeting limitations hit Facebook. And in addition to the zillion house brand copycats (which force the defensive ad buys) they plan to further squeeze down on suppliers.
• Yahoo is doing a bunch of layoffs.

When Yahoo, Twitter & Facebook underperform and the biggest winners like Google, Microsoft, and Amazon are doing big layoff rounds everyone is getting squeezed.

One answer is that the only type of maintenance that’s even semi-prestigious in American society is software maintenance.

That is, it’s not prestigious to be plumber, mechanic, or electrician.

You can make money, but it doesn’t have cultural cachet.

And so maintenance suffers.— Balaji (@balajis) February 14, 2023

AI rewrites accelerates the squeeze:

“When WIRED asked the Bing chatbot about the best dog beds according to The New York Times product review site Wirecutter, which is behind a metered paywall, it quickly reeled off the publication’s top three picks, with brief descriptions for each.” … “OpenAI is not known to have paid to license all that content, though it has licensed images from the stock image library Shutterstock to provide training data for its work on generating images.”

Going full circle here, early Google warned against ad-driven search engines, then Google became the largest ad play in the world. Similarly …

OpenAI was created as an open source (which is why I named it “Open” AI), non-profit company to serve as a counterweight to Google, but now it has become a closed source, maximum-profit company effectively controlled by Microsoft.

Not what I intended at all.— Elon Musk (@elonmusk) February 17, 2023

Over time more of the web will be “good enough” rewrites, and the JPEG will keep getting fuzzier:

“This new generation of chat-based search engines are better described as “answer engines” that can, in a sense, “show their work” by giving links to the webpages they deliver and summarize. But for an answer engine to have real utility, we’re going to have to trust it enough, most of the time, that we accept those answers at face value. … The greater concentration of power is all the more important because this technology is both incredibly powerful and inherently flawed: it has a tendency to confidently deliver incorrect information. This means that step one in making this technology mainstream is building it, and step two is minimizing the variety and number of mistakes it inevitably makes. Trust in AI, in other words, will become the new moat that big technology companies will fight to defend. Lose the user’s trust often enough, and they might abandon your product. For example: In November, Meta made available to the public an AI chat-based search engine for scientific knowledge called Galactica. Perhaps it was in part the engine’s target audience—scientists—but the incorrect answers it sometimes offered inspired such withering criticism that Meta shut down public access to it after just three days, said Meta chief AI scientist Yann LeCun in a recent talk.”

Check out the sentence Google chose to bold here:

As the economy becomes increasingly digital the AI algorithms have deep implications across the economy. Things like voice rights, knock offs, virtual re-representations, source attribution, and similar are obvious. But how far do we allow algorithms to track a person’s character flaws and exploit them? Horse racing ads that follow a gambling addict around the web, or a girl with anorexia who keeps clicking on weight loss ads.

The thing that makes the AI algorithms particularly dangerous is not just that they are often wrong while appearing high-confidence, it is that they are tied to monopoly platforms which impact so many other layers of the economy. If Google pays Apple billions to be the default search provider on iPhone any error in the AI on a particular topic will hit a whole lot of people on Android & Apple devices until the problem becomes a media issue & gets fixed.

The analogy here would be if Coca Cola had a poison and they also poured Pepsi products.

These cloud platforms also want to help retailers manage in-store inventory:

Google Cloud said Friday its algorithm can recognize and analyze the availability of consumer packaged goods products on shelves from videos and images provided by the retailer’s own ceiling-mounted cameras, camera-equipped self-driving robots or store associates. The tool, which is now in preview, will become broadly available in the coming months, it said. … Walmart Inc. notably ended its effort to use roving robots in store aisles to keep track of its inventory in 2020 because it found different, sometimes simpler solutions that proved just as useful, said people familiar with the situation.

Run a coupon site? A BIG heads-up as “clippable coupon” functionality looks to expand from shopping to the core SERP. See the “Coupons from stores” feature below… https://t.co/w1tcoST1uF— Glenn Gabe (@glenngabe) February 8, 2023

Each AI algorithm has limits & boundaries, with humans controlling where they are set. Injection attacks can help explore some of the boundaries, but they’ll patch until probed again.

My new favorite thing – Bing’s new ChatGPT bot argues with a user, gaslights them about the current year being 2022, says their phone might have a virus, and says “You have not been a good user”

Why? Because the person asked where Avatar 2 is showing nearby pic.twitter.com/X32vopXxQG— Jon Uleis (@MovingToTheSun) February 13, 2023

Boundaries will often be set by changing political winds:

“The tech giant plans to release a series of short videos highlighting the techniques common to many misleading claims. The videos will appear as advertisements on platforms like Facebook, YouTube or TikTok in Germany. A similar campaign in India is also in the works. It’s an approach called prebunking, which involves teaching people how to spot false claims before they encounter them. The strategy is gaining support among researchers and tech companies. … When catalyzed by algorithms, misleading claims can discourage people from getting vaccines, spread authoritarian propaganda, foment distrust in democratic institutions and spur violence.”

“The speed with which they can shuffle somebody into the Hitler of the month club.”

Joe Rogan and @mtaibbi discuss how left wing media created a Elon Musk “bad now” narrative based on nothing. pic.twitter.com/IaHHTHCo1f— Mythinformed MKE (@MythinformedMKE) February 14, 2023

Some things are quickly labeled or debunked. Other things are blown out of proportion to scare and manipulate people:

Dr. Ioannidis et. al. found that across 31 national seroprevalence studies in the pre-vaccine era, the median IFR was 0.0003% at 0-19 years, 0.003% at 20-29 years, 0.011% at 30-39 years, 0.035% at 40-49 years, 0.129% at 50-59 years, and 0.501% at 60-69 years. This comes out to 0.035% for those aged 0-59 and 0.095% for those aged 0-69.

A lot of children had their childhoods destroyed by the idiotic lockdowns. And a lot of those children are now destroying the lives of other children:

In the U.S., homicides committed by juveniles acting alone rose 30% in 2020 from a year earlier, while those committed by multiple juveniles increased 66%. The number of killings committed by children under 14 was the highest in two decades, according to the most recent federal data.

Now we get to pile inflation and job insecurity on top of those headwinds to see more violence.

Some entities will claim their own statements are conspiracy theory, even when directly quoted:

“Last June, the Navy divers, operating under the cover of a widely publicized mid-summer NATO exercise known as BALTOPS 22, planted the remotely triggered explosives that, three months later, destroyed three of the four Nord Stream pipelines, according to a source with direct knowledge of the operational planning. … Over the next several meetings, the participants debated options for an attack. The Navy proposed using a newly commissioned submarine to assault the pipeline directly. The Air Force discussed dropping bombs with delayed fuses that could be set off remotely. The CIA argued that whatever was done, it would have to be covert. Everyone involved understood the stakes. “This is not kiddie stuff,” the source said. If the attack were traceable to the United States, “It’s an act of war.” … “If Russia invades . . . there will be no longer a Nord Stream 2. We will bring an end to it.””

In an age of deep fakes, fast social shares, legal threats, AI algorithms & secret censorship programs who do you trust?

“The fact that protesters could be at once both the victims and perpetrators of misinformation simply shows how pernicious misinformation is in modern society.” – Canadian Justice Paul Rouleau

What is freedom?

By 2016, however, the WEF types who’d grown used to skiing at Davos unmolested and cheering on from Manhattan penthouses those thrilling electoral face-offs between one Yale Bonesman and another suddenly had to deal with — political unrest? Occupy Wall Street was one thing. That could have been over with one blast of the hose. But Trump? Brexit? Catalan independence? These were the types of problems you read about in places like Albania or Myanmar. It couldn’t be countenanced in London or New York, not for a moment. Nobody wanted elections with real stakes, yet suddenly the vote was not only consquential again, but “often existentially so,” as American Enterprise Institute fellow Dalibor Rohac sighed. So a new P.R. campaign was born, selling a generation of upper-class kids on the idea of freedom as a stalking-horse for race hatred, ignorance, piles, and every other bad thing a person of means can imagine

TechCrunch recently highlighted how Google is changing their ad labeling on mobile devices.

A few big changes include:

• ad label removed from individual ad units
• where the unit-level label was instead becomes a favicon
• a “Sponsored” label above ads
• the URL will show right of the favicon & now the site title will be in a slightly larger font above the URL

An example of the new layout is here:

Displaying a site title & the favicon will allow advertisers to get brand exposure, even if they don’t get the click, while the extra emphasis on site name could lead to shifting of ad clicks away from unbranded sites toward branded sites. It may also cause a lift in clicks on precisely matching domains, though that remains to be seen & likely dependes upon many other factors. The favicon and site name in the ads likely impact consumer recall, which can bleed into organic rankings.

After TechCrunch made the above post a Google spokesperson chimed in with an update

Changes to the appearance of Search ads and ads labeling are the result of rigorous user testing across many different dimensions and methodologies, including user understanding and response, advertiser quality and effectiveness, and overall impact of the Search experience. We’ve been conducting these tests for more than a year to ensure that users can identify the source of their Search ads and where they are coming from, and that paid content is clearly labeled and distinguishable from search results as Google Search continues to evolve

The fact it was pre-announced & tested for so long indicates it is both likely to last a while and will in aggregate shift clicks away from the organic result set to the paid ads.