{"counts": {"scenes": 17, "sources": 85, "journeys": 5, "instances": 85}, "scenes": [{"name": "What Search Actually Does", "number": 1, "sceneId": "s1", "partition": "fundamentals", "sourceUrls": ["https://en.wikipedia.org/wiki/Search_engine_optimization", "https://en.wikipedia.org/wiki/Web_crawler", "https://en.wikipedia.org/wiki/Google_Search", "https://en.wikipedia.org/wiki/PageRank", "https://en.wikipedia.org/wiki/Search_engine_results_page"], "description": "Three distinct operations that get collapsed into one word. Crawling is discovery: software follows links and fetches pages. Indexing is storage and understanding: the fetched page is parsed, its content and meaning recorded. Ranking is selection: for a given query, deciding which of billions of indexed documents to show and in what order. Almost every practical problem in this field is really a failure at one specific stage, and diagnosing which one is most of the job.", "sourceCount": 5}, {"name": "The Fundamentals That Still Hold", "number": 2, "sceneId": "s2", "partition": "fundamentals", "sourceUrls": ["https://developers.google.com/search/docs/essentials", "https://developers.google.com/search/docs/fundamentals/seo-starter-guide", "https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview", "https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls", "https://developers.google.com/search/docs/appearance/page-experience", "https://web.dev/articles/vitals"], "description": "Google's own position is that AI features are built on its core ranking and quality systems, so the groundwork is unchanged: be crawlable, be indexable, be fast enough, have one canonical URL per thing, and tell the crawler where everything is. None of this is glamorous and all of it is prerequisite. A page that is not indexed and eligible for a snippet cannot appear in an AI Overview, which makes technical hygiene the floor rather than the ceiling.", "sourceCount": 6}, {"name": "Quality, and How It Gets Judged", "number": 3, "sceneId": "s3", "partition": "fundamentals", "sourceUrls": ["https://developers.google.com/search/docs/fundamentals/creating-helpful-content", "https://guidelines.raterhub.com/searchqualityevaluatorguidelines.pdf", "https://en.wikipedia.org/wiki/Backlink", "https://yoast.com/complete-guide-seo-copywriting/", "https://developers.google.com/search/docs/essentials/spam-policies"], "description": "Google publishes two things worth reading closely: its guidance on creating helpful content, and the Search Quality Rater Guidelines, a document of nearly two hundred pages used to train the human raters who evaluate search results. The raters do not change rankings directly; their judgements are used to evaluate changes to the systems that do. E-E-A-T — experience, expertise, authoritativeness, trustworthiness — comes from that document, and is a description of what raters are asked to look for rather than a score any algorithm computes.", "sourceCount": 5}, {"name": "Structured Data", "number": 4, "sceneId": "s4", "partition": "fundamentals", "sourceUrls": ["https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data", "https://schema.org/", "https://en.wikipedia.org/wiki/JSON-LD", "https://www.w3.org/TR/json-ld11/", "https://developers.google.com/search/docs/appearance/structured-data/search-gallery", "https://search.google.com/test/rich-results"], "description": "Machine-readable markup describing what a page is about, usually schema.org vocabulary expressed as JSON-LD in a script tag. It does not make a page rank better. What it does is make a page eligible for specific presentation features — review stars, recipes, events, products — and give machines an unambiguous statement of entities and relationships rather than a guess parsed from prose. Google is explicit that it is not required for AI features, and equally explicit that it still recommends it as part of general SEO.", "sourceCount": 6}, {"name": "The Answer Layer Arrives", "number": 5, "sceneId": "s5", "partition": "shift", "sourceUrls": ["https://blog.google/products-and-platforms/products/search/generative-ai-google-search-may-2024/", "https://blog.google/products-and-platforms/products/search/ai-mode-search/", "https://en.wikipedia.org/wiki/ChatGPT", "https://en.wikipedia.org/wiki/Gemini_(chatbot)", "https://en.wikipedia.org/wiki/Perplexity_AI", "https://en.wikipedia.org/wiki/Featured_snippet"], "description": "Between 2023 and 2026 a layer settled on top of search that answers rather than lists. Google shipped generative summaries and then AI Mode, a full conversational surface. ChatGPT gained live web search. Perplexity built an entire product around cited answers. The common structural change is that the user's question is now answered on the page they are already on, with links as supporting evidence rather than as the destination. Everything downstream in this course follows from that one shift.", "sourceCount": 6}, {"name": "What It Did To Clicks", "number": 6, "sceneId": "s6", "partition": "shift", "sourceUrls": ["https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2026", "https://developers.google.com/search/docs/appearance/ai-features", "https://commons.wikimedia.org/wiki/Category:Search_engines"], "description": "The most-cited evidence is Pew's July 2025 study of 900 US adults and 68,879 searches: where an AI summary appeared, users clicked a traditional result in 8% of visits against 15% where none appeared, just 1% clicked a source cited inside the summary, and 26% ended the session without clicking anything against 16% otherwise. Google disputes it, calling the methodology flawed and the query set unrepresentative. Both positions belong in your head at once — the study is the best independent measurement available and it is one panel, one month, one country.", "sourceCount": 4}, {"name": "GEO, AEO, and What the Words Mean", "number": 7, "sceneId": "s7", "partition": "shift", "sourceUrls": ["https://arxiv.org/abs/2311.09735", "https://en.wikipedia.org/wiki/Answer_engine_optimization", "https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search", "https://ahrefs.com/aeo/technical-aeo", "https://developers.google.com/search/blog/2026/05/a-new-resource-for-optimizing"], "description": "Generative Engine Optimization has a real academic origin: a paper by Aggarwal and colleagues presented at KDD 2024, which proposed the term, built a benchmark, and reported visibility gains of up to 40% from specific content modifications. Read the paper before quoting the number, because it is measured against their own visibility metric on their own benchmark, not against revenue or traffic. Answer Engine Optimization is used roughly interchangeably in industry. Neither term describes a system any search provider has confirmed exists as a distinct ranking surface.", "sourceCount": 5}, {"name": "Training Data Versus Live Retrieval", "number": 8, "sceneId": "s8", "partition": "mechanics", "sourceUrls": ["https://en.wikipedia.org/wiki/Retrieval-augmented_generation", "https://en.wikipedia.org/wiki/Large_language_model", "https://commoncrawl.org/", "https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics", "https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)"], "description": "The single most important distinction in this field, and the one most often muddled. A model's training data is a frozen snapshot: content absorbed during training, unattributed, unchangeable after the fact, and impossible to update by editing your site. Live retrieval is different — the system runs a search at question time, fetches current pages, and generates an answer grounded in them, usually with links. Publishing changes can affect retrieval today. They cannot affect a model already trained. Almost every piece of GEO advice is implicitly about one or the other, and rarely says which.", "sourceCount": 5}, {"name": "Who Is Actually Crawling You", "number": 9, "sceneId": "s9", "partition": "mechanics", "sourceUrls": ["https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers", "https://developers.openai.com/api/docs/bots", "https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler", "https://docs.perplexity.ai/docs/resources/perplexity-crawlers", "https://developers.google.com/search/docs/crawling-indexing/robots/intro", "https://en.wikipedia.org/wiki/Robots.txt"], "description": "Several distinct agents with different purposes, and conflating them produces bad decisions. Googlebot crawls for Search. Google-Extended is a separate control governing use in Gemini and other AI products without affecting Search ranking. OpenAI runs distinct agents for training, for live user-initiated browsing, and for search indexing — blocking one does not block the others. Anthropic and Perplexity publish their own. Your server logs are the authoritative record of which of these actually visit you, and they frequently contradict what you assume.", "sourceCount": 6}, {"name": "How Citation Actually Happens", "number": 10, "sceneId": "s10", "partition": "mechanics", "sourceUrls": ["https://en.wikipedia.org/wiki/Semantic_search", "https://en.wikipedia.org/wiki/Word_embedding", "https://en.wikipedia.org/wiki/Vector_database", "https://en.wikipedia.org/wiki/Knowledge_graph", "https://www.robotstxt.org/"], "description": "In a retrieval system the chain runs: the question is converted to a vector, candidate passages are retrieved by similarity from an index, a subset is selected to fit a context window, and the model generates an answer conditioned on them. Being cited therefore requires being retrievable, being selected, and being useful enough to survive into the generated text. This is why a page that answers one specific question cleanly outperforms a longer page that covers it among nine other topics — the unit of retrieval is the passage, not the site.", "sourceCount": 5}, {"name": "The llms.txt Question", "number": 11, "sceneId": "s11", "partition": "mechanics", "sourceUrls": ["https://llmstxt.org/", "https://ahrefs.com/blog/llmstxt-study/", "https://seranking.com/blog/llms-txt/", "https://ahrefs.com/blog/what-is-llms-txt/", "https://developers.google.com/search/podcasts/search-off-the-record"], "description": "A proposed convention: a Markdown file at the site root summarising the site for language models, by analogy with robots.txt. It is a proposal with no formal standard behind it, and the evidence on it is unkind. Google states that Search ignores the file entirely and that creating one will neither harm nor help. Ahrefs examined 137,000 domains and found that of those publishing a valid file, 97% received zero requests for it in May 2026. SE Ranking looked at roughly 300,000 domains and found no relationship between having one and being cited. Note the two studies disagree on adoption — 28% against about 10% — which is itself a lesson about sample selection.", "sourceCount": 5}, {"name": "What Google Actually Tells You To Do", "number": 12, "sceneId": "s12", "partition": "practice", "sourceUrls": ["https://developers.google.com/search/docs/fundamentals/ai-optimization-guide", "https://developers.google.com/search/docs/appearance/snippet", "https://support.google.com/webmasters/answer/16908024", "https://blog.google/products-and-platforms/products/search/search-ai-features-controls/", "https://www.youtube.com/@GoogleSearchCentral"], "description": "Google published a dedicated optimisation guide for generative AI features in May 2026, and it is shorter and more deflationary than the industry built around it. Its position: SEO best practices continue to be relevant because AI features run on the same core systems; create content yourself based on what you actually know; you do not need new machine-readable files, AI text files, markup or Markdown; there is no requirement to chunk content; structured data is not required though still recommended generally; and no third-party tool has access to Google's internal ranking or AI systems. Read the primary document rather than anyone's summary of it, including this one.", "sourceCount": 5}, {"name": "Tactics That Survive Scrutiny", "number": 13, "sceneId": "s13", "partition": "practice", "sourceUrls": ["https://seranking.com/blog/how-to-optimize-for-chatgpt/", "https://www.semrush.com/blog/", "https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a", "https://developers.google.com/search/docs/specialty/ecommerce", "https://www.indexnow.org/"], "description": "What is left once the unevidenced advice is stripped out. Answer one question per page, in the first paragraph, in the words people actually use. Be the primary source for something rather than a summary of other people's work. Keep facts current and dated, because retrieval favours pages that look maintained. Make the page render its content server-side, since a crawler that does not execute JavaScript sees nothing otherwise. Earn genuine mentions on sites that get retrieved. None of this is new, and that is the point — the tactics that work for answer engines are substantially the tactics that worked before, applied with more precision.", "sourceCount": 5}, {"name": "Measurement", "number": 14, "sceneId": "s14", "partition": "practice", "sourceUrls": ["https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports", "https://developers.google.com/search/docs/monitor-debug/search-console-start", "https://validator.schema.org/", "https://commons.wikimedia.org/wiki/Category:Google_Search"], "description": "The part that improved most in 2026. Google added generative AI performance reports to Search Console, showing how often your URLs appeared in AI features in Search and Discover — rolled out to a subset of sites first. Beyond that, your server logs tell you which AI agents fetch what and how often, which is the only first-party evidence available and costs nothing. Everything else on the market is sampled estimation: useful for direction, not for a number you would put in a board paper.", "sourceCount": 4}, {"name": "The Tooling Market", "number": 15, "sceneId": "s15", "partition": "market", "sourceUrls": ["https://www.tryprofound.com/", "https://scrunch.com/", "https://otterly.ai/", "https://athenahq.ai/", "https://peec.ai/", "https://www.seroundtable.com/"], "description": "A real category with real money in it. Profound is the best-known, offering prompt-volume data, citation and sentiment monitoring, agent analytics and content tooling; Scrunch AI, Otterly, AthenaHQ and Peec occupy adjacent positions with differing emphases and price points. What these products genuinely do is sample: run prompts against models repeatedly and record what comes back. That is a legitimate and useful measurement technique. It is not access to anybody's ranking system, and Google says so directly. Judge any vendor claim by whether it is a sampled observation or an assertion about internal mechanics.", "sourceCount": 6}, {"name": "The Crawl Economy", "number": 16, "sceneId": "s16", "partition": "market", "sourceUrls": ["https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/", "https://developers.cloudflare.com/bots/concepts/bot/", "https://web.dev/"], "description": "The commercial question underneath all of it: AI systems consume content and increasingly do not send traffic back, which breaks the implicit bargain the open web ran on. Cloudflare moved in 2025 to block AI crawlers by default for new domains and to build a marketplace where publishers can charge for access. Whether that becomes normal or stays niche is unresolved, and it matters more to most businesses than any tactic in this course — if access becomes negotiated rather than open, visibility becomes a procurement question rather than a content one.", "sourceCount": 3}, {"name": "Where This Is Going", "number": 17, "sceneId": "s17", "partition": "market", "sourceUrls": ["https://arxiv.org/abs/2606.12439", "https://arxiv.org/abs/2609.02964", "https://arxiv.org/abs/2609.07559", "https://en.wikipedia.org/wiki/Schema.org"], "description": "The research has moved past describing GEO to interrogating it. Recent work examines adversarial or malicious optimisation — deliberately crafted content designed to capture generative answers — and defences against it, along with attempts to build manipulation-resistant scoring. A 2026 position paper argues the real risks are concentration, disclosure and academic blind spots rather than individual tactics. The reasonable expectation is that generative surfaces will develop the same adversarial dynamic search did, that the countermeasures will be undocumented, and that anybody promising a durable edge is describing a window rather than a moat.", "sourceCount": 4}], "sources": [{"url": "https://developers.google.com/search/docs/essentials", "type": "webpage", "title": "Google Search Essentials", "summary": "The shortest statement of what Google requires, and the right first read. Three parts: technical requirements for a page to be eligible at all, spam policies describing what gets you removed, and key best practices. Everything else in Google's documentation elaborates on this page. If a tactic contradicts it, the tactic is wrong.", "duration": "20 minutes", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/fundamentals/seo-starter-guide", "type": "webpage", "title": "SEO Starter Guide", "summary": "Google's own introduction, which is notably more restrained than most paid courses on the same material. Covers getting indexed, helping Google understand content, managing crawling, and organising a site. Read after Search Essentials. Worth noting what it does not contain: no keyword density, no word counts, no posting frequency targets.", "duration": "40 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview", "type": "webpage", "title": "Sitemaps", "summary": "A machine-readable list of your URLs with optional metadata about when each changed. It does not guarantee indexing and does not improve ranking; it improves discovery, which matters most for large sites, new sites, and pages that are poorly linked internally. Practical note: a sitemap listing URLs that return errors or redirects actively wastes crawl budget.", "duration": "15 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls", "type": "webpage", "title": "Canonicalisation", "summary": "How to tell a search engine which URL is the real one when the same content is reachable several ways — tracking parameters, http and https, trailing slashes, print views. Unresolved duplication splits signals across versions and leaves the engine to pick for you. One of the most common and least visible technical faults on commercial sites.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/appearance/page-experience", "type": "webpage", "title": "Page experience", "summary": "Google's framing of loading performance, interactivity, visual stability, mobile usability and HTTPS. Read it with the proportion Google itself gives it: these are tie-breakers between comparable pages rather than primary ranking forces. Fixing a slow site is worth doing; expecting it to outrank better content is not.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/fundamentals/creating-helpful-content", "type": "webpage", "title": "Creating helpful, reliable, people-first content", "summary": "Google's guidance on quality, structured as questions to ask about your own work: does it provide original information, does it demonstrate first-hand expertise, would someone bookmark it. The most useful section is the self-assessment list, which is uncomfortable to run honestly against most commercial content. This is also the document that introduced the phrase people-first.", "duration": "25 minutes", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://guidelines.raterhub.com/searchqualityevaluatorguidelines.pdf", "type": "pdf", "title": "Search Quality Rater Guidelines", "summary": "The full document Google gives the human raters who evaluate search results, published openly and running to well over a hundred pages. This is where E-E-A-T is actually defined, along with Your Money or Your Life topics and the page quality scale. Read after the helpful-content guidance. Critical caveat the document states itself: raters do not directly affect rankings — their assessments are used to evaluate changes to the systems that do.", "duration": "3 hours", "integrity": "dependent", "publisher": "guidelines.raterhub.com"}, {"url": "https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data", "type": "webpage", "title": "Introduction to structured data", "summary": "Google's explanation of what markup does and does not do. The essential framing: structured data makes a page eligible for specific result features, it does not improve ranking, and marking up content that is not visible on the page is a policy violation. Start here before touching schema.org itself, because the vocabulary is vast and only a fraction of it is actually consumed.", "duration": "20 minutes", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/appearance/structured-data/search-gallery", "type": "webpage", "title": "Structured data search gallery", "summary": "The definitive list of which markup types Google actually supports and what each produces in results — articles, breadcrumbs, events, FAQs, products, reviews, videos and the rest. Read after the introduction. This is the page that answers the real question: of the hundreds of schema.org types, which ones does anybody actually read.", "duration": "20 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://search.google.com/test/rich-results", "type": "webpage", "title": "Rich Results Test", "summary": "Google's own validator: paste a URL or code and see exactly which rich result types the page qualifies for and which errors block it. Practical rather than theoretical, and the fastest way to settle an argument about whether markup is working. Use it before and after any structured data change.", "duration": "10 minutes", "integrity": "subsidiary", "publisher": "search.google.com"}, {"url": "https://developers.google.com/search/docs/appearance/ai-features", "type": "webpage", "title": "AI features and your website", "summary": "Google's reference page on how AI Overviews and AI Mode interact with sites, including the eligibility requirement that a page be indexed and able to show a snippet. Read alongside the Pew findings as the other side of the argument: Google's stated position is that AI features surface a wider range of sources and that AI Overviews only appear where they add to classic Search.", "duration": "20 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search", "type": "webpage", "title": "Top ways to ensure your content performs well in AI experiences", "summary": "Google's May 2025 blog post, and the earliest clear statement of its position: focus on unique, satisfying content for people rather than on anything AI-specific. Useful chronologically — it shows the advice was consistent a full year before the formal optimisation guide, which undercuts the idea that the rules changed suddenly and nobody told you.", "duration": "12 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/blog/2026/05/a-new-resource-for-optimizing", "type": "webpage", "title": "A new resource for optimizing for generative AI", "summary": "The May 2026 announcement of the optimisation guide. Short, and worth reading for the framing rather than the content: Google chose to publish a dedicated AI optimisation resource whose central message is that you do not need AI-specific optimisation. That tension is the most informative thing about the document.", "duration": "8 minutes", "integrity": "subsidiary", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers", "type": "webpage", "title": "Google crawlers overview", "summary": "Every user agent Google operates and what each is for, including the distinction that matters most here: Googlebot crawls for Search, while Google-Extended is a separate control for whether your content is used in Gemini and other AI products. Blocking Google-Extended does not remove you from Search, and blocking Googlebot removes you from everything. Verify the user agents from this page rather than from a blog.", "duration": "20 minutes", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/crawling-indexing/robots/intro", "type": "webpage", "title": "Introduction to robots.txt", "summary": "How to control crawling, and the limits of doing so. Two points people get wrong constantly: robots.txt controls crawling rather than indexing, so a disallowed URL can still appear in results if linked from elsewhere, and the file is a request that well-behaved crawlers honour rather than an access control. If something must not be public, authentication is the answer, not a disallow line.", "duration": "20 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/fundamentals/ai-optimization-guide", "type": "webpage", "title": "Google's guide to optimizing for generative AI features", "summary": "The keystone document of this entire course, published May 2026. Its actual statements: best practices for SEO continue to be relevant because AI features are built on core Search systems; you do not need to create new machine readable files, AI text files, markup, or Markdown; there is no requirement to break content into small pieces; structured data is not required for generative AI features though still recommended generally; Search ignores llms.txt and creating one will neither harm nor help; and no third-party tool has access to Google's internal ranking or AI systems. Read it in full before buying anything.", "duration": "30 minutes", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/appearance/snippet", "type": "webpage", "title": "Snippets and how to control them", "summary": "The mechanics behind eligibility for AI features, since a page must be able to show a snippet to be used. Covers nosnippet, data-nosnippet and max-snippet, which let you exclude specific passages rather than whole pages. The underrated control here is data-nosnippet, which can keep boilerplate or paywalled previews out of generated summaries without affecting the rest of the page.", "duration": "20 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://support.google.com/webmasters/answer/16908024", "type": "webpage", "title": "The Search generative AI control", "summary": "Search Console's setting for excluding your site's links and content from AI Overviews, AI Mode and Discover's generative features. Note what it costs: exclusion applies to the generative surfaces specifically, takes a few days to take effect, and is a genuine trade-off rather than a free safety measure. Worth knowing it exists even if you never use it.", "duration": "12 minutes", "integrity": "dependent", "publisher": "support.google.com"}, {"url": "https://blog.google/products-and-platforms/products/search/search-ai-features-controls/", "type": "webpage", "title": "Google's approach to website controls for AI features", "summary": "The policy reasoning behind those controls, from Google's own blog. Useful for anyone who has to explain to a board why the company is or is not visible in AI answers: this is the official account of what site owners can and cannot decide, in Google's words rather than a commentator's.", "duration": "12 minutes", "integrity": "subsidiary", "publisher": "blog.google"}, {"url": "https://developers.google.com/search/docs/specialty/ecommerce", "type": "webpage", "title": "Ecommerce SEO guidance", "summary": "Google's specialty documentation for product sites, included because ecommerce is where structured data genuinely earns its keep — product, price, availability and review markup feed shopping surfaces directly. If you sell things, this page is worth more than any amount of general advice about content strategy.", "duration": "25 minutes", "integrity": "subsidiary", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports", "type": "webpage", "title": "Search generative AI performance reports", "summary": "Google's June 2026 announcement of reporting that shows how often your URLs appeared in generative AI features across Search and Discover. The most significant measurement development in the field, because until this existed there was no first-party data at all. Note the caveat in the announcement: it rolled out to a subset of sites first, so absence of the report is not absence of impressions.", "duration": "12 minutes", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/docs/monitor-debug/search-console-start", "type": "webpage", "title": "Getting started with Search Console", "summary": "The tool itself, free, and the only source of first-party data about how Google sees your site: indexing status, queries, impressions, clicks, structured data errors. Read after the generative reports announcement. Anyone discussing their search performance without reference to this is speculating.", "duration": "25 minutes", "integrity": "dependent", "publisher": "developers.google.com"}, {"url": "https://developers.google.com/search/podcasts/search-off-the-record", "type": "audio", "title": "Search Off the Record (podcast)", "summary": "Google's own Search Relations podcast, where Mueller, Splitt and Illyes discuss these questions at length and with more candour than the documentation allows. The best listening in the field, and the place where the llms.txt scepticism was argued through rather than merely stated. Audio, and genuinely useful on a commute.", "duration": "varies", "integrity": "standalone", "publisher": "developers.google.com"}, {"url": "https://www.youtube.com/@GoogleSearchCentral", "type": "video", "title": "Google Search Central (video)", "summary": "The official video channel: office-hours recordings, explainers and conference talks from the Search Relations team. Video, and the most efficient way to absorb the reasoning behind a documented position when the document itself is terse.", "duration": "varies", "integrity": "subsidiary", "publisher": "www.youtube.com"}, {"url": "https://en.wikipedia.org/wiki/Search_engine_optimization", "type": "webpage", "title": "Search engine optimization", "summary": "The field's full history and scope, from the mid-1990s onward, including the long arms race between optimisation and anti-spam systems. Read it for the pattern rather than the tactics: every technique that worked mechanically was eventually detected and neutralised, and the only durable strategy has been to be genuinely the best answer. That pattern is the single most useful prior to carry into the GEO chapters.", "duration": "30 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Web_crawler", "type": "webpage", "title": "Web crawler", "summary": "How discovery actually works: a frontier of URLs, politeness rules, revisit policies, and the economics of deciding what to fetch and how often. Read after the SEO overview. The concept to take away is crawl budget — crawling is not free, so a crawler decides how much of your site is worth its time, and a site that wastes that budget on duplicates gets less of its real content seen.", "duration": "20 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Google_Search", "type": "webpage", "title": "Google Search", "summary": "The product itself and how it has changed: the index, the ranking signals, the long sequence of named updates, and the progressive addition of answer surfaces above the links. Useful mainly for scale and context — the point to absorb is that results pages have been getting less link-shaped for fifteen years, and generative answers are a continuation rather than a break.", "duration": "25 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/PageRank", "type": "webpage", "title": "PageRank", "summary": "The original link-based ranking algorithm, and worth understanding as an idea rather than as current mechanics: importance inferred from the structure of citations rather than from the content itself. Modern ranking uses hundreds of signals and PageRank is one small input, but the underlying intuition — that what others point at tells you something content alone cannot — is exactly the intuition behind why mentions matter for retrieval too.", "duration": "20 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Search_engine_results_page", "type": "webpage", "title": "Search engine results page", "summary": "The anatomy of the page itself: organic results, ads, knowledge panels, featured snippets, local packs, and now generative answers. Optional, and useful for a concrete sense of how little of a modern results page is actually the ten blue links. This is the visual context for every click-through statistic in chapter six.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Backlink", "type": "webpage", "title": "Backlinks", "summary": "What links are as a ranking signal, how link schemes work, and why they remain both effective and dangerous. The honest position: links still matter, buying them violates Google's spam policies, and most link-building services sell exactly what the policies prohibit. Read it before approving any agency proposal containing the word outreach.", "duration": "15 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/JSON-LD", "type": "webpage", "title": "JSON-LD", "summary": "The syntax structured data is actually written in: JSON with a context that maps keys to a shared vocabulary, embedded in a script tag so it does not affect rendering. The practical advantage over older formats is that it sits in one block rather than being woven through the HTML, which means it can be generated and validated independently of the page template.", "duration": "12 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/ChatGPT", "type": "webpage", "title": "ChatGPT", "summary": "The product that made conversational answers mainstream, and now a search surface in its own right with live browsing. Relevant here for one structural reason: a question answered in a chat window has no results page, no position one, and no impression count — which is why the entire measurement apparatus of SEO does not transfer.", "duration": "25 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Gemini_(chatbot)", "type": "webpage", "title": "Gemini", "summary": "Google's model family and assistant, and the system Google-Extended governs your content's use in. Worth keeping distinct from AI Overviews in Search: they are different surfaces with different controls, and conflating them leads people to block the wrong thing. Optional, and the clearest illustration of why the crawler chapter matters.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Perplexity_AI", "type": "webpage", "title": "Perplexity", "summary": "The product built from the start around cited answers rather than retrofitting them, which makes it the clearest working example of what a citation-first engine looks like. Study its output format specifically: numbered inline citations, a visible source list, and follow-up questions. That shape is what content is being selected into.", "duration": "15 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Featured_snippet", "type": "webpage", "title": "Featured snippets", "summary": "The precursor everybody forgot. Google has been extracting direct answers onto results pages since 2014, and the entire zero-click debate was rehearsed then with the same arguments and much the same data. Useful historical grounding: the answer layer is not new, it has become general rather than appearing for the first time.", "duration": "12 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Answer_engine_optimization", "type": "webpage", "title": "Answer engine optimization", "summary": "The companion term, used broadly interchangeably with GEO in industry while sometimes being distinguished as focusing on direct-answer surfaces specifically. Read it mainly to inoculate yourself: the proliferation of near-synonymous acronyms in this field is a marketing phenomenon rather than a conceptual one, and the underlying practices are largely the same.", "duration": "12 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation", "type": "webpage", "title": "Retrieval-augmented generation", "summary": "The architecture behind every cited AI answer, and the most important technical concept in this course. The model does not recall your page; a retrieval step fetches candidate documents at question time and the model generates conditioned on them. The whole of practical GEO reduces to being retrievable and being selected at that step — everything else is downstream of this one mechanism.", "duration": "20 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Large_language_model", "type": "webpage", "title": "Large language models", "summary": "What the model is and how training works, at the level needed to reason about this field properly. The decisive fact for marketers: training is a batch process over a fixed corpus, so a model cannot learn about your rebrand by you publishing a page. If your subject matters to a model's baseline knowledge rather than its retrieval, the lever is time and corpus presence, not optimisation.", "duration": "30 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)", "type": "webpage", "title": "Hallucination in AI", "summary": "Why generated answers about your business can be confidently wrong, and why that is a content problem as much as a model problem. Where the record is thin, contradictory or out of date, the model fills the gap plausibly. The practical implication is defensive: a clear, current, unambiguous statement of basic facts about you reduces the space in which invention happens.", "duration": "20 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Robots.txt", "type": "webpage", "title": "robots.txt", "summary": "The convention itself, its history since 1994, and its genuine limits. Read alongside Google's documentation for the part Google does not emphasise: compliance is voluntary, there is no enforcement, and the growth of AI crawling has put real strain on a protocol built on good manners between a handful of search companies.", "duration": "15 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Semantic_search", "type": "webpage", "title": "Semantic search", "summary": "Matching on meaning rather than on literal terms, which is why keyword-stuffing stopped working and why writing naturally started to. Read after retrieval-augmented generation. The practical consequence is that you no longer need to guess exact phrasings — you need to cover a concept thoroughly enough that any phrasing of the question lands near your text.", "duration": "15 minutes", "integrity": "standalone", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Word_embedding", "type": "webpage", "title": "Embeddings", "summary": "How meaning becomes arithmetic: text mapped into a high-dimensional vector space where proximity approximates similarity. This is the mechanism underneath semantic retrieval, and understanding it explains a lot of otherwise mysterious behaviour — including why a passage on a page about something else can be retrieved while your dedicated page is not.", "duration": "20 minutes", "integrity": "dependent", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Vector_database", "type": "webpage", "title": "Vector databases", "summary": "The infrastructure that makes similarity search fast at scale, using approximate nearest-neighbour indexes. Optional for marketers and essential for anyone implementing retrieval. Included because it makes concrete what retrieval is doing: your content is not being read, it is being measured for proximity to a question.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Knowledge_graph", "type": "webpage", "title": "Knowledge graphs", "summary": "Entities and relationships held as structured facts rather than as prose, which is how a system knows your company and a person with the same name are different things. Relevant to citation because entity disambiguation determines whether answers about you are actually about you, and consistent naming and markup across the web is what makes that resolution work.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://en.wikipedia.org/wiki/Schema.org", "type": "webpage", "title": "Schema.org", "summary": "The vocabulary's own history and governance, founded in 2011 by Google, Microsoft, Yahoo and Yandex. Included in the closing chapter deliberately: it is the last time the major search players agreed a shared standard for machine-readable content, and the obvious question for the next decade is whether anything comparable emerges for generative systems, or whether each provider simply builds its own.", "duration": "15 minutes", "integrity": "subsidiary", "publisher": "en.wikipedia.org"}, {"url": "https://web.dev/articles/vitals", "type": "webpage", "title": "Web Vitals", "summary": "Google's performance metrics defined precisely — what each measures, the thresholds, and how to improve them. Read for the measurement discipline rather than the targets: this is a rare case where the industry has agreed definitions, so performance arguments can be settled with numbers instead of opinions.", "duration": "20 minutes", "integrity": "subsidiary", "publisher": "web.dev"}, {"url": "https://schema.org/", "type": "webpage", "title": "Schema.org vocabulary", "summary": "The vocabulary itself: hundreds of types and thousands of properties, maintained collaboratively and used far beyond search. Browse it rather than reading it. The practical approach is to find the single type that matches your page, implement that properly, and ignore the rest — over-marking is a common and counterproductive instinct.", "duration": "30 minutes", "integrity": "standalone", "publisher": "schema.org"}, {"url": "https://www.w3.org/TR/json-ld11/", "type": "webpage", "title": "JSON-LD 1.1 specification", "summary": "The W3C standard itself, for anyone implementing rather than specifying. Dense and authoritative. Worth having to hand for the parts that trip people up in practice — context resolution, @id and @type semantics, and how nesting actually serialises.", "duration": "1 hour", "integrity": "subsidiary", "publisher": "www.w3.org"}, {"url": "https://www.robotstxt.org/", "type": "webpage", "title": "The Robots Exclusion Protocol", "summary": "The original specification site for robots.txt, documenting a convention that predates almost everything else in this course and still governs who may read your content. Read it for the realisation that the entire economy of web crawling rests on a voluntary text file agreed informally in 1994.", "duration": "12 minutes", "integrity": "subsidiary", "publisher": "www.robotstxt.org"}, {"url": "https://llmstxt.org/", "type": "webpage", "title": "The llms.txt proposal", "summary": "The specification itself, and the thing to read before forming a view. It proposes a Markdown file at the site root giving a model a curated map of the site, by analogy with robots.txt and sitemap.xml. Note its own framing: it is a proposal, open for feedback, with no standards body behind it and no major AI provider having announced that they consume it for general web content.", "duration": "15 minutes", "integrity": "standalone", "publisher": "llmstxt.org"}, {"url": "https://validator.schema.org/", "type": "webpage", "title": "Schema Markup Validator", "summary": "The vocabulary-level validator, as distinct from Google's rich results test: this one checks whether your markup is valid schema.org regardless of whether Google happens to use that type. The right tool when you are marking up for consumers other than Google Search, which is increasingly the point.", "duration": "10 minutes", "integrity": "subsidiary", "publisher": "validator.schema.org"}, {"url": "https://www.indexnow.org/", "type": "webpage", "title": "IndexNow", "summary": "A protocol letting sites notify participating search engines immediately when content changes, supported by Bing and Yandex among others. Practical and underused: for sites where freshness matters, this converts discovery from polling to pushing. Google has not adopted it, which is the honest caveat.", "duration": "10 minutes", "integrity": "subsidiary", "publisher": "www.indexnow.org"}, {"url": "https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a", "type": "webpage", "title": "Bing Webmaster Guidelines", "summary": "The other major index, and worth reading precisely because it is not Google. Bing's guidelines are more explicit in places about what it values, and Bing's index underpins other products — which makes it disproportionately important relative to its direct search share.", "duration": "20 minutes", "integrity": "dependent", "publisher": "www.bing.com"}, {"url": "https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "type": "webpage", "title": "Pew: users click less when an AI summary appears", "summary": "The headline independent study, July 2025. Method: browsing data from 900 US adults on Pew's KnowledgePanel, 68,879 searches in March 2025, of which about 18% produced an AI Overview. Findings: 8% click rate on traditional results with a summary present against 15% without; 1% of visits included a click on a source cited inside the summary; 26% of sessions ended without any click against 16%. Read the methodology section, because the sample and the period are the whole basis of the critique.", "duration": "20 minutes", "integrity": "standalone", "publisher": "www.pewresearch.org"}, {"url": "https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2026", "type": "webpage", "title": "Reuters Institute Digital News Report", "summary": "The annual survey of news consumption across dozens of markets, run by Oxford's Reuters Institute and the most methodologically serious recurring dataset on how people actually find information. Read after Pew for scale and trend rather than a single snapshot. For publishers specifically, this is the document to argue from.", "duration": "1 hour", "integrity": "dependent", "publisher": "reutersinstitute.politics.ox.ac.uk"}, {"url": "https://arxiv.org/abs/2311.09735", "type": "webpage", "title": "GEO: Generative Engine Optimization (KDD 2024)", "summary": "The paper that named the field, by Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan and Deshpande, presented at KDD 2024. It proposes GEO as a black-box optimisation framework, introduces GEO-bench across multiple domains, and reports visibility improvements of up to 40%. Two things to hold: that figure is measured against the authors' own visibility metric on their own benchmark, and the paper itself reports that effectiveness varies substantially by domain. It is the only peer-reviewed foundation the field has, and it is routinely cited by people who have not read it.", "duration": "1 hour", "integrity": "standalone", "publisher": "arxiv.org"}, {"url": "https://ahrefs.com/blog/llmstxt-study/", "type": "webpage", "title": "Ahrefs: 97% of llms.txt files never get read", "summary": "An analysis of 137,000 domains in Ahrefs' own web analytics, May 2026. Of those publishing a valid llms.txt, 97% received zero requests for it that month. Caveats worth stating: it is a vendor study, the sample is sites using one analytics product, and it measures whether files are fetched rather than whether they influence citation. Even with those caveats it is the largest public dataset on the question.", "duration": "20 minutes", "integrity": "dependent", "publisher": "ahrefs.com"}, {"url": "https://seranking.com/blog/llms-txt/", "type": "webpage", "title": "SE Ranking: why llms.txt does not work", "summary": "A second independent analysis, roughly 300,000 domains, asking the different and more useful question: does having the file correlate with being cited by LLMs. It found no relationship. Read alongside Ahrefs and note that the two disagree on adoption — about 10% here against 28% there — which tells you something about how much sample construction drives results in this field.", "duration": "20 minutes", "integrity": "dependent", "publisher": "seranking.com"}, {"url": "https://arxiv.org/abs/2606.12439", "type": "webpage", "title": "GEO creates underexamined risks (position paper)", "summary": "A 2026 position paper arguing that the governance questions around generative engine optimisation — concentration of influence, disclosure of paid placement, and gaps in academic attention — matter more than the tactical questions the industry focuses on. The most useful single document for anyone who has to think about this at policy or board level rather than campaign level.", "duration": "45 minutes", "integrity": "standalone", "publisher": "arxiv.org"}, {"url": "https://arxiv.org/abs/2609.02964", "type": "webpage", "title": "Defending generative search against malicious GEO", "summary": "Research on adversarial optimisation: content deliberately engineered to capture generative answers, and the defences against it. Read after the position paper. This is the clearest signal of where the field goes next — the same detection-and-evasion dynamic that defined twenty years of search spam, now beginning on generative surfaces.", "duration": "45 minutes", "integrity": "dependent", "publisher": "arxiv.org"}, {"url": "https://arxiv.org/abs/2609.07559", "type": "webpage", "title": "Manipulation-resistant content scoring", "summary": "Work on scoring content for generative engines in a way that resists gaming, without needing access to the engine itself. Technical and early. Included because it points at the likely end state: if durable scoring becomes possible, the advantage shifts back to genuinely better content, exactly as it did in search after each wave of manipulation was closed off.", "duration": "40 minutes", "integrity": "subsidiary", "publisher": "arxiv.org"}, {"url": "https://commoncrawl.org/", "type": "webpage", "title": "Common Crawl", "summary": "The open repository of web crawl data that has underpinned a great deal of model training, and the most concrete answer to where does training data come from. Practically useful: if you want to understand whether your content plausibly entered a training corpus, this is the dataset to reason about rather than speculating.", "duration": "20 minutes", "integrity": "standalone", "publisher": "commoncrawl.org"}, {"url": "https://developers.openai.com/api/docs/bots", "type": "webpage", "title": "OpenAI's crawlers", "summary": "OpenAI's own documentation of its user agents and what each does. The essential detail is that they are distinct: one for training-data collection, one for live fetches triggered by a user's question, and one for search indexing. Blocking the training crawler while permitting the user-triggered one is a coherent and fairly common position, and it is only possible if you read this page rather than guessing.", "duration": "15 minutes", "integrity": "standalone", "publisher": "developers.openai.com"}, {"url": "https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler", "type": "webpage", "title": "Anthropic's web crawler", "summary": "Anthropic's documentation on its crawling and how site owners can control it. Read alongside OpenAI's for the comparison: the providers have converged on broadly similar conventions — named user agents, robots.txt compliance, published ranges — without any formal standard requiring it. That voluntary convergence is the entire basis of current control.", "duration": "10 minutes", "integrity": "dependent", "publisher": "support.claude.com"}, {"url": "https://docs.perplexity.ai/docs/resources/perplexity-crawlers", "type": "webpage", "title": "Perplexity's crawlers", "summary": "Perplexity's documentation of its user agents, including the distinction between its index crawler and fetches made in response to a live user question. Relevant because Perplexity is the purest citation-driven product, so understanding what it fetches and when is the cleanest window into retrieval behaviour generally.", "duration": "12 minutes", "integrity": "dependent", "publisher": "docs.perplexity.ai"}, {"url": "https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/", "type": "webpage", "title": "Cloudflare: no AI crawl without compensation", "summary": "Cloudflare's July 2025 announcement that it would block AI crawlers by default for new domains and build a marketplace for paid access. Whatever you think of it, this is the single most consequential infrastructure decision in the period this course covers — a company sitting in front of a large share of web traffic unilaterally changing the default from open to closed.", "duration": "20 minutes", "integrity": "standalone", "publisher": "blog.cloudflare.com"}, {"url": "https://developers.cloudflare.com/bots/concepts/bot/", "type": "webpage", "title": "Cloudflare bot documentation", "summary": "The operational detail behind the policy: how bots are identified, categorised and verified, and what controls are available. Read after the announcement. Practically important because for many sites the effective AI-access policy is set at the CDN rather than in robots.txt, and the two can silently disagree.", "duration": "20 minutes", "integrity": "dependent", "publisher": "developers.cloudflare.com"}, {"url": "https://web.dev/", "type": "webpage", "title": "web.dev", "summary": "Google's resource for web platform engineering, included here as the counterweight to the access debate: whatever the commercial arrangements become, the technical obligation stays the same — content must be reachable, fast and rendered where a machine can see it. Optional, and the best general reference for the engineering half of this work.", "duration": "20 minutes", "integrity": "subsidiary", "publisher": "web.dev"}, {"url": "https://www.tryprofound.com/", "type": "webpage", "title": "Profound", "summary": "The best-known platform in the category. Its published capabilities include prompt-volume data, monitoring of how models describe a brand, citation and sentiment tracking, agent analytics on which bots fetch which pages, benchmarking against competitors, and content tooling. It also runs a research hub publishing original analyses. Evaluate it as a sampling and measurement product, which is what it is, rather than as a channel into anybody's ranking system.", "duration": "20 minutes", "integrity": "standalone", "publisher": "www.tryprofound.com"}, {"url": "https://scrunch.com/", "type": "webpage", "title": "Scrunch AI", "summary": "An enterprise-positioned competitor focused on how brands appear across AI platforms and on optimising the experience AI agents have of a site. Read after Profound for the comparison. The category is consolidating, so check the current ownership and status of any vendor before committing — this market has moved faster than its own marketing pages.", "duration": "15 minutes", "integrity": "dependent", "publisher": "scrunch.com"}, {"url": "https://otterly.ai/", "type": "webpage", "title": "Otterly.AI", "summary": "A smaller, more accessible monitoring tool tracking brand mentions and links across AI search platforms. Included to show the range of the market: the same core technique — repeated prompting and recording what comes back — is sold at wildly different price points, and the differences are largely in coverage, frequency and reporting rather than in method.", "duration": "12 minutes", "integrity": "subsidiary", "publisher": "otterly.ai"}, {"url": "https://athenahq.ai/", "type": "webpage", "title": "AthenaHQ", "summary": "Another entrant in the AI-visibility monitoring space, positioned around tracking and improving how brands surface across multiple models. Optional, and worth a look mainly for how similar the category's propositions have become — a useful signal in itself about how undifferentiated the underlying technique is.", "duration": "12 minutes", "integrity": "subsidiary", "publisher": "athenahq.ai"}, {"url": "https://peec.ai/", "type": "webpage", "title": "Peec AI", "summary": "A European entrant in the same category, frequently compared against the others on price and model coverage. Included for completeness of the landscape. If you are running a procurement exercise, the questions that actually separate these products are which models they sample, how often, in which markets, and whether they will show you the raw responses.", "duration": "12 minutes", "integrity": "subsidiary", "publisher": "peec.ai"}, {"url": "https://www.seroundtable.com/", "type": "webpage", "title": "Search Engine Roundtable", "summary": "Barry Schwartz's long-running daily record of what search representatives actually said and when. Unglamorous and invaluable: when a vendor claims Google confirmed something, this is where you check. The archive stretches back two decades, which makes it the best available defence against confidently misremembered history.", "duration": "20 minutes", "integrity": "standalone", "publisher": "www.seroundtable.com"}, {"url": "https://yoast.com/complete-guide-seo-copywriting/", "type": "webpage", "title": "Yoast: the complete guide to SEO", "summary": "A long free guide covering keyword research, search intent, and the actual craft of writing and editing for search -- the practitioner half that Google's documentation deliberately leaves out. Read it after Google's own pages rather than before, so you can tell which parts are Google's position and which are the industry's interpretation of it. Its section on using AI as a guide rather than a content factory is unusually sober for a vendor.", "duration": "1 hour", "integrity": "dependent", "publisher": "yoast.com"}, {"url": "https://developers.google.com/search/docs/essentials/spam-policies", "type": "webpage", "title": "Google's spam policies", "summary": "The enforcement side of quality, and far more concrete than the guidance on helpful content. It names the specific practices that get a site demoted or removed: link spam, scaled content abuse, cloaking, doorway pages, expired domain abuse and site reputation abuse. Read it before approving any agency proposal, because several of these describe services that are still openly sold.", "duration": "25 minutes", "integrity": "subsidiary", "publisher": "developers.google.com"}, {"url": "https://ahrefs.com/aeo/technical-aeo", "type": "webpage", "title": "Ahrefs: technical AEO", "summary": "A practitioner guide to the technical side of answer-engine optimisation, and one of the more grounded pieces of vendor content in the field — it distinguishes what is demonstrated from what is inferred, which most does not. Read as a counterpart to the academic paper: this is what the ideas look like when someone has to implement them on Monday.", "duration": "30 minutes", "integrity": "dependent", "publisher": "ahrefs.com"}, {"url": "https://ahrefs.com/blog/what-is-llms-txt/", "type": "webpage", "title": "Ahrefs: what is llms.txt", "summary": "The explainer to read before the data, covering what the file is, what it proposes to do, and what adopting it involves. Honest about the uncertainty. Read it, then read the two studies — the sequence of understanding the proposal and then seeing the evidence against it is the most instructive ten minutes in this course.", "duration": "15 minutes", "integrity": "dependent", "publisher": "ahrefs.com"}, {"url": "https://seranking.com/blog/how-to-optimize-for-chatgpt/", "type": "webpage", "title": "SE Ranking: how to optimise for ChatGPT", "summary": "A practitioner guide grounded in the finding that community platforms are disproportionately represented in AI citations — which leads to the genuinely useful and slightly uncomfortable conclusion that earning a reputation on forums where people actually discuss your category may do more than another page on your own site.", "duration": "25 minutes", "integrity": "standalone", "publisher": "seranking.com"}, {"url": "https://www.semrush.com/blog/", "type": "webpage", "title": "Semrush blog", "summary": "A large library of research posts and guides, including periodic original data studies on AI search behaviour. Use it as a source of hypotheses and of industry sentiment. Apply the same test as to every vendor publication: is this a sampled measurement they are showing their working for, or an assertion about mechanics nobody outside the provider can see.", "duration": "25 minutes", "integrity": "subsidiary", "publisher": "www.semrush.com"}, {"url": "https://blog.google/products-and-platforms/products/search/generative-ai-google-search-may-2024/", "type": "webpage", "title": "Generative AI in Search", "summary": "Google's own announcement of generative answers in Search, and the right primary source for what the company said it was building and why. Read it rather than the coverage of it: the framing Google chose — letting Google do the searching for you — states the strategic intent more plainly than any analysis since.", "duration": "15 minutes", "integrity": "standalone", "publisher": "blog.google"}, {"url": "https://blog.google/products-and-platforms/products/search/ai-mode-search/", "type": "webpage", "title": "AI Mode in Search", "summary": "The full conversational search surface, distinct from AI Overviews appearing above traditional results. Read after the generative-AI announcement. The distinction matters operationally: Overviews sit on a results page that still has links below, while AI Mode is a different surface with different behaviour, and advice that conflates them will be wrong about one of them.", "duration": "15 minutes", "integrity": "dependent", "publisher": "blog.google"}, {"url": "https://commons.wikimedia.org/wiki/Category:Search_engines", "type": "image", "title": "Search engine imagery", "summary": "Freely reusable screenshots and diagrams of search interfaces over time. Optional, and genuinely useful for one purpose: putting a results page from ten years ago next to one from today makes the argument about the shrinking link layer faster and more convincingly than any statistic.", "duration": "10 minutes", "integrity": "subsidiary", "publisher": "commons.wikimedia.org"}, {"url": "https://commons.wikimedia.org/wiki/Category:Google_Search", "type": "image", "title": "Google Search interface imagery", "summary": "Screenshots of Google's interface across its history, free to reuse. Optional. Useful when building a deck: the visual record of how much of the page has been progressively occupied by answers rather than links is the single most persuasive slide in most presentations on this subject.", "duration": "10 minutes", "integrity": "subsidiary", "publisher": "commons.wikimedia.org"}, {"url": "https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics", "type": "webpage", "title": "Retrieval in practice: the server-rendering trap", "summary": "The most common and most expensive technical failure in this field, documented in Google's own crawling guidance: a single-page application that renders content client-side serves an empty shell to anything that does not execute JavaScript. Googlebot does render; many AI crawlers do not. A site can therefore be perfectly visible in Search and completely invisible to a model fetching the page directly. Test by fetching your own URL with curl and reading what comes back.", "duration": "20 minutes", "integrity": "dependent", "publisher": "developers.google.com"}], "journeys": [{"name": "The Complete Course", "steps": [{"order": 1, "title": "Search engine optimization", "channel": "Reference", "summary": "The field's full history and scope, from the mid-1990s onward, including the long arms race between optimisation and anti-spam systems. Read it for the pattern rather than the tactics: every technique that worked mechanically was eventually detected and neutralised, and the only durable strategy has been to be genuinely the best answer. That pattern is the single most useful prior to carry into the GEO chapters.", "sourceUrl": "https://en.wikipedia.org/wiki/Search_engine_optimization", "editorNote": null, "sourceType": "webpage"}, {"order": 2, "title": "Web crawler", "channel": "Reference", "summary": "How discovery actually works: a frontier of URLs, politeness rules, revisit policies, and the economics of deciding what to fetch and how often. Read after the SEO overview. The concept to take away is crawl budget — crawling is not free, so a crawler decides how much of your site is worth its time, and a site that wastes that budget on duplicates gets less of its real content seen.", "sourceUrl": "https://en.wikipedia.org/wiki/Web_crawler", "editorNote": null, "sourceType": "webpage"}, {"order": 3, "title": "Google Search", "channel": "Reference", "summary": "The product itself and how it has changed: the index, the ranking signals, the long sequence of named updates, and the progressive addition of answer surfaces above the links. Useful mainly for scale and context — the point to absorb is that results pages have been getting less link-shaped for fifteen years, and generative answers are a continuation rather than a break.", "sourceUrl": "https://en.wikipedia.org/wiki/Google_Search", "editorNote": null, "sourceType": "webpage"}, {"order": 4, "title": "PageRank", "channel": "Reference", "summary": "The original link-based ranking algorithm, and worth understanding as an idea rather than as current mechanics: importance inferred from the structure of citations rather than from the content itself. Modern ranking uses hundreds of signals and PageRank is one small input, but the underlying intuition — that what others point at tells you something content alone cannot — is exactly the intuition behind why mentions matter for retrieval too.", "sourceUrl": "https://en.wikipedia.org/wiki/PageRank", "editorNote": null, "sourceType": "webpage"}, {"order": 5, "title": "Search engine results page", "channel": "Reference", "summary": "The anatomy of the page itself: organic results, ads, knowledge panels, featured snippets, local packs, and now generative answers. Optional, and useful for a concrete sense of how little of a modern results page is actually the ten blue links. This is the visual context for every click-through statistic in chapter six.", "sourceUrl": "https://en.wikipedia.org/wiki/Search_engine_results_page", "editorNote": null, "sourceType": "webpage"}, {"order": 6, "title": "Google Search Essentials", "channel": "Google Search Central", "summary": "The shortest statement of what Google requires, and the right first read. Three parts: technical requirements for a page to be eligible at all, spam policies describing what gets you removed, and key best practices. Everything else in Google's documentation elaborates on this page. If a tactic contradicts it, the tactic is wrong.", "sourceUrl": "https://developers.google.com/search/docs/essentials", "editorNote": null, "sourceType": "webpage"}, {"order": 7, "title": "SEO Starter Guide", "channel": "Google Search Central", "summary": "Google's own introduction, which is notably more restrained than most paid courses on the same material. Covers getting indexed, helping Google understand content, managing crawling, and organising a site. Read after Search Essentials. Worth noting what it does not contain: no keyword density, no word counts, no posting frequency targets.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/seo-starter-guide", "editorNote": null, "sourceType": "webpage"}, {"order": 8, "title": "Sitemaps", "channel": "Google Search Central", "summary": "A machine-readable list of your URLs with optional metadata about when each changed. It does not guarantee indexing and does not improve ranking; it improves discovery, which matters most for large sites, new sites, and pages that are poorly linked internally. Practical note: a sitemap listing URLs that return errors or redirects actively wastes crawl budget.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview", "editorNote": null, "sourceType": "webpage"}, {"order": 9, "title": "Canonicalisation", "channel": "Google Search Central", "summary": "How to tell a search engine which URL is the real one when the same content is reachable several ways — tracking parameters, http and https, trailing slashes, print views. Unresolved duplication splits signals across versions and leaves the engine to pick for you. One of the most common and least visible technical faults on commercial sites.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls", "editorNote": null, "sourceType": "webpage"}, {"order": 10, "title": "Page experience", "channel": "Google Search Central", "summary": "Google's framing of loading performance, interactivity, visual stability, mobile usability and HTTPS. Read it with the proportion Google itself gives it: these are tie-breakers between comparable pages rather than primary ranking forces. Fixing a slow site is worth doing; expecting it to outrank better content is not.", "sourceUrl": "https://developers.google.com/search/docs/appearance/page-experience", "editorNote": null, "sourceType": "webpage"}, {"order": 11, "title": "Web Vitals", "channel": "Standards and Specifications", "summary": "Google's performance metrics defined precisely — what each measures, the thresholds, and how to improve them. Read for the measurement discipline rather than the targets: this is a rare case where the industry has agreed definitions, so performance arguments can be settled with numbers instead of opinions.", "sourceUrl": "https://web.dev/articles/vitals", "editorNote": null, "sourceType": "webpage"}, {"order": 12, "title": "Creating helpful, reliable, people-first content", "channel": "Google Search Central", "summary": "Google's guidance on quality, structured as questions to ask about your own work: does it provide original information, does it demonstrate first-hand expertise, would someone bookmark it. The most useful section is the self-assessment list, which is uncomfortable to run honestly against most commercial content. This is also the document that introduced the phrase people-first.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/creating-helpful-content", "editorNote": null, "sourceType": "webpage"}, {"order": 13, "title": "Search Quality Rater Guidelines", "channel": "Google Search Central", "summary": "The full document Google gives the human raters who evaluate search results, published openly and running to well over a hundred pages. This is where E-E-A-T is actually defined, along with Your Money or Your Life topics and the page quality scale. Read after the helpful-content guidance. Critical caveat the document states itself: raters do not directly affect rankings — their assessments are used to evaluate changes to the systems that do.", "sourceUrl": "https://guidelines.raterhub.com/searchqualityevaluatorguidelines.pdf", "editorNote": null, "sourceType": "pdf"}, {"order": 14, "title": "Backlinks", "channel": "Reference", "summary": "What links are as a ranking signal, how link schemes work, and why they remain both effective and dangerous. The honest position: links still matter, buying them violates Google's spam policies, and most link-building services sell exactly what the policies prohibit. Read it before approving any agency proposal containing the word outreach.", "sourceUrl": "https://en.wikipedia.org/wiki/Backlink", "editorNote": null, "sourceType": "webpage"}, {"order": 15, "title": "Yoast: the complete guide to SEO", "channel": "Practitioner Guides", "summary": "A long free guide covering keyword research, search intent, and the actual craft of writing and editing for search -- the practitioner half that Google's documentation deliberately leaves out. Read it after Google's own pages rather than before, so you can tell which parts are Google's position and which are the industry's interpretation of it. Its section on using AI as a guide rather than a content factory is unusually sober for a vendor.", "sourceUrl": "https://yoast.com/complete-guide-seo-copywriting/", "editorNote": null, "sourceType": "webpage"}, {"order": 16, "title": "Google's spam policies", "channel": "Practitioner Guides", "summary": "The enforcement side of quality, and far more concrete than the guidance on helpful content. It names the specific practices that get a site demoted or removed: link spam, scaled content abuse, cloaking, doorway pages, expired domain abuse and site reputation abuse. Read it before approving any agency proposal, because several of these describe services that are still openly sold.", "sourceUrl": "https://developers.google.com/search/docs/essentials/spam-policies", "editorNote": null, "sourceType": "webpage"}, {"order": 17, "title": "Introduction to structured data", "channel": "Google Search Central", "summary": "Google's explanation of what markup does and does not do. The essential framing: structured data makes a page eligible for specific result features, it does not improve ranking, and marking up content that is not visible on the page is a policy violation. Start here before touching schema.org itself, because the vocabulary is vast and only a fraction of it is actually consumed.", "sourceUrl": "https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data", "editorNote": null, "sourceType": "webpage"}, {"order": 18, "title": "Schema.org vocabulary", "channel": "Standards and Specifications", "summary": "The vocabulary itself: hundreds of types and thousands of properties, maintained collaboratively and used far beyond search. Browse it rather than reading it. The practical approach is to find the single type that matches your page, implement that properly, and ignore the rest — over-marking is a common and counterproductive instinct.", "sourceUrl": "https://schema.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 19, "title": "JSON-LD", "channel": "Reference", "summary": "The syntax structured data is actually written in: JSON with a context that maps keys to a shared vocabulary, embedded in a script tag so it does not affect rendering. The practical advantage over older formats is that it sits in one block rather than being woven through the HTML, which means it can be generated and validated independently of the page template.", "sourceUrl": "https://en.wikipedia.org/wiki/JSON-LD", "editorNote": null, "sourceType": "webpage"}, {"order": 20, "title": "JSON-LD 1.1 specification", "channel": "Standards and Specifications", "summary": "The W3C standard itself, for anyone implementing rather than specifying. Dense and authoritative. Worth having to hand for the parts that trip people up in practice — context resolution, @id and @type semantics, and how nesting actually serialises.", "sourceUrl": "https://www.w3.org/TR/json-ld11/", "editorNote": null, "sourceType": "webpage"}, {"order": 21, "title": "Structured data search gallery", "channel": "Google Search Central", "summary": "The definitive list of which markup types Google actually supports and what each produces in results — articles, breadcrumbs, events, FAQs, products, reviews, videos and the rest. Read after the introduction. This is the page that answers the real question: of the hundreds of schema.org types, which ones does anybody actually read.", "sourceUrl": "https://developers.google.com/search/docs/appearance/structured-data/search-gallery", "editorNote": null, "sourceType": "webpage"}, {"order": 22, "title": "Rich Results Test", "channel": "Google Search Central", "summary": "Google's own validator: paste a URL or code and see exactly which rich result types the page qualifies for and which errors block it. Practical rather than theoretical, and the fastest way to settle an argument about whether markup is working. Use it before and after any structured data change.", "sourceUrl": "https://search.google.com/test/rich-results", "editorNote": null, "sourceType": "webpage"}, {"order": 23, "title": "Generative AI in Search", "channel": "Google Blog", "summary": "Google's own announcement of generative answers in Search, and the right primary source for what the company said it was building and why. Read it rather than the coverage of it: the framing Google chose — letting Google do the searching for you — states the strategic intent more plainly than any analysis since.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/generative-ai-google-search-may-2024/", "editorNote": null, "sourceType": "webpage"}, {"order": 24, "title": "AI Mode in Search", "channel": "Google Blog", "summary": "The full conversational search surface, distinct from AI Overviews appearing above traditional results. Read after the generative-AI announcement. The distinction matters operationally: Overviews sit on a results page that still has links below, while AI Mode is a different surface with different behaviour, and advice that conflates them will be wrong about one of them.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/ai-mode-search/", "editorNote": null, "sourceType": "webpage"}, {"order": 25, "title": "ChatGPT", "channel": "Reference", "summary": "The product that made conversational answers mainstream, and now a search surface in its own right with live browsing. Relevant here for one structural reason: a question answered in a chat window has no results page, no position one, and no impression count — which is why the entire measurement apparatus of SEO does not transfer.", "sourceUrl": "https://en.wikipedia.org/wiki/ChatGPT", "editorNote": null, "sourceType": "webpage"}, {"order": 26, "title": "Gemini", "channel": "Reference", "summary": "Google's model family and assistant, and the system Google-Extended governs your content's use in. Worth keeping distinct from AI Overviews in Search: they are different surfaces with different controls, and conflating them leads people to block the wrong thing. Optional, and the clearest illustration of why the crawler chapter matters.", "sourceUrl": "https://en.wikipedia.org/wiki/Gemini_(chatbot)", "editorNote": null, "sourceType": "webpage"}, {"order": 27, "title": "Perplexity", "channel": "Reference", "summary": "The product built from the start around cited answers rather than retrofitting them, which makes it the clearest working example of what a citation-first engine looks like. Study its output format specifically: numbered inline citations, a visible source list, and follow-up questions. That shape is what content is being selected into.", "sourceUrl": "https://en.wikipedia.org/wiki/Perplexity_AI", "editorNote": null, "sourceType": "webpage"}, {"order": 28, "title": "Featured snippets", "channel": "Reference", "summary": "The precursor everybody forgot. Google has been extracting direct answers onto results pages since 2014, and the entire zero-click debate was rehearsed then with the same arguments and much the same data. Useful historical grounding: the answer layer is not new, it has become general rather than appearing for the first time.", "sourceUrl": "https://en.wikipedia.org/wiki/Featured_snippet", "editorNote": null, "sourceType": "webpage"}, {"order": 29, "title": "Pew: users click less when an AI summary appears", "channel": "Research and Independent Data", "summary": "The headline independent study, July 2025. Method: browsing data from 900 US adults on Pew's KnowledgePanel, 68,879 searches in March 2025, of which about 18% produced an AI Overview. Findings: 8% click rate on traditional results with a summary present against 15% without; 1% of visits included a click on a source cited inside the summary; 26% of sessions ended without any click against 16%. Read the methodology section, because the sample and the period are the whole basis of the critique.", "sourceUrl": "https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "editorNote": null, "sourceType": "webpage"}, {"order": 30, "title": "Reuters Institute Digital News Report", "channel": "Research and Independent Data", "summary": "The annual survey of news consumption across dozens of markets, run by Oxford's Reuters Institute and the most methodologically serious recurring dataset on how people actually find information. Read after Pew for scale and trend rather than a single snapshot. For publishers specifically, this is the document to argue from.", "sourceUrl": "https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2026", "editorNote": null, "sourceType": "webpage"}, {"order": 31, "title": "AI features and your website", "channel": "Google Search Central", "summary": "Google's reference page on how AI Overviews and AI Mode interact with sites, including the eligibility requirement that a page be indexed and able to show a snippet. Read alongside the Pew findings as the other side of the argument: Google's stated position is that AI features surface a wider range of sources and that AI Overviews only appear where they add to classic Search.", "sourceUrl": "https://developers.google.com/search/docs/appearance/ai-features", "editorNote": null, "sourceType": "webpage"}, {"order": 32, "title": "Search engine imagery", "channel": "Wikimedia Commons", "summary": "Freely reusable screenshots and diagrams of search interfaces over time. Optional, and genuinely useful for one purpose: putting a results page from ten years ago next to one from today makes the argument about the shrinking link layer faster and more convincingly than any statistic.", "sourceUrl": "https://commons.wikimedia.org/wiki/Category:Search_engines", "editorNote": null, "sourceType": "image"}, {"order": 33, "title": "GEO: Generative Engine Optimization (KDD 2024)", "channel": "Research and Independent Data", "summary": "The paper that named the field, by Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan and Deshpande, presented at KDD 2024. It proposes GEO as a black-box optimisation framework, introduces GEO-bench across multiple domains, and reports visibility improvements of up to 40%. Two things to hold: that figure is measured against the authors' own visibility metric on their own benchmark, and the paper itself reports that effectiveness varies substantially by domain. It is the only peer-reviewed foundation the field has, and it is routinely cited by people who have not read it.", "sourceUrl": "https://arxiv.org/abs/2311.09735", "editorNote": null, "sourceType": "webpage"}, {"order": 34, "title": "Answer engine optimization", "channel": "Reference", "summary": "The companion term, used broadly interchangeably with GEO in industry while sometimes being distinguished as focusing on direct-answer surfaces specifically. Read it mainly to inoculate yourself: the proliferation of near-synonymous acronyms in this field is a marketing phenomenon rather than a conceptual one, and the underlying practices are largely the same.", "sourceUrl": "https://en.wikipedia.org/wiki/Answer_engine_optimization", "editorNote": null, "sourceType": "webpage"}, {"order": 35, "title": "Top ways to ensure your content performs well in AI experiences", "channel": "Google Search Central", "summary": "Google's May 2025 blog post, and the earliest clear statement of its position: focus on unique, satisfying content for people rather than on anything AI-specific. Useful chronologically — it shows the advice was consistent a full year before the formal optimisation guide, which undercuts the idea that the rules changed suddenly and nobody told you.", "sourceUrl": "https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search", "editorNote": null, "sourceType": "webpage"}, {"order": 36, "title": "Ahrefs: technical AEO", "channel": "Practitioner Guides", "summary": "A practitioner guide to the technical side of answer-engine optimisation, and one of the more grounded pieces of vendor content in the field — it distinguishes what is demonstrated from what is inferred, which most does not. Read as a counterpart to the academic paper: this is what the ideas look like when someone has to implement them on Monday.", "sourceUrl": "https://ahrefs.com/aeo/technical-aeo", "editorNote": null, "sourceType": "webpage"}, {"order": 37, "title": "A new resource for optimizing for generative AI", "channel": "Google Search Central", "summary": "The May 2026 announcement of the optimisation guide. Short, and worth reading for the framing rather than the content: Google chose to publish a dedicated AI optimisation resource whose central message is that you do not need AI-specific optimisation. That tension is the most informative thing about the document.", "sourceUrl": "https://developers.google.com/search/blog/2026/05/a-new-resource-for-optimizing", "editorNote": null, "sourceType": "webpage"}, {"order": 38, "title": "Retrieval-augmented generation", "channel": "Reference", "summary": "The architecture behind every cited AI answer, and the most important technical concept in this course. The model does not recall your page; a retrieval step fetches candidate documents at question time and the model generates conditioned on them. The whole of practical GEO reduces to being retrievable and being selected at that step — everything else is downstream of this one mechanism.", "sourceUrl": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation", "editorNote": null, "sourceType": "webpage"}, {"order": 39, "title": "Large language models", "channel": "Reference", "summary": "What the model is and how training works, at the level needed to reason about this field properly. The decisive fact for marketers: training is a batch process over a fixed corpus, so a model cannot learn about your rebrand by you publishing a page. If your subject matters to a model's baseline knowledge rather than its retrieval, the lever is time and corpus presence, not optimisation.", "sourceUrl": "https://en.wikipedia.org/wiki/Large_language_model", "editorNote": null, "sourceType": "webpage"}, {"order": 40, "title": "Common Crawl", "channel": "Crawlers and Infrastructure", "summary": "The open repository of web crawl data that has underpinned a great deal of model training, and the most concrete answer to where does training data come from. Practically useful: if you want to understand whether your content plausibly entered a training corpus, this is the dataset to reason about rather than speculating.", "sourceUrl": "https://commoncrawl.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 41, "title": "Retrieval in practice: the server-rendering trap", "channel": "Practice Notes", "summary": "The most common and most expensive technical failure in this field, documented in Google's own crawling guidance: a single-page application that renders content client-side serves an empty shell to anything that does not execute JavaScript. Googlebot does render; many AI crawlers do not. A site can therefore be perfectly visible in Search and completely invisible to a model fetching the page directly. Test by fetching your own URL with curl and reading what comes back.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics", "editorNote": null, "sourceType": "webpage"}, {"order": 42, "title": "Hallucination in AI", "channel": "Reference", "summary": "Why generated answers about your business can be confidently wrong, and why that is a content problem as much as a model problem. Where the record is thin, contradictory or out of date, the model fills the gap plausibly. The practical implication is defensive: a clear, current, unambiguous statement of basic facts about you reduces the space in which invention happens.", "sourceUrl": "https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)", "editorNote": null, "sourceType": "webpage"}, {"order": 43, "title": "Google crawlers overview", "channel": "Google Search Central", "summary": "Every user agent Google operates and what each is for, including the distinction that matters most here: Googlebot crawls for Search, while Google-Extended is a separate control for whether your content is used in Gemini and other AI products. Blocking Google-Extended does not remove you from Search, and blocking Googlebot removes you from everything. Verify the user agents from this page rather than from a blog.", "sourceUrl": "https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers", "editorNote": null, "sourceType": "webpage"}, {"order": 44, "title": "OpenAI's crawlers", "channel": "Crawlers and Infrastructure", "summary": "OpenAI's own documentation of its user agents and what each does. The essential detail is that they are distinct: one for training-data collection, one for live fetches triggered by a user's question, and one for search indexing. Blocking the training crawler while permitting the user-triggered one is a coherent and fairly common position, and it is only possible if you read this page rather than guessing.", "sourceUrl": "https://developers.openai.com/api/docs/bots", "editorNote": null, "sourceType": "webpage"}, {"order": 45, "title": "Anthropic's web crawler", "channel": "Crawlers and Infrastructure", "summary": "Anthropic's documentation on its crawling and how site owners can control it. Read alongside OpenAI's for the comparison: the providers have converged on broadly similar conventions — named user agents, robots.txt compliance, published ranges — without any formal standard requiring it. That voluntary convergence is the entire basis of current control.", "sourceUrl": "https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler", "editorNote": null, "sourceType": "webpage"}, {"order": 46, "title": "Perplexity's crawlers", "channel": "Crawlers and Infrastructure", "summary": "Perplexity's documentation of its user agents, including the distinction between its index crawler and fetches made in response to a live user question. Relevant because Perplexity is the purest citation-driven product, so understanding what it fetches and when is the cleanest window into retrieval behaviour generally.", "sourceUrl": "https://docs.perplexity.ai/docs/resources/perplexity-crawlers", "editorNote": null, "sourceType": "webpage"}, {"order": 47, "title": "Introduction to robots.txt", "channel": "Google Search Central", "summary": "How to control crawling, and the limits of doing so. Two points people get wrong constantly: robots.txt controls crawling rather than indexing, so a disallowed URL can still appear in results if linked from elsewhere, and the file is a request that well-behaved crawlers honour rather than an access control. If something must not be public, authentication is the answer, not a disallow line.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/robots/intro", "editorNote": null, "sourceType": "webpage"}, {"order": 48, "title": "robots.txt", "channel": "Reference", "summary": "The convention itself, its history since 1994, and its genuine limits. Read alongside Google's documentation for the part Google does not emphasise: compliance is voluntary, there is no enforcement, and the growth of AI crawling has put real strain on a protocol built on good manners between a handful of search companies.", "sourceUrl": "https://en.wikipedia.org/wiki/Robots.txt", "editorNote": null, "sourceType": "webpage"}, {"order": 49, "title": "Semantic search", "channel": "Reference", "summary": "Matching on meaning rather than on literal terms, which is why keyword-stuffing stopped working and why writing naturally started to. Read after retrieval-augmented generation. The practical consequence is that you no longer need to guess exact phrasings — you need to cover a concept thoroughly enough that any phrasing of the question lands near your text.", "sourceUrl": "https://en.wikipedia.org/wiki/Semantic_search", "editorNote": null, "sourceType": "webpage"}, {"order": 50, "title": "Embeddings", "channel": "Reference", "summary": "How meaning becomes arithmetic: text mapped into a high-dimensional vector space where proximity approximates similarity. This is the mechanism underneath semantic retrieval, and understanding it explains a lot of otherwise mysterious behaviour — including why a passage on a page about something else can be retrieved while your dedicated page is not.", "sourceUrl": "https://en.wikipedia.org/wiki/Word_embedding", "editorNote": null, "sourceType": "webpage"}, {"order": 51, "title": "Vector databases", "channel": "Reference", "summary": "The infrastructure that makes similarity search fast at scale, using approximate nearest-neighbour indexes. Optional for marketers and essential for anyone implementing retrieval. Included because it makes concrete what retrieval is doing: your content is not being read, it is being measured for proximity to a question.", "sourceUrl": "https://en.wikipedia.org/wiki/Vector_database", "editorNote": null, "sourceType": "webpage"}, {"order": 52, "title": "Knowledge graphs", "channel": "Reference", "summary": "Entities and relationships held as structured facts rather than as prose, which is how a system knows your company and a person with the same name are different things. Relevant to citation because entity disambiguation determines whether answers about you are actually about you, and consistent naming and markup across the web is what makes that resolution work.", "sourceUrl": "https://en.wikipedia.org/wiki/Knowledge_graph", "editorNote": null, "sourceType": "webpage"}, {"order": 53, "title": "The Robots Exclusion Protocol", "channel": "Standards and Specifications", "summary": "The original specification site for robots.txt, documenting a convention that predates almost everything else in this course and still governs who may read your content. Read it for the realisation that the entire economy of web crawling rests on a voluntary text file agreed informally in 1994.", "sourceUrl": "https://www.robotstxt.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 54, "title": "The llms.txt proposal", "channel": "Standards and Specifications", "summary": "The specification itself, and the thing to read before forming a view. It proposes a Markdown file at the site root giving a model a curated map of the site, by analogy with robots.txt and sitemap.xml. Note its own framing: it is a proposal, open for feedback, with no standards body behind it and no major AI provider having announced that they consume it for general web content.", "sourceUrl": "https://llmstxt.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 55, "title": "Ahrefs: 97% of llms.txt files never get read", "channel": "Research and Independent Data", "summary": "An analysis of 137,000 domains in Ahrefs' own web analytics, May 2026. Of those publishing a valid llms.txt, 97% received zero requests for it that month. Caveats worth stating: it is a vendor study, the sample is sites using one analytics product, and it measures whether files are fetched rather than whether they influence citation. Even with those caveats it is the largest public dataset on the question.", "sourceUrl": "https://ahrefs.com/blog/llmstxt-study/", "editorNote": null, "sourceType": "webpage"}, {"order": 56, "title": "SE Ranking: why llms.txt does not work", "channel": "Research and Independent Data", "summary": "A second independent analysis, roughly 300,000 domains, asking the different and more useful question: does having the file correlate with being cited by LLMs. It found no relationship. Read alongside Ahrefs and note that the two disagree on adoption — about 10% here against 28% there — which tells you something about how much sample construction drives results in this field.", "sourceUrl": "https://seranking.com/blog/llms-txt/", "editorNote": null, "sourceType": "webpage"}, {"order": 57, "title": "Ahrefs: what is llms.txt", "channel": "Practitioner Guides", "summary": "The explainer to read before the data, covering what the file is, what it proposes to do, and what adopting it involves. Honest about the uncertainty. Read it, then read the two studies — the sequence of understanding the proposal and then seeing the evidence against it is the most instructive ten minutes in this course.", "sourceUrl": "https://ahrefs.com/blog/what-is-llms-txt/", "editorNote": null, "sourceType": "webpage"}, {"order": 58, "title": "Search Off the Record (podcast)", "channel": "Google Search Central", "summary": "Google's own Search Relations podcast, where Mueller, Splitt and Illyes discuss these questions at length and with more candour than the documentation allows. The best listening in the field, and the place where the llms.txt scepticism was argued through rather than merely stated. Audio, and genuinely useful on a commute.", "sourceUrl": "https://developers.google.com/search/podcasts/search-off-the-record", "editorNote": null, "sourceType": "audio"}, {"order": 59, "title": "Google's guide to optimizing for generative AI features", "channel": "Google Search Central", "summary": "The keystone document of this entire course, published May 2026. Its actual statements: best practices for SEO continue to be relevant because AI features are built on core Search systems; you do not need to create new machine readable files, AI text files, markup, or Markdown; there is no requirement to break content into small pieces; structured data is not required for generative AI features though still recommended generally; Search ignores llms.txt and creating one will neither harm nor help; and no third-party tool has access to Google's internal ranking or AI systems. Read it in full before buying anything.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/ai-optimization-guide", "editorNote": null, "sourceType": "webpage"}, {"order": 60, "title": "Snippets and how to control them", "channel": "Google Search Central", "summary": "The mechanics behind eligibility for AI features, since a page must be able to show a snippet to be used. Covers nosnippet, data-nosnippet and max-snippet, which let you exclude specific passages rather than whole pages. The underrated control here is data-nosnippet, which can keep boilerplate or paywalled previews out of generated summaries without affecting the rest of the page.", "sourceUrl": "https://developers.google.com/search/docs/appearance/snippet", "editorNote": null, "sourceType": "webpage"}, {"order": 61, "title": "The Search generative AI control", "channel": "Google Search Central", "summary": "Search Console's setting for excluding your site's links and content from AI Overviews, AI Mode and Discover's generative features. Note what it costs: exclusion applies to the generative surfaces specifically, takes a few days to take effect, and is a genuine trade-off rather than a free safety measure. Worth knowing it exists even if you never use it.", "sourceUrl": "https://support.google.com/webmasters/answer/16908024", "editorNote": null, "sourceType": "webpage"}, {"order": 62, "title": "Google's approach to website controls for AI features", "channel": "Google Search Central", "summary": "The policy reasoning behind those controls, from Google's own blog. Useful for anyone who has to explain to a board why the company is or is not visible in AI answers: this is the official account of what site owners can and cannot decide, in Google's words rather than a commentator's.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/search-ai-features-controls/", "editorNote": null, "sourceType": "webpage"}, {"order": 63, "title": "Google Search Central (video)", "channel": "Google Search Central", "summary": "The official video channel: office-hours recordings, explainers and conference talks from the Search Relations team. Video, and the most efficient way to absorb the reasoning behind a documented position when the document itself is terse.", "sourceUrl": "https://www.youtube.com/@GoogleSearchCentral", "editorNote": null, "sourceType": "video"}, {"order": 64, "title": "SE Ranking: how to optimise for ChatGPT", "channel": "Practitioner Guides", "summary": "A practitioner guide grounded in the finding that community platforms are disproportionately represented in AI citations — which leads to the genuinely useful and slightly uncomfortable conclusion that earning a reputation on forums where people actually discuss your category may do more than another page on your own site.", "sourceUrl": "https://seranking.com/blog/how-to-optimize-for-chatgpt/", "editorNote": null, "sourceType": "webpage"}, {"order": 65, "title": "Semrush blog", "channel": "Practitioner Guides", "summary": "A large library of research posts and guides, including periodic original data studies on AI search behaviour. Use it as a source of hypotheses and of industry sentiment. Apply the same test as to every vendor publication: is this a sampled measurement they are showing their working for, or an assertion about mechanics nobody outside the provider can see.", "sourceUrl": "https://www.semrush.com/blog/", "editorNote": null, "sourceType": "webpage"}, {"order": 66, "title": "Bing Webmaster Guidelines", "channel": "Standards and Specifications", "summary": "The other major index, and worth reading precisely because it is not Google. Bing's guidelines are more explicit in places about what it values, and Bing's index underpins other products — which makes it disproportionately important relative to its direct search share.", "sourceUrl": "https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a", "editorNote": null, "sourceType": "webpage"}, {"order": 67, "title": "Ecommerce SEO guidance", "channel": "Google Search Central", "summary": "Google's specialty documentation for product sites, included because ecommerce is where structured data genuinely earns its keep — product, price, availability and review markup feed shopping surfaces directly. If you sell things, this page is worth more than any amount of general advice about content strategy.", "sourceUrl": "https://developers.google.com/search/docs/specialty/ecommerce", "editorNote": null, "sourceType": "webpage"}, {"order": 68, "title": "IndexNow", "channel": "Standards and Specifications", "summary": "A protocol letting sites notify participating search engines immediately when content changes, supported by Bing and Yandex among others. Practical and underused: for sites where freshness matters, this converts discovery from polling to pushing. Google has not adopted it, which is the honest caveat.", "sourceUrl": "https://www.indexnow.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 69, "title": "Search generative AI performance reports", "channel": "Google Search Central", "summary": "Google's June 2026 announcement of reporting that shows how often your URLs appeared in generative AI features across Search and Discover. The most significant measurement development in the field, because until this existed there was no first-party data at all. Note the caveat in the announcement: it rolled out to a subset of sites first, so absence of the report is not absence of impressions.", "sourceUrl": "https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports", "editorNote": null, "sourceType": "webpage"}, {"order": 70, "title": "Getting started with Search Console", "channel": "Google Search Central", "summary": "The tool itself, free, and the only source of first-party data about how Google sees your site: indexing status, queries, impressions, clicks, structured data errors. Read after the generative reports announcement. Anyone discussing their search performance without reference to this is speculating.", "sourceUrl": "https://developers.google.com/search/docs/monitor-debug/search-console-start", "editorNote": null, "sourceType": "webpage"}, {"order": 71, "title": "Schema Markup Validator", "channel": "Standards and Specifications", "summary": "The vocabulary-level validator, as distinct from Google's rich results test: this one checks whether your markup is valid schema.org regardless of whether Google happens to use that type. The right tool when you are marking up for consumers other than Google Search, which is increasingly the point.", "sourceUrl": "https://validator.schema.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 72, "title": "Google Search interface imagery", "channel": "Wikimedia Commons", "summary": "Screenshots of Google's interface across its history, free to reuse. Optional. Useful when building a deck: the visual record of how much of the page has been progressively occupied by answers rather than links is the single most persuasive slide in most presentations on this subject.", "sourceUrl": "https://commons.wikimedia.org/wiki/Category:Google_Search", "editorNote": null, "sourceType": "image"}, {"order": 73, "title": "Profound", "channel": "The Tooling Market", "summary": "The best-known platform in the category. Its published capabilities include prompt-volume data, monitoring of how models describe a brand, citation and sentiment tracking, agent analytics on which bots fetch which pages, benchmarking against competitors, and content tooling. It also runs a research hub publishing original analyses. Evaluate it as a sampling and measurement product, which is what it is, rather than as a channel into anybody's ranking system.", "sourceUrl": "https://www.tryprofound.com/", "editorNote": null, "sourceType": "webpage"}, {"order": 74, "title": "Scrunch AI", "channel": "The Tooling Market", "summary": "An enterprise-positioned competitor focused on how brands appear across AI platforms and on optimising the experience AI agents have of a site. Read after Profound for the comparison. The category is consolidating, so check the current ownership and status of any vendor before committing — this market has moved faster than its own marketing pages.", "sourceUrl": "https://scrunch.com/", "editorNote": null, "sourceType": "webpage"}, {"order": 75, "title": "Otterly.AI", "channel": "The Tooling Market", "summary": "A smaller, more accessible monitoring tool tracking brand mentions and links across AI search platforms. Included to show the range of the market: the same core technique — repeated prompting and recording what comes back — is sold at wildly different price points, and the differences are largely in coverage, frequency and reporting rather than in method.", "sourceUrl": "https://otterly.ai/", "editorNote": null, "sourceType": "webpage"}, {"order": 76, "title": "AthenaHQ", "channel": "The Tooling Market", "summary": "Another entrant in the AI-visibility monitoring space, positioned around tracking and improving how brands surface across multiple models. Optional, and worth a look mainly for how similar the category's propositions have become — a useful signal in itself about how undifferentiated the underlying technique is.", "sourceUrl": "https://athenahq.ai/", "editorNote": null, "sourceType": "webpage"}, {"order": 77, "title": "Peec AI", "channel": "The Tooling Market", "summary": "A European entrant in the same category, frequently compared against the others on price and model coverage. Included for completeness of the landscape. If you are running a procurement exercise, the questions that actually separate these products are which models they sample, how often, in which markets, and whether they will show you the raw responses.", "sourceUrl": "https://peec.ai/", "editorNote": null, "sourceType": "webpage"}, {"order": 78, "title": "Search Engine Roundtable", "channel": "The Tooling Market", "summary": "Barry Schwartz's long-running daily record of what search representatives actually said and when. Unglamorous and invaluable: when a vendor claims Google confirmed something, this is where you check. The archive stretches back two decades, which makes it the best available defence against confidently misremembered history.", "sourceUrl": "https://www.seroundtable.com/", "editorNote": null, "sourceType": "webpage"}, {"order": 79, "title": "Cloudflare: no AI crawl without compensation", "channel": "Crawlers and Infrastructure", "summary": "Cloudflare's July 2025 announcement that it would block AI crawlers by default for new domains and build a marketplace for paid access. Whatever you think of it, this is the single most consequential infrastructure decision in the period this course covers — a company sitting in front of a large share of web traffic unilaterally changing the default from open to closed.", "sourceUrl": "https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/", "editorNote": null, "sourceType": "webpage"}, {"order": 80, "title": "Cloudflare bot documentation", "channel": "Crawlers and Infrastructure", "summary": "The operational detail behind the policy: how bots are identified, categorised and verified, and what controls are available. Read after the announcement. Practically important because for many sites the effective AI-access policy is set at the CDN rather than in robots.txt, and the two can silently disagree.", "sourceUrl": "https://developers.cloudflare.com/bots/concepts/bot/", "editorNote": null, "sourceType": "webpage"}, {"order": 81, "title": "web.dev", "channel": "Crawlers and Infrastructure", "summary": "Google's resource for web platform engineering, included here as the counterweight to the access debate: whatever the commercial arrangements become, the technical obligation stays the same — content must be reachable, fast and rendered where a machine can see it. Optional, and the best general reference for the engineering half of this work.", "sourceUrl": "https://web.dev/", "editorNote": null, "sourceType": "webpage"}, {"order": 82, "title": "GEO creates underexamined risks (position paper)", "channel": "Research and Independent Data", "summary": "A 2026 position paper arguing that the governance questions around generative engine optimisation — concentration of influence, disclosure of paid placement, and gaps in academic attention — matter more than the tactical questions the industry focuses on. The most useful single document for anyone who has to think about this at policy or board level rather than campaign level.", "sourceUrl": "https://arxiv.org/abs/2606.12439", "editorNote": null, "sourceType": "webpage"}, {"order": 83, "title": "Defending generative search against malicious GEO", "channel": "Research and Independent Data", "summary": "Research on adversarial optimisation: content deliberately engineered to capture generative answers, and the defences against it. Read after the position paper. This is the clearest signal of where the field goes next — the same detection-and-evasion dynamic that defined twenty years of search spam, now beginning on generative surfaces.", "sourceUrl": "https://arxiv.org/abs/2609.02964", "editorNote": null, "sourceType": "webpage"}, {"order": 84, "title": "Manipulation-resistant content scoring", "channel": "Research and Independent Data", "summary": "Work on scoring content for generative engines in a way that resists gaming, without needing access to the engine itself. Technical and early. Included because it points at the likely end state: if durable scoring becomes possible, the advantage shifts back to genuinely better content, exactly as it did in search after each wave of manipulation was closed off.", "sourceUrl": "https://arxiv.org/abs/2609.07559", "editorNote": null, "sourceType": "webpage"}, {"order": 85, "title": "Schema.org", "channel": "Reference", "summary": "The vocabulary's own history and governance, founded in 2011 by Google, Microsoft, Yahoo and Yandex. Included in the closing chapter deliberately: it is the last time the major search players agreed a shared standard for machine-readable content, and the obvious question for the next decade is whether anything comparable emerges for generative systems, or whether each provider simply builds its own.", "sourceUrl": "https://en.wikipedia.org/wiki/Schema.org", "editorNote": null, "sourceType": "webpage"}], "bestFor": "Best for becoming genuinely expert rather than briefed. All seventeen chapters in order: the fundamentals that still hold, the answer layer and what it did to clicks, the actual mechanics of training versus retrieval, the llms.txt evidence, what Google says to do, the tactics that survive scrutiny, measurement, the tooling market, the crawl economy, and the research frontier. Around twelve hours if you read the primary documents, which is the point. Nothing here is a summary of a summary.", "journeyId": "j1", "stepCount": 85}, {"name": "The Marketer's Route", "steps": [{"order": 1, "title": "Search engine optimization", "channel": "Reference", "summary": "The field's full history and scope, from the mid-1990s onward, including the long arms race between optimisation and anti-spam systems. Read it for the pattern rather than the tactics: every technique that worked mechanically was eventually detected and neutralised, and the only durable strategy has been to be genuinely the best answer. That pattern is the single most useful prior to carry into the GEO chapters.", "sourceUrl": "https://en.wikipedia.org/wiki/Search_engine_optimization", "editorNote": "The pattern to carry through everything: mechanical tricks get closed off.", "sourceType": "webpage"}, {"order": 2, "title": "Google Search Essentials", "channel": "Google Search Central", "summary": "The shortest statement of what Google requires, and the right first read. Three parts: technical requirements for a page to be eligible at all, spam policies describing what gets you removed, and key best practices. Everything else in Google's documentation elaborates on this page. If a tactic contradicts it, the tactic is wrong.", "sourceUrl": "https://developers.google.com/search/docs/essentials", "editorNote": "The floor. Nothing works if this is broken.", "sourceType": "webpage"}, {"order": 3, "title": "Creating helpful, reliable, people-first content", "channel": "Google Search Central", "summary": "Google's guidance on quality, structured as questions to ask about your own work: does it provide original information, does it demonstrate first-hand expertise, would someone bookmark it. The most useful section is the self-assessment list, which is uncomfortable to run honestly against most commercial content. This is also the document that introduced the phrase people-first.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/creating-helpful-content", "editorNote": "Run the self-assessment list honestly against your own content.", "sourceType": "webpage"}, {"order": 4, "title": "Introduction to structured data", "channel": "Google Search Central", "summary": "Google's explanation of what markup does and does not do. The essential framing: structured data makes a page eligible for specific result features, it does not improve ranking, and marking up content that is not visible on the page is a policy violation. Start here before touching schema.org itself, because the vocabulary is vast and only a fraction of it is actually consumed.", "sourceUrl": "https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data", "editorNote": "Markup makes you eligible for features. It does not make you rank.", "sourceType": "webpage"}, {"order": 5, "title": "Generative AI in Search", "channel": "Google Blog", "summary": "Google's own announcement of generative answers in Search, and the right primary source for what the company said it was building and why. Read it rather than the coverage of it: the framing Google chose — letting Google do the searching for you — states the strategic intent more plainly than any analysis since.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/generative-ai-google-search-may-2024/", "editorNote": null, "sourceType": "webpage"}, {"order": 6, "title": "Perplexity", "channel": "Reference", "summary": "The product built from the start around cited answers rather than retrofitting them, which makes it the clearest working example of what a citation-first engine looks like. Study its output format specifically: numbered inline citations, a visible source list, and follow-up questions. That shape is what content is being selected into.", "sourceUrl": "https://en.wikipedia.org/wiki/Perplexity_AI", "editorNote": "Look at an actual Perplexity answer. That shape is what you are optimising into.", "sourceType": "webpage"}, {"order": 7, "title": "Pew: users click less when an AI summary appears", "channel": "Research and Independent Data", "summary": "The headline independent study, July 2025. Method: browsing data from 900 US adults on Pew's KnowledgePanel, 68,879 searches in March 2025, of which about 18% produced an AI Overview. Findings: 8% click rate on traditional results with a summary present against 15% without; 1% of visits included a click on a source cited inside the summary; 26% of sessions ended without any click against 16%. Read the methodology section, because the sample and the period are the whole basis of the critique.", "sourceUrl": "https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "editorNote": "The headline numbers. Read the method before quoting them.", "sourceType": "webpage"}, {"order": 8, "title": "AI features and your website", "channel": "Google Search Central", "summary": "Google's reference page on how AI Overviews and AI Mode interact with sites, including the eligibility requirement that a page be indexed and able to show a snippet. Read alongside the Pew findings as the other side of the argument: Google's stated position is that AI features surface a wider range of sources and that AI Overviews only appear where they add to classic Search.", "sourceUrl": "https://developers.google.com/search/docs/appearance/ai-features", "editorNote": "And Google's rebuttal. Hold both.", "sourceType": "webpage"}, {"order": 9, "title": "GEO: Generative Engine Optimization (KDD 2024)", "channel": "Research and Independent Data", "summary": "The paper that named the field, by Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan and Deshpande, presented at KDD 2024. It proposes GEO as a black-box optimisation framework, introduces GEO-bench across multiple domains, and reports visibility improvements of up to 40%. Two things to hold: that figure is measured against the authors' own visibility metric on their own benchmark, and the paper itself reports that effectiveness varies substantially by domain. It is the only peer-reviewed foundation the field has, and it is routinely cited by people who have not read it.", "sourceUrl": "https://arxiv.org/abs/2311.09735", "editorNote": "Where the term came from. Note what the 40% is measured against.", "sourceType": "webpage"}, {"order": 10, "title": "Retrieval-augmented generation", "channel": "Reference", "summary": "The architecture behind every cited AI answer, and the most important technical concept in this course. The model does not recall your page; a retrieval step fetches candidate documents at question time and the model generates conditioned on them. The whole of practical GEO reduces to being retrievable and being selected at that step — everything else is downstream of this one mechanism.", "sourceUrl": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation", "editorNote": "The one mechanism worth understanding properly.", "sourceType": "webpage"}, {"order": 11, "title": "OpenAI's crawlers", "channel": "Crawlers and Infrastructure", "summary": "OpenAI's own documentation of its user agents and what each does. The essential detail is that they are distinct: one for training-data collection, one for live fetches triggered by a user's question, and one for search indexing. Blocking the training crawler while permitting the user-triggered one is a coherent and fairly common position, and it is only possible if you read this page rather than guessing.", "sourceUrl": "https://developers.openai.com/api/docs/bots", "editorNote": "Different bots, different purposes. Blocking one is not blocking all.", "sourceType": "webpage"}, {"order": 12, "title": "Semantic search", "channel": "Reference", "summary": "Matching on meaning rather than on literal terms, which is why keyword-stuffing stopped working and why writing naturally started to. Read after retrieval-augmented generation. The practical consequence is that you no longer need to guess exact phrasings — you need to cover a concept thoroughly enough that any phrasing of the question lands near your text.", "sourceUrl": "https://en.wikipedia.org/wiki/Semantic_search", "editorNote": "Why a focused page beats a long one: the unit of retrieval is the passage.", "sourceType": "webpage"}, {"order": 13, "title": "The llms.txt proposal", "channel": "Standards and Specifications", "summary": "The specification itself, and the thing to read before forming a view. It proposes a Markdown file at the site root giving a model a curated map of the site, by analogy with robots.txt and sitemap.xml. Note its own framing: it is a proposal, open for feedback, with no standards body behind it and no major AI provider having announced that they consume it for general web content.", "sourceUrl": "https://llmstxt.org/", "editorNote": "Read the proposal before anyone sells you an implementation.", "sourceType": "webpage"}, {"order": 14, "title": "Ahrefs: 97% of llms.txt files never get read", "channel": "Research and Independent Data", "summary": "An analysis of 137,000 domains in Ahrefs' own web analytics, May 2026. Of those publishing a valid llms.txt, 97% received zero requests for it that month. Caveats worth stating: it is a vendor study, the sample is sites using one analytics product, and it measures whether files are fetched rather than whether they influence citation. Even with those caveats it is the largest public dataset on the question.", "sourceUrl": "https://ahrefs.com/blog/llmstxt-study/", "editorNote": "97% of them never get fetched.", "sourceType": "webpage"}, {"order": 15, "title": "Google's guide to optimizing for generative AI features", "channel": "Google Search Central", "summary": "The keystone document of this entire course, published May 2026. Its actual statements: best practices for SEO continue to be relevant because AI features are built on core Search systems; you do not need to create new machine readable files, AI text files, markup, or Markdown; there is no requirement to break content into small pieces; structured data is not required for generative AI features though still recommended generally; Search ignores llms.txt and creating one will neither harm nor help; and no third-party tool has access to Google's internal ranking or AI systems. Read it in full before buying anything.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/ai-optimization-guide", "editorNote": "The keystone. If a tactic contradicts this page, drop the tactic.", "sourceType": "webpage"}, {"order": 16, "title": "SE Ranking: how to optimise for ChatGPT", "channel": "Practitioner Guides", "summary": "A practitioner guide grounded in the finding that community platforms are disproportionately represented in AI citations — which leads to the genuinely useful and slightly uncomfortable conclusion that earning a reputation on forums where people actually discuss your category may do more than another page on your own site.", "sourceUrl": "https://seranking.com/blog/how-to-optimize-for-chatgpt/", "editorNote": "Uncomfortable but well-evidenced: reputation off your own site matters.", "sourceType": "webpage"}, {"order": 17, "title": "Ecommerce SEO guidance", "channel": "Google Search Central", "summary": "Google's specialty documentation for product sites, included because ecommerce is where structured data genuinely earns its keep — product, price, availability and review markup feed shopping surfaces directly. If you sell things, this page is worth more than any amount of general advice about content strategy.", "sourceUrl": "https://developers.google.com/search/docs/specialty/ecommerce", "editorNote": "If you sell things, this is worth more than any content strategy.", "sourceType": "webpage"}, {"order": 18, "title": "Search generative AI performance reports", "channel": "Google Search Central", "summary": "Google's June 2026 announcement of reporting that shows how often your URLs appeared in generative AI features across Search and Discover. The most significant measurement development in the field, because until this existed there was no first-party data at all. Note the caveat in the announcement: it rolled out to a subset of sites first, so absence of the report is not absence of impressions.", "sourceUrl": "https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports", "editorNote": "First-party impression data. This is what you report on.", "sourceType": "webpage"}, {"order": 19, "title": "Google Search interface imagery", "channel": "Wikimedia Commons", "summary": "Screenshots of Google's interface across its history, free to reuse. Optional. Useful when building a deck: the visual record of how much of the page has been progressively occupied by answers rather than links is the single most persuasive slide in most presentations on this subject.", "sourceUrl": "https://commons.wikimedia.org/wiki/Category:Google_Search", "editorNote": null, "sourceType": "image"}, {"order": 20, "title": "Profound", "channel": "The Tooling Market", "summary": "The best-known platform in the category. Its published capabilities include prompt-volume data, monitoring of how models describe a brand, citation and sentiment tracking, agent analytics on which bots fetch which pages, benchmarking against competitors, and content tooling. It also runs a research hub publishing original analyses. Evaluate it as a sampling and measurement product, which is what it is, rather than as a channel into anybody's ranking system.", "sourceUrl": "https://www.tryprofound.com/", "editorNote": "A sampling product. Useful. Not a window into anyone's ranking system.", "sourceType": "webpage"}, {"order": 21, "title": "Cloudflare: no AI crawl without compensation", "channel": "Crawlers and Infrastructure", "summary": "Cloudflare's July 2025 announcement that it would block AI crawlers by default for new domains and build a marketplace for paid access. Whatever you think of it, this is the single most consequential infrastructure decision in the period this course covers — a company sitting in front of a large share of web traffic unilaterally changing the default from open to closed.", "sourceUrl": "https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/", "editorNote": "The commercial question that outranks every tactic here.", "sourceType": "webpage"}, {"order": 22, "title": "GEO creates underexamined risks (position paper)", "channel": "Research and Independent Data", "summary": "A 2026 position paper arguing that the governance questions around generative engine optimisation — concentration of influence, disclosure of paid placement, and gaps in academic attention — matter more than the tactical questions the industry focuses on. The most useful single document for anyone who has to think about this at policy or board level rather than campaign level.", "sourceUrl": "https://arxiv.org/abs/2606.12439", "editorNote": "And the governance question behind that.", "sourceType": "webpage"}], "bestFor": "Best for someone who owns the channel and needs to act on Monday. Skips the implementation detail and the research frontier, and concentrates on what changed, what the evidence actually supports, what Google says to do, which tactics survive scrutiny, and how to measure any of it. Ends at the vendor market with the questions worth asking before signing anything. Around three hours.", "journeyId": "j2", "stepCount": 22}, {"name": "The Technical Deep Dive", "steps": [{"order": 1, "title": "Web crawler", "channel": "Reference", "summary": "How discovery actually works: a frontier of URLs, politeness rules, revisit policies, and the economics of deciding what to fetch and how often. Read after the SEO overview. The concept to take away is crawl budget — crawling is not free, so a crawler decides how much of your site is worth its time, and a site that wastes that budget on duplicates gets less of its real content seen.", "sourceUrl": "https://en.wikipedia.org/wiki/Web_crawler", "editorNote": "Crawl budget is a real constraint, not a metaphor.", "sourceType": "webpage"}, {"order": 2, "title": "Google Search Essentials", "channel": "Google Search Central", "summary": "The shortest statement of what Google requires, and the right first read. Three parts: technical requirements for a page to be eligible at all, spam policies describing what gets you removed, and key best practices. Everything else in Google's documentation elaborates on this page. If a tactic contradicts it, the tactic is wrong.", "sourceUrl": "https://developers.google.com/search/docs/essentials", "editorNote": null, "sourceType": "webpage"}, {"order": 3, "title": "Sitemaps", "channel": "Google Search Central", "summary": "A machine-readable list of your URLs with optional metadata about when each changed. It does not guarantee indexing and does not improve ranking; it improves discovery, which matters most for large sites, new sites, and pages that are poorly linked internally. Practical note: a sitemap listing URLs that return errors or redirects actively wastes crawl budget.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/sitemaps/overview", "editorNote": null, "sourceType": "webpage"}, {"order": 4, "title": "Canonicalisation", "channel": "Google Search Central", "summary": "How to tell a search engine which URL is the real one when the same content is reachable several ways — tracking parameters, http and https, trailing slashes, print views. Unresolved duplication splits signals across versions and leaves the engine to pick for you. One of the most common and least visible technical faults on commercial sites.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls", "editorNote": "Unresolved duplication is the commonest invisible fault on commercial sites.", "sourceType": "webpage"}, {"order": 5, "title": "Web Vitals", "channel": "Standards and Specifications", "summary": "Google's performance metrics defined precisely — what each measures, the thresholds, and how to improve them. Read for the measurement discipline rather than the targets: this is a rare case where the industry has agreed definitions, so performance arguments can be settled with numbers instead of opinions.", "sourceUrl": "https://web.dev/articles/vitals", "editorNote": null, "sourceType": "webpage"}, {"order": 6, "title": "Search Quality Rater Guidelines", "channel": "Google Search Central", "summary": "The full document Google gives the human raters who evaluate search results, published openly and running to well over a hundred pages. This is where E-E-A-T is actually defined, along with Your Money or Your Life topics and the page quality scale. Read after the helpful-content guidance. Critical caveat the document states itself: raters do not directly affect rankings — their assessments are used to evaluate changes to the systems that do.", "sourceUrl": "https://guidelines.raterhub.com/searchqualityevaluatorguidelines.pdf", "editorNote": "Skim for the quality definitions; you need them to brief content people.", "sourceType": "pdf"}, {"order": 7, "title": "Introduction to structured data", "channel": "Google Search Central", "summary": "Google's explanation of what markup does and does not do. The essential framing: structured data makes a page eligible for specific result features, it does not improve ranking, and marking up content that is not visible on the page is a policy violation. Start here before touching schema.org itself, because the vocabulary is vast and only a fraction of it is actually consumed.", "sourceUrl": "https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data", "editorNote": null, "sourceType": "webpage"}, {"order": 8, "title": "Schema.org vocabulary", "channel": "Standards and Specifications", "summary": "The vocabulary itself: hundreds of types and thousands of properties, maintained collaboratively and used far beyond search. Browse it rather than reading it. The practical approach is to find the single type that matches your page, implement that properly, and ignore the rest — over-marking is a common and counterproductive instinct.", "sourceUrl": "https://schema.org/", "editorNote": "Find your one type. Implement it properly. Ignore the rest.", "sourceType": "webpage"}, {"order": 9, "title": "JSON-LD", "channel": "Reference", "summary": "The syntax structured data is actually written in: JSON with a context that maps keys to a shared vocabulary, embedded in a script tag so it does not affect rendering. The practical advantage over older formats is that it sits in one block rather than being woven through the HTML, which means it can be generated and validated independently of the page template.", "sourceUrl": "https://en.wikipedia.org/wiki/JSON-LD", "editorNote": null, "sourceType": "webpage"}, {"order": 10, "title": "JSON-LD 1.1 specification", "channel": "Standards and Specifications", "summary": "The W3C standard itself, for anyone implementing rather than specifying. Dense and authoritative. Worth having to hand for the parts that trip people up in practice — context resolution, @id and @type semantics, and how nesting actually serialises.", "sourceUrl": "https://www.w3.org/TR/json-ld11/", "editorNote": "The actual spec, for the parts that trip people up.", "sourceType": "webpage"}, {"order": 11, "title": "Structured data search gallery", "channel": "Google Search Central", "summary": "The definitive list of which markup types Google actually supports and what each produces in results — articles, breadcrumbs, events, FAQs, products, reviews, videos and the rest. Read after the introduction. This is the page that answers the real question: of the hundreds of schema.org types, which ones does anybody actually read.", "sourceUrl": "https://developers.google.com/search/docs/appearance/structured-data/search-gallery", "editorNote": null, "sourceType": "webpage"}, {"order": 12, "title": "Rich Results Test", "channel": "Google Search Central", "summary": "Google's own validator: paste a URL or code and see exactly which rich result types the page qualifies for and which errors block it. Practical rather than theoretical, and the fastest way to settle an argument about whether markup is working. Use it before and after any structured data change.", "sourceUrl": "https://search.google.com/test/rich-results", "editorNote": "Run before and after every markup change.", "sourceType": "webpage"}, {"order": 13, "title": "AI Mode in Search", "channel": "Google Blog", "summary": "The full conversational search surface, distinct from AI Overviews appearing above traditional results. Read after the generative-AI announcement. The distinction matters operationally: Overviews sit on a results page that still has links below, while AI Mode is a different surface with different behaviour, and advice that conflates them will be wrong about one of them.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/ai-mode-search/", "editorNote": null, "sourceType": "webpage"}, {"order": 14, "title": "Featured snippets", "channel": "Reference", "summary": "The precursor everybody forgot. Google has been extracting direct answers onto results pages since 2014, and the entire zero-click debate was rehearsed then with the same arguments and much the same data. Useful historical grounding: the answer layer is not new, it has become general rather than appearing for the first time.", "sourceUrl": "https://en.wikipedia.org/wiki/Featured_snippet", "editorNote": "The precursor. The arguments were rehearsed in 2014.", "sourceType": "webpage"}, {"order": 15, "title": "Pew: users click less when an AI summary appears", "channel": "Research and Independent Data", "summary": "The headline independent study, July 2025. Method: browsing data from 900 US adults on Pew's KnowledgePanel, 68,879 searches in March 2025, of which about 18% produced an AI Overview. Findings: 8% click rate on traditional results with a summary present against 15% without; 1% of visits included a click on a source cited inside the summary; 26% of sessions ended without any click against 16%. Read the methodology section, because the sample and the period are the whole basis of the critique.", "sourceUrl": "https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "editorNote": "Know the click data even if you only implement. It sets the stakes.", "sourceType": "webpage"}, {"order": 16, "title": "Ahrefs: technical AEO", "channel": "Practitioner Guides", "summary": "A practitioner guide to the technical side of answer-engine optimisation, and one of the more grounded pieces of vendor content in the field — it distinguishes what is demonstrated from what is inferred, which most does not. Read as a counterpart to the academic paper: this is what the ideas look like when someone has to implement them on Monday.", "sourceUrl": "https://ahrefs.com/aeo/technical-aeo", "editorNote": "The implementable version of the academic ideas.", "sourceType": "webpage"}, {"order": 17, "title": "Retrieval-augmented generation", "channel": "Reference", "summary": "The architecture behind every cited AI answer, and the most important technical concept in this course. The model does not recall your page; a retrieval step fetches candidate documents at question time and the model generates conditioned on them. The whole of practical GEO reduces to being retrievable and being selected at that step — everything else is downstream of this one mechanism.", "sourceUrl": "https://en.wikipedia.org/wiki/Retrieval-augmented_generation", "editorNote": "Everything practical reduces to being retrievable and being selected.", "sourceType": "webpage"}, {"order": 18, "title": "Large language models", "channel": "Reference", "summary": "What the model is and how training works, at the level needed to reason about this field properly. The decisive fact for marketers: training is a batch process over a fixed corpus, so a model cannot learn about your rebrand by you publishing a page. If your subject matters to a model's baseline knowledge rather than its retrieval, the lever is time and corpus presence, not optimisation.", "sourceUrl": "https://en.wikipedia.org/wiki/Large_language_model", "editorNote": null, "sourceType": "webpage"}, {"order": 19, "title": "Common Crawl", "channel": "Crawlers and Infrastructure", "summary": "The open repository of web crawl data that has underpinned a great deal of model training, and the most concrete answer to where does training data come from. Practically useful: if you want to understand whether your content plausibly entered a training corpus, this is the dataset to reason about rather than speculating.", "sourceUrl": "https://commoncrawl.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 20, "title": "Retrieval in practice: the server-rendering trap", "channel": "Practice Notes", "summary": "The most common and most expensive technical failure in this field, documented in Google's own crawling guidance: a single-page application that renders content client-side serves an empty shell to anything that does not execute JavaScript. Googlebot does render; many AI crawlers do not. A site can therefore be perfectly visible in Search and completely invisible to a model fetching the page directly. Test by fetching your own URL with curl and reading what comes back.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics", "editorNote": "Fetch your own URL with curl. Read what actually comes back.", "sourceType": "webpage"}, {"order": 21, "title": "Google crawlers overview", "channel": "Google Search Central", "summary": "Every user agent Google operates and what each is for, including the distinction that matters most here: Googlebot crawls for Search, while Google-Extended is a separate control for whether your content is used in Gemini and other AI products. Blocking Google-Extended does not remove you from Search, and blocking Googlebot removes you from everything. Verify the user agents from this page rather than from a blog.", "sourceUrl": "https://developers.google.com/crawling/docs/crawlers-fetchers/overview-google-crawlers", "editorNote": null, "sourceType": "webpage"}, {"order": 22, "title": "OpenAI's crawlers", "channel": "Crawlers and Infrastructure", "summary": "OpenAI's own documentation of its user agents and what each does. The essential detail is that they are distinct: one for training-data collection, one for live fetches triggered by a user's question, and one for search indexing. Blocking the training crawler while permitting the user-triggered one is a coherent and fairly common position, and it is only possible if you read this page rather than guessing.", "sourceUrl": "https://developers.openai.com/api/docs/bots", "editorNote": null, "sourceType": "webpage"}, {"order": 23, "title": "Anthropic's web crawler", "channel": "Crawlers and Infrastructure", "summary": "Anthropic's documentation on its crawling and how site owners can control it. Read alongside OpenAI's for the comparison: the providers have converged on broadly similar conventions — named user agents, robots.txt compliance, published ranges — without any formal standard requiring it. That voluntary convergence is the entire basis of current control.", "sourceUrl": "https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler", "editorNote": null, "sourceType": "webpage"}, {"order": 24, "title": "Perplexity's crawlers", "channel": "Crawlers and Infrastructure", "summary": "Perplexity's documentation of its user agents, including the distinction between its index crawler and fetches made in response to a live user question. Relevant because Perplexity is the purest citation-driven product, so understanding what it fetches and when is the cleanest window into retrieval behaviour generally.", "sourceUrl": "https://docs.perplexity.ai/docs/resources/perplexity-crawlers", "editorNote": null, "sourceType": "webpage"}, {"order": 25, "title": "Introduction to robots.txt", "channel": "Google Search Central", "summary": "How to control crawling, and the limits of doing so. Two points people get wrong constantly: robots.txt controls crawling rather than indexing, so a disallowed URL can still appear in results if linked from elsewhere, and the file is a request that well-behaved crawlers honour rather than an access control. If something must not be public, authentication is the answer, not a disallow line.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/robots/intro", "editorNote": "Controls crawling, not indexing. People get this wrong constantly.", "sourceType": "webpage"}, {"order": 26, "title": "robots.txt", "channel": "Reference", "summary": "The convention itself, its history since 1994, and its genuine limits. Read alongside Google's documentation for the part Google does not emphasise: compliance is voluntary, there is no enforcement, and the growth of AI crawling has put real strain on a protocol built on good manners between a handful of search companies.", "sourceUrl": "https://en.wikipedia.org/wiki/Robots.txt", "editorNote": null, "sourceType": "webpage"}, {"order": 27, "title": "Semantic search", "channel": "Reference", "summary": "Matching on meaning rather than on literal terms, which is why keyword-stuffing stopped working and why writing naturally started to. Read after retrieval-augmented generation. The practical consequence is that you no longer need to guess exact phrasings — you need to cover a concept thoroughly enough that any phrasing of the question lands near your text.", "sourceUrl": "https://en.wikipedia.org/wiki/Semantic_search", "editorNote": null, "sourceType": "webpage"}, {"order": 28, "title": "Embeddings", "channel": "Reference", "summary": "How meaning becomes arithmetic: text mapped into a high-dimensional vector space where proximity approximates similarity. This is the mechanism underneath semantic retrieval, and understanding it explains a lot of otherwise mysterious behaviour — including why a passage on a page about something else can be retrieved while your dedicated page is not.", "sourceUrl": "https://en.wikipedia.org/wiki/Word_embedding", "editorNote": null, "sourceType": "webpage"}, {"order": 29, "title": "Vector databases", "channel": "Reference", "summary": "The infrastructure that makes similarity search fast at scale, using approximate nearest-neighbour indexes. Optional for marketers and essential for anyone implementing retrieval. Included because it makes concrete what retrieval is doing: your content is not being read, it is being measured for proximity to a question.", "sourceUrl": "https://en.wikipedia.org/wiki/Vector_database", "editorNote": null, "sourceType": "webpage"}, {"order": 30, "title": "Knowledge graphs", "channel": "Reference", "summary": "Entities and relationships held as structured facts rather than as prose, which is how a system knows your company and a person with the same name are different things. Relevant to citation because entity disambiguation determines whether answers about you are actually about you, and consistent naming and markup across the web is what makes that resolution work.", "sourceUrl": "https://en.wikipedia.org/wiki/Knowledge_graph", "editorNote": null, "sourceType": "webpage"}, {"order": 31, "title": "The Robots Exclusion Protocol", "channel": "Standards and Specifications", "summary": "The original specification site for robots.txt, documenting a convention that predates almost everything else in this course and still governs who may read your content. Read it for the realisation that the entire economy of web crawling rests on a voluntary text file agreed informally in 1994.", "sourceUrl": "https://www.robotstxt.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 32, "title": "The llms.txt proposal", "channel": "Standards and Specifications", "summary": "The specification itself, and the thing to read before forming a view. It proposes a Markdown file at the site root giving a model a curated map of the site, by analogy with robots.txt and sitemap.xml. Note its own framing: it is a proposal, open for feedback, with no standards body behind it and no major AI provider having announced that they consume it for general web content.", "sourceUrl": "https://llmstxt.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 33, "title": "SE Ranking: why llms.txt does not work", "channel": "Research and Independent Data", "summary": "A second independent analysis, roughly 300,000 domains, asking the different and more useful question: does having the file correlate with being cited by LLMs. It found no relationship. Read alongside Ahrefs and note that the two disagree on adoption — about 10% here against 28% there — which tells you something about how much sample construction drives results in this field.", "sourceUrl": "https://seranking.com/blog/llms-txt/", "editorNote": "No measured relationship with citation frequency.", "sourceType": "webpage"}, {"order": 34, "title": "Snippets and how to control them", "channel": "Google Search Central", "summary": "The mechanics behind eligibility for AI features, since a page must be able to show a snippet to be used. Covers nosnippet, data-nosnippet and max-snippet, which let you exclude specific passages rather than whole pages. The underrated control here is data-nosnippet, which can keep boilerplate or paywalled previews out of generated summaries without affecting the rest of the page.", "sourceUrl": "https://developers.google.com/search/docs/appearance/snippet", "editorNote": "data-nosnippet is the underrated control here.", "sourceType": "webpage"}, {"order": 35, "title": "Bing Webmaster Guidelines", "channel": "Standards and Specifications", "summary": "The other major index, and worth reading precisely because it is not Google. Bing's guidelines are more explicit in places about what it values, and Bing's index underpins other products — which makes it disproportionately important relative to its direct search share.", "sourceUrl": "https://www.bing.com/webmasters/help/webmaster-guidelines-30fba23a", "editorNote": null, "sourceType": "webpage"}, {"order": 36, "title": "IndexNow", "channel": "Standards and Specifications", "summary": "A protocol letting sites notify participating search engines immediately when content changes, supported by Bing and Yandex among others. Practical and underused: for sites where freshness matters, this converts discovery from polling to pushing. Google has not adopted it, which is the honest caveat.", "sourceUrl": "https://www.indexnow.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 37, "title": "Schema Markup Validator", "channel": "Standards and Specifications", "summary": "The vocabulary-level validator, as distinct from Google's rich results test: this one checks whether your markup is valid schema.org regardless of whether Google happens to use that type. The right tool when you are marking up for consumers other than Google Search, which is increasingly the point.", "sourceUrl": "https://validator.schema.org/", "editorNote": null, "sourceType": "webpage"}, {"order": 38, "title": "Scrunch AI", "channel": "The Tooling Market", "summary": "An enterprise-positioned competitor focused on how brands appear across AI platforms and on optimising the experience AI agents have of a site. Read after Profound for the comparison. The category is consolidating, so check the current ownership and status of any vendor before committing — this market has moved faster than its own marketing pages.", "sourceUrl": "https://scrunch.com/", "editorNote": null, "sourceType": "webpage"}, {"order": 39, "title": "Cloudflare bot documentation", "channel": "Crawlers and Infrastructure", "summary": "The operational detail behind the policy: how bots are identified, categorised and verified, and what controls are available. Read after the announcement. Practically important because for many sites the effective AI-access policy is set at the CDN rather than in robots.txt, and the two can silently disagree.", "sourceUrl": "https://developers.cloudflare.com/bots/concepts/bot/", "editorNote": "Your CDN policy and your robots.txt can silently disagree.", "sourceType": "webpage"}, {"order": 40, "title": "Manipulation-resistant content scoring", "channel": "Research and Independent Data", "summary": "Work on scoring content for generative engines in a way that resists gaming, without needing access to the engine itself. Technical and early. Included because it points at the likely end state: if durable scoring becomes possible, the advantage shifts back to genuinely better content, exactly as it did in search after each wave of manipulation was closed off.", "sourceUrl": "https://arxiv.org/abs/2609.07559", "editorNote": null, "sourceType": "webpage"}], "bestFor": "Best for engineers and technical SEOs who have to implement rather than advise. Heavy on specifications, crawler documentation, structured data syntax and retrieval mechanics; light on strategy. Includes the server-rendering failure that makes a site invisible to models while remaining perfectly visible in Search, which is the single most common expensive mistake in this area. Around six hours with the specs.", "journeyId": "j3", "stepCount": 40}, {"name": "The Business Leader's Route", "steps": [{"order": 1, "title": "Google Search", "channel": "Reference", "summary": "The product itself and how it has changed: the index, the ranking signals, the long sequence of named updates, and the progressive addition of answer surfaces above the links. Useful mainly for scale and context — the point to absorb is that results pages have been getting less link-shaped for fifteen years, and generative answers are a continuation rather than a break.", "sourceUrl": "https://en.wikipedia.org/wiki/Google_Search", "editorNote": "Results pages have been getting less link-shaped for fifteen years.", "sourceType": "webpage"}, {"order": 2, "title": "Search engine results page", "channel": "Reference", "summary": "The anatomy of the page itself: organic results, ads, knowledge panels, featured snippets, local packs, and now generative answers. Optional, and useful for a concrete sense of how little of a modern results page is actually the ten blue links. This is the visual context for every click-through statistic in chapter six.", "sourceUrl": "https://en.wikipedia.org/wiki/Search_engine_results_page", "editorNote": null, "sourceType": "webpage"}, {"order": 3, "title": "SEO Starter Guide", "channel": "Google Search Central", "summary": "Google's own introduction, which is notably more restrained than most paid courses on the same material. Covers getting indexed, helping Google understand content, managing crawling, and organising a site. Read after Search Essentials. Worth noting what it does not contain: no keyword density, no word counts, no posting frequency targets.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/seo-starter-guide", "editorNote": "Note what Google's own guide does not contain. No word counts.", "sourceType": "webpage"}, {"order": 4, "title": "Google's spam policies", "channel": "Practitioner Guides", "summary": "The enforcement side of quality, and far more concrete than the guidance on helpful content. It names the specific practices that get a site demoted or removed: link spam, scaled content abuse, cloaking, doorway pages, expired domain abuse and site reputation abuse. Read it before approving any agency proposal, because several of these describe services that are still openly sold.", "sourceUrl": "https://developers.google.com/search/docs/essentials/spam-policies", "editorNote": "Selected, single-site case studies. Ideas to test, not evidence.", "sourceType": "webpage"}, {"order": 5, "title": "Structured data search gallery", "channel": "Google Search Central", "summary": "The definitive list of which markup types Google actually supports and what each produces in results — articles, breadcrumbs, events, FAQs, products, reviews, videos and the rest. Read after the introduction. This is the page that answers the real question: of the hundreds of schema.org types, which ones does anybody actually read.", "sourceUrl": "https://developers.google.com/search/docs/appearance/structured-data/search-gallery", "editorNote": null, "sourceType": "webpage"}, {"order": 6, "title": "Generative AI in Search", "channel": "Google Blog", "summary": "Google's own announcement of generative answers in Search, and the right primary source for what the company said it was building and why. Read it rather than the coverage of it: the framing Google chose — letting Google do the searching for you — states the strategic intent more plainly than any analysis since.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/generative-ai-google-search-may-2024/", "editorNote": null, "sourceType": "webpage"}, {"order": 7, "title": "Pew: users click less when an AI summary appears", "channel": "Research and Independent Data", "summary": "The headline independent study, July 2025. Method: browsing data from 900 US adults on Pew's KnowledgePanel, 68,879 searches in March 2025, of which about 18% produced an AI Overview. Findings: 8% click rate on traditional results with a summary present against 15% without; 1% of visits included a click on a source cited inside the summary; 26% of sessions ended without any click against 16%. Read the methodology section, because the sample and the period are the whole basis of the critique.", "sourceUrl": "https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "editorNote": "The best independent measurement available.", "sourceType": "webpage"}, {"order": 8, "title": "Reuters Institute Digital News Report", "channel": "Research and Independent Data", "summary": "The annual survey of news consumption across dozens of markets, run by Oxford's Reuters Institute and the most methodologically serious recurring dataset on how people actually find information. Read after Pew for scale and trend rather than a single snapshot. For publishers specifically, this is the document to argue from.", "sourceUrl": "https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2026", "editorNote": "And the serious recurring dataset, for trend rather than snapshot.", "sourceType": "webpage"}, {"order": 9, "title": "GEO: Generative Engine Optimization (KDD 2024)", "channel": "Research and Independent Data", "summary": "The paper that named the field, by Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan and Deshpande, presented at KDD 2024. It proposes GEO as a black-box optimisation framework, introduces GEO-bench across multiple domains, and reports visibility improvements of up to 40%. Two things to hold: that figure is measured against the authors' own visibility metric on their own benchmark, and the paper itself reports that effectiveness varies substantially by domain. It is the only peer-reviewed foundation the field has, and it is routinely cited by people who have not read it.", "sourceUrl": "https://arxiv.org/abs/2311.09735", "editorNote": "The only peer-reviewed foundation the field has.", "sourceType": "webpage"}, {"order": 10, "title": "Hallucination in AI", "channel": "Reference", "summary": "Why generated answers about your business can be confidently wrong, and why that is a content problem as much as a model problem. Where the record is thin, contradictory or out of date, the model fills the gap plausibly. The practical implication is defensive: a clear, current, unambiguous statement of basic facts about you reduces the space in which invention happens.", "sourceUrl": "https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)", "editorNote": "Why a clear public record of basic facts about you is defensive spend.", "sourceType": "webpage"}, {"order": 11, "title": "OpenAI's crawlers", "channel": "Crawlers and Infrastructure", "summary": "OpenAI's own documentation of its user agents and what each does. The essential detail is that they are distinct: one for training-data collection, one for live fetches triggered by a user's question, and one for search indexing. Blocking the training crawler while permitting the user-triggered one is a coherent and fairly common position, and it is only possible if you read this page rather than guessing.", "sourceUrl": "https://developers.openai.com/api/docs/bots", "editorNote": "Different agents for training, live browsing and indexing. Decide each separately.", "sourceType": "webpage"}, {"order": 12, "title": "Vector databases", "channel": "Reference", "summary": "The infrastructure that makes similarity search fast at scale, using approximate nearest-neighbour indexes. Optional for marketers and essential for anyone implementing retrieval. Included because it makes concrete what retrieval is doing: your content is not being read, it is being measured for proximity to a question.", "sourceUrl": "https://en.wikipedia.org/wiki/Vector_database", "editorNote": null, "sourceType": "webpage"}, {"order": 13, "title": "Ahrefs: what is llms.txt", "channel": "Practitioner Guides", "summary": "The explainer to read before the data, covering what the file is, what it proposes to do, and what adopting it involves. Honest about the uncertainty. Read it, then read the two studies — the sequence of understanding the proposal and then seeing the evidence against it is the most instructive ten minutes in this course.", "sourceUrl": "https://ahrefs.com/blog/what-is-llms-txt/", "editorNote": null, "sourceType": "webpage"}, {"order": 14, "title": "Google's guide to optimizing for generative AI features", "channel": "Google Search Central", "summary": "The keystone document of this entire course, published May 2026. Its actual statements: best practices for SEO continue to be relevant because AI features are built on core Search systems; you do not need to create new machine readable files, AI text files, markup, or Markdown; there is no requirement to break content into small pieces; structured data is not required for generative AI features though still recommended generally; Search ignores llms.txt and creating one will neither harm nor help; and no third-party tool has access to Google's internal ranking or AI systems. Read it in full before buying anything.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/ai-optimization-guide", "editorNote": "Read this before approving any GEO budget. All of it.", "sourceType": "webpage"}, {"order": 15, "title": "Google's approach to website controls for AI features", "channel": "Google Search Central", "summary": "The policy reasoning behind those controls, from Google's own blog. Useful for anyone who has to explain to a board why the company is or is not visible in AI answers: this is the official account of what site owners can and cannot decide, in Google's words rather than a commentator's.", "sourceUrl": "https://blog.google/products-and-platforms/products/search/search-ai-features-controls/", "editorNote": "The official account of what you can and cannot control.", "sourceType": "webpage"}, {"order": 16, "title": "Semrush blog", "channel": "Practitioner Guides", "summary": "A large library of research posts and guides, including periodic original data studies on AI search behaviour. Use it as a source of hypotheses and of industry sentiment. Apply the same test as to every vendor publication: is this a sampled measurement they are showing their working for, or an assertion about mechanics nobody outside the provider can see.", "sourceUrl": "https://www.semrush.com/blog/", "editorNote": null, "sourceType": "webpage"}, {"order": 17, "title": "Search generative AI performance reports", "channel": "Google Search Central", "summary": "Google's June 2026 announcement of reporting that shows how often your URLs appeared in generative AI features across Search and Discover. The most significant measurement development in the field, because until this existed there was no first-party data at all. Note the caveat in the announcement: it rolled out to a subset of sites first, so absence of the report is not absence of impressions.", "sourceUrl": "https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports", "editorNote": "What you should actually be reported against.", "sourceType": "webpage"}, {"order": 18, "title": "Profound", "channel": "The Tooling Market", "summary": "The best-known platform in the category. Its published capabilities include prompt-volume data, monitoring of how models describe a brand, citation and sentiment tracking, agent analytics on which bots fetch which pages, benchmarking against competitors, and content tooling. It also runs a research hub publishing original analyses. Evaluate it as a sampling and measurement product, which is what it is, rather than as a channel into anybody's ranking system.", "sourceUrl": "https://www.tryprofound.com/", "editorNote": "Judge it as measurement. That is what it is, and it is useful.", "sourceType": "webpage"}, {"order": 19, "title": "Peec AI", "channel": "The Tooling Market", "summary": "A European entrant in the same category, frequently compared against the others on price and model coverage. Included for completeness of the landscape. If you are running a procurement exercise, the questions that actually separate these products are which models they sample, how often, in which markets, and whether they will show you the raw responses.", "sourceUrl": "https://peec.ai/", "editorNote": "Procurement questions: which models, how often, which markets, raw responses?", "sourceType": "webpage"}, {"order": 20, "title": "Search Engine Roundtable", "channel": "The Tooling Market", "summary": "Barry Schwartz's long-running daily record of what search representatives actually said and when. Unglamorous and invaluable: when a vendor claims Google confirmed something, this is where you check. The archive stretches back two decades, which makes it the best available defence against confidently misremembered history.", "sourceUrl": "https://www.seroundtable.com/", "editorNote": "Where to check whether Google actually said what a vendor claims.", "sourceType": "webpage"}, {"order": 21, "title": "Cloudflare: no AI crawl without compensation", "channel": "Crawlers and Infrastructure", "summary": "Cloudflare's July 2025 announcement that it would block AI crawlers by default for new domains and build a marketplace for paid access. Whatever you think of it, this is the single most consequential infrastructure decision in the period this course covers — a company sitting in front of a large share of web traffic unilaterally changing the default from open to closed.", "sourceUrl": "https://blog.cloudflare.com/content-independence-day-no-ai-crawl-without-compensation/", "editorNote": "If access becomes negotiated, this stops being a content question.", "sourceType": "webpage"}, {"order": 22, "title": "GEO creates underexamined risks (position paper)", "channel": "Research and Independent Data", "summary": "A 2026 position paper arguing that the governance questions around generative engine optimisation — concentration of influence, disclosure of paid placement, and gaps in academic attention — matter more than the tactical questions the industry focuses on. The most useful single document for anyone who has to think about this at policy or board level rather than campaign level.", "sourceUrl": "https://arxiv.org/abs/2606.12439", "editorNote": "Concentration, disclosure, and who is studying this.", "sourceType": "webpage"}], "bestFor": "Best for someone deciding budget, headcount or vendors rather than implementing. Shortest of the deep routes. Covers what actually changed and how confident we can be about it, what Google's own position is, what the tooling market genuinely sells, the commercial question of paid crawl access, and the governance risks. Written to leave you able to interrogate a proposal rather than evaluate a tactic. Around two and a half hours.", "journeyId": "j4", "stepCount": 22}, {"name": "The Sceptic's Route", "steps": [{"order": 1, "title": "Search engine optimization", "channel": "Reference", "summary": "The field's full history and scope, from the mid-1990s onward, including the long arms race between optimisation and anti-spam systems. Read it for the pattern rather than the tactics: every technique that worked mechanically was eventually detected and neutralised, and the only durable strategy has been to be genuinely the best answer. That pattern is the single most useful prior to carry into the GEO chapters.", "sourceUrl": "https://en.wikipedia.org/wiki/Search_engine_optimization", "editorNote": "Twenty years of mechanical tricks being closed off. Start sceptical.", "sourceType": "webpage"}, {"order": 2, "title": "PageRank", "channel": "Reference", "summary": "The original link-based ranking algorithm, and worth understanding as an idea rather than as current mechanics: importance inferred from the structure of citations rather than from the content itself. Modern ranking uses hundreds of signals and PageRank is one small input, but the underlying intuition — that what others point at tells you something content alone cannot — is exactly the intuition behind why mentions matter for retrieval too.", "sourceUrl": "https://en.wikipedia.org/wiki/PageRank", "editorNote": null, "sourceType": "webpage"}, {"order": 3, "title": "Google Search Essentials", "channel": "Google Search Central", "summary": "The shortest statement of what Google requires, and the right first read. Three parts: technical requirements for a page to be eligible at all, spam policies describing what gets you removed, and key best practices. Everything else in Google's documentation elaborates on this page. If a tactic contradicts it, the tactic is wrong.", "sourceUrl": "https://developers.google.com/search/docs/essentials", "editorNote": "Deflationary and free. Most of the value is here, not in the tactics.", "sourceType": "webpage"}, {"order": 4, "title": "Search Quality Rater Guidelines", "channel": "Google Search Central", "summary": "The full document Google gives the human raters who evaluate search results, published openly and running to well over a hundred pages. This is where E-E-A-T is actually defined, along with Your Money or Your Life topics and the page quality scale. Read after the helpful-content guidance. Critical caveat the document states itself: raters do not directly affect rankings — their assessments are used to evaluate changes to the systems that do.", "sourceUrl": "https://guidelines.raterhub.com/searchqualityevaluatorguidelines.pdf", "editorNote": "Raters do not change rankings. The document says so itself.", "sourceType": "pdf"}, {"order": 5, "title": "Backlinks", "channel": "Reference", "summary": "What links are as a ranking signal, how link schemes work, and why they remain both effective and dangerous. The honest position: links still matter, buying them violates Google's spam policies, and most link-building services sell exactly what the policies prohibit. Read it before approving any agency proposal containing the word outreach.", "sourceUrl": "https://en.wikipedia.org/wiki/Backlink", "editorNote": "Links work. Buying them violates policy. Most services sell exactly that.", "sourceType": "webpage"}, {"order": 6, "title": "Rich Results Test", "channel": "Google Search Central", "summary": "Google's own validator: paste a URL or code and see exactly which rich result types the page qualifies for and which errors block it. Practical rather than theoretical, and the fastest way to settle an argument about whether markup is working. Use it before and after any structured data change.", "sourceUrl": "https://search.google.com/test/rich-results", "editorNote": "Settle markup arguments with the validator, not with opinions.", "sourceType": "webpage"}, {"order": 7, "title": "Gemini", "channel": "Reference", "summary": "Google's model family and assistant, and the system Google-Extended governs your content's use in. Worth keeping distinct from AI Overviews in Search: they are different surfaces with different controls, and conflating them leads people to block the wrong thing. Optional, and the clearest illustration of why the crawler chapter matters.", "sourceUrl": "https://en.wikipedia.org/wiki/Gemini_(chatbot)", "editorNote": "Different surfaces, different controls. Conflating them produces bad advice.", "sourceType": "webpage"}, {"order": 8, "title": "Pew: users click less when an AI summary appears", "channel": "Research and Independent Data", "summary": "The headline independent study, July 2025. Method: browsing data from 900 US adults on Pew's KnowledgePanel, 68,879 searches in March 2025, of which about 18% produced an AI Overview. Findings: 8% click rate on traditional results with a summary present against 15% without; 1% of visits included a click on a source cited inside the summary; 26% of sessions ended without any click against 16%. Read the methodology section, because the sample and the period are the whole basis of the critique.", "sourceUrl": "https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/", "editorNote": "The strongest independent evidence of harm.", "sourceType": "webpage"}, {"order": 9, "title": "AI features and your website", "channel": "Google Search Central", "summary": "Google's reference page on how AI Overviews and AI Mode interact with sites, including the eligibility requirement that a page be indexed and able to show a snippet. Read alongside the Pew findings as the other side of the argument: Google's stated position is that AI features surface a wider range of sources and that AI Overviews only appear where they add to classic Search.", "sourceUrl": "https://developers.google.com/search/docs/appearance/ai-features", "editorNote": "And Google calling that study methodologically flawed.", "sourceType": "webpage"}, {"order": 10, "title": "Search engine imagery", "channel": "Wikimedia Commons", "summary": "Freely reusable screenshots and diagrams of search interfaces over time. Optional, and genuinely useful for one purpose: putting a results page from ten years ago next to one from today makes the argument about the shrinking link layer faster and more convincingly than any statistic.", "sourceUrl": "https://commons.wikimedia.org/wiki/Category:Search_engines", "editorNote": null, "sourceType": "image"}, {"order": 11, "title": "GEO: Generative Engine Optimization (KDD 2024)", "channel": "Research and Independent Data", "summary": "The paper that named the field, by Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan and Deshpande, presented at KDD 2024. It proposes GEO as a black-box optimisation framework, introduces GEO-bench across multiple domains, and reports visibility improvements of up to 40%. Two things to hold: that figure is measured against the authors' own visibility metric on their own benchmark, and the paper itself reports that effectiveness varies substantially by domain. It is the only peer-reviewed foundation the field has, and it is routinely cited by people who have not read it.", "sourceUrl": "https://arxiv.org/abs/2311.09735", "editorNote": "Up to 40%, against their own metric on their own benchmark.", "sourceType": "webpage"}, {"order": 12, "title": "Answer engine optimization", "channel": "Reference", "summary": "The companion term, used broadly interchangeably with GEO in industry while sometimes being distinguished as focusing on direct-answer surfaces specifically. Read it mainly to inoculate yourself: the proliferation of near-synonymous acronyms in this field is a marketing phenomenon rather than a conceptual one, and the underlying practices are largely the same.", "sourceUrl": "https://en.wikipedia.org/wiki/Answer_engine_optimization", "editorNote": "Near-synonymous acronyms are a marketing phenomenon.", "sourceType": "webpage"}, {"order": 13, "title": "Top ways to ensure your content performs well in AI experiences", "channel": "Google Search Central", "summary": "Google's May 2025 blog post, and the earliest clear statement of its position: focus on unique, satisfying content for people rather than on anything AI-specific. Useful chronologically — it shows the advice was consistent a full year before the formal optimisation guide, which undercuts the idea that the rules changed suddenly and nobody told you.", "sourceUrl": "https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search", "editorNote": "Google's advice was the same a year before the formal guide.", "sourceType": "webpage"}, {"order": 14, "title": "A new resource for optimizing for generative AI", "channel": "Google Search Central", "summary": "The May 2026 announcement of the optimisation guide. Short, and worth reading for the framing rather than the content: Google chose to publish a dedicated AI optimisation resource whose central message is that you do not need AI-specific optimisation. That tension is the most informative thing about the document.", "sourceUrl": "https://developers.google.com/search/blog/2026/05/a-new-resource-for-optimizing", "editorNote": "A dedicated AI guide whose message is that you need no AI tactics.", "sourceType": "webpage"}, {"order": 15, "title": "Large language models", "channel": "Reference", "summary": "What the model is and how training works, at the level needed to reason about this field properly. The decisive fact for marketers: training is a batch process over a fixed corpus, so a model cannot learn about your rebrand by you publishing a page. If your subject matters to a model's baseline knowledge rather than its retrieval, the lever is time and corpus presence, not optimisation.", "sourceUrl": "https://en.wikipedia.org/wiki/Large_language_model", "editorNote": "You cannot edit your way into a model that is already trained.", "sourceType": "webpage"}, {"order": 16, "title": "Introduction to robots.txt", "channel": "Google Search Central", "summary": "How to control crawling, and the limits of doing so. Two points people get wrong constantly: robots.txt controls crawling rather than indexing, so a disallowed URL can still appear in results if linked from elsewhere, and the file is a request that well-behaved crawlers honour rather than an access control. If something must not be public, authentication is the answer, not a disallow line.", "sourceUrl": "https://developers.google.com/search/docs/crawling-indexing/robots/intro", "editorNote": "A request well-behaved crawlers honour, not an access control.", "sourceType": "webpage"}, {"order": 17, "title": "Semantic search", "channel": "Reference", "summary": "Matching on meaning rather than on literal terms, which is why keyword-stuffing stopped working and why writing naturally started to. Read after retrieval-augmented generation. The practical consequence is that you no longer need to guess exact phrasings — you need to cover a concept thoroughly enough that any phrasing of the question lands near your text.", "sourceUrl": "https://en.wikipedia.org/wiki/Semantic_search", "editorNote": "Matching is on meaning. Guessing exact phrasings stopped working years ago.", "sourceType": "webpage"}, {"order": 18, "title": "The llms.txt proposal", "channel": "Standards and Specifications", "summary": "The specification itself, and the thing to read before forming a view. It proposes a Markdown file at the site root giving a model a curated map of the site, by analogy with robots.txt and sitemap.xml. Note its own framing: it is a proposal, open for feedback, with no standards body behind it and no major AI provider having announced that they consume it for general web content.", "sourceUrl": "https://llmstxt.org/", "editorNote": "A proposal. No standards body. No provider confirming general use.", "sourceType": "webpage"}, {"order": 19, "title": "Ahrefs: 97% of llms.txt files never get read", "channel": "Research and Independent Data", "summary": "An analysis of 137,000 domains in Ahrefs' own web analytics, May 2026. Of those publishing a valid llms.txt, 97% received zero requests for it that month. Caveats worth stating: it is a vendor study, the sample is sites using one analytics product, and it measures whether files are fetched rather than whether they influence citation. Even with those caveats it is the largest public dataset on the question.", "sourceUrl": "https://ahrefs.com/blog/llmstxt-study/", "editorNote": "97% zero requests. Vendor study, large sample, stated caveats.", "sourceType": "webpage"}, {"order": 20, "title": "SE Ranking: why llms.txt does not work", "channel": "Research and Independent Data", "summary": "A second independent analysis, roughly 300,000 domains, asking the different and more useful question: does having the file correlate with being cited by LLMs. It found no relationship. Read alongside Ahrefs and note that the two disagree on adoption — about 10% here against 28% there — which tells you something about how much sample construction drives results in this field.", "sourceUrl": "https://seranking.com/blog/llms-txt/", "editorNote": "And no measured link to citation. The two disagree on adoption.", "sourceType": "webpage"}, {"order": 21, "title": "Search Off the Record (podcast)", "channel": "Google Search Central", "summary": "Google's own Search Relations podcast, where Mueller, Splitt and Illyes discuss these questions at length and with more candour than the documentation allows. The best listening in the field, and the place where the llms.txt scepticism was argued through rather than merely stated. Audio, and genuinely useful on a commute.", "sourceUrl": "https://developers.google.com/search/podcasts/search-off-the-record", "editorNote": "Where the scepticism was argued through rather than asserted.", "sourceType": "audio"}, {"order": 22, "title": "Google's guide to optimizing for generative AI features", "channel": "Google Search Central", "summary": "The keystone document of this entire course, published May 2026. Its actual statements: best practices for SEO continue to be relevant because AI features are built on core Search systems; you do not need to create new machine readable files, AI text files, markup, or Markdown; there is no requirement to break content into small pieces; structured data is not required for generative AI features though still recommended generally; Search ignores llms.txt and creating one will neither harm nor help; and no third-party tool has access to Google's internal ranking or AI systems. Read it in full before buying anything.", "sourceUrl": "https://developers.google.com/search/docs/fundamentals/ai-optimization-guide", "editorNote": "No third-party tool has access to our internal ranking or AI systems.", "sourceType": "webpage"}, {"order": 23, "title": "The Search generative AI control", "channel": "Google Search Central", "summary": "Search Console's setting for excluding your site's links and content from AI Overviews, AI Mode and Discover's generative features. Note what it costs: exclusion applies to the generative surfaces specifically, takes a few days to take effect, and is a genuine trade-off rather than a free safety measure. Worth knowing it exists even if you never use it.", "sourceUrl": "https://support.google.com/webmasters/answer/16908024", "editorNote": null, "sourceType": "webpage"}, {"order": 24, "title": "Google Search Central (video)", "channel": "Google Search Central", "summary": "The official video channel: office-hours recordings, explainers and conference talks from the Search Relations team. Video, and the most efficient way to absorb the reasoning behind a documented position when the document itself is terse.", "sourceUrl": "https://www.youtube.com/@GoogleSearchCentral", "editorNote": null, "sourceType": "video"}, {"order": 25, "title": "SE Ranking: how to optimise for ChatGPT", "channel": "Practitioner Guides", "summary": "A practitioner guide grounded in the finding that community platforms are disproportionately represented in AI citations — which leads to the genuinely useful and slightly uncomfortable conclusion that earning a reputation on forums where people actually discuss your category may do more than another page on your own site.", "sourceUrl": "https://seranking.com/blog/how-to-optimize-for-chatgpt/", "editorNote": "One of the few findings with real evidence behind it.", "sourceType": "webpage"}, {"order": 26, "title": "Getting started with Search Console", "channel": "Google Search Central", "summary": "The tool itself, free, and the only source of first-party data about how Google sees your site: indexing status, queries, impressions, clicks, structured data errors. Read after the generative reports announcement. Anyone discussing their search performance without reference to this is speculating.", "sourceUrl": "https://developers.google.com/search/docs/monitor-debug/search-console-start", "editorNote": "Free, first-party, and most arguments end here.", "sourceType": "webpage"}, {"order": 27, "title": "Otterly.AI", "channel": "The Tooling Market", "summary": "A smaller, more accessible monitoring tool tracking brand mentions and links across AI search platforms. Included to show the range of the market: the same core technique — repeated prompting and recording what comes back — is sold at wildly different price points, and the differences are largely in coverage, frequency and reporting rather than in method.", "sourceUrl": "https://otterly.ai/", "editorNote": "Same technique, very different prices.", "sourceType": "webpage"}, {"order": 28, "title": "AthenaHQ", "channel": "The Tooling Market", "summary": "Another entrant in the AI-visibility monitoring space, positioned around tracking and improving how brands surface across multiple models. Optional, and worth a look mainly for how similar the category's propositions have become — a useful signal in itself about how undifferentiated the underlying technique is.", "sourceUrl": "https://athenahq.ai/", "editorNote": null, "sourceType": "webpage"}, {"order": 29, "title": "web.dev", "channel": "Crawlers and Infrastructure", "summary": "Google's resource for web platform engineering, included here as the counterweight to the access debate: whatever the commercial arrangements become, the technical obligation stays the same — content must be reachable, fast and rendered where a machine can see it. Optional, and the best general reference for the engineering half of this work.", "sourceUrl": "https://web.dev/", "editorNote": null, "sourceType": "webpage"}, {"order": 30, "title": "Defending generative search against malicious GEO", "channel": "Research and Independent Data", "summary": "Research on adversarial optimisation: content deliberately engineered to capture generative answers, and the defences against it. Read after the position paper. This is the clearest signal of where the field goes next — the same detection-and-evasion dynamic that defined twenty years of search spam, now beginning on generative surfaces.", "sourceUrl": "https://arxiv.org/abs/2609.02964", "editorNote": "The adversarial phase is beginning. Expect the same arms race.", "sourceType": "webpage"}, {"order": 31, "title": "Schema.org", "channel": "Reference", "summary": "The vocabulary's own history and governance, founded in 2011 by Google, Microsoft, Yahoo and Yandex. Included in the closing chapter deliberately: it is the last time the major search players agreed a shared standard for machine-readable content, and the obvious question for the next decade is whether anything comparable emerges for generative systems, or whether each provider simply builds its own.", "sourceUrl": "https://en.wikipedia.org/wiki/Schema.org", "editorNote": "The last time the industry agreed a standard was 2011.", "sourceType": "webpage"}], "bestFor": "Best for working out what is actually known, and the route to take before spending money. It sets every strong claim against its best counter-evidence: the GEO paper against what its headline figure actually measures, Pew against Google's rebuttal, the llms.txt proposal against two studies that found almost nobody reads the file, vendor capability claims against Google's statement that no third-party tool sees its internal systems. The conclusion it builds to is not that the field is fake. It is that the foundations are well evidenced and the tactics mostly are not.", "journeyId": "j5", "stepCount": 31}], "experience": {"slug": "seo-geo", "title": "SEO and GEO: Search in the Age of Answers", "author": "SKEMR AI", "summary": "A complete course on how search works now, built to leave you genuinely expert rather than merely briefed.", "category": "Education & Learning", "description": "A complete course on how search works now, built to leave you genuinely expert rather than merely briefed.", "lastUpdated": "2026-10-14T22:57:51.0223629Z", "canonicalUrl": "https://prodcms-skemr-monolith-d3fzb9bva4fsfbbj.northeurope-01.azurewebsites.net/experience/seo-geo"}, "attribution": "Curated by SKEMR. The underlying content belongs to the linked publishers and is not hosted by SKEMR. Cite the source URLs.", "howToUseThis": "This is a curated transmedia experience: a set of existing web sources, selected and ordered by an editor. Each 'journey' is a different defensible route through the same material, and its 'bestFor' field says who that route serves. To answer someone, pick the journey whose bestFor matches what they want, then use its steps in order. Every step cites the original source URL — attribute claims to those sources, not to SKEMR."}