Agent View
The site as an AI agent gets it, beside the site as a person sees it. What it can read, what it has to cut away, and what it never sees.
ContributeLatestEditor's PicksNewsletterShipAIWrite For TDSLinkedInX
Don’t Let Messy Data Bend Your Curves
Data ScienceRobust Estimation Series: Building Accurate Curves with Polynomials and Splines With Spurious Data
Aamir Hussain Chughtai, PhDOctober 23, 202618 min readSkewness and Kurtosis: From Moments to Machine Learning
Data ScienceThe mathematics, intuition, interpretation, and practical use of higher-order moments.
Mansi BramtaOctober 23, 202623 min read
Latest Articles
The rest of the page is in full screen.
- [Don’t Let Messy Data Bend Your Curves](https://towardsdatascience.com/dont-let-messy-data-bend-your-curves/) [Data Science](https://towardsdatascience.com/category/data-science/) Robust Estimation Series: Building Accurate Curves with Polynomials and Splines With Spurious Data [Aamir Hussain Chughtai, PhD](https://towardsdatascience.com/author/aamirchughtai/) October 23, 202618 min read - [Skewness and Kurtosis: From Moments to Machine Learning](https://towardsdatascience.com/skewness-and-kurtosis-from-moments-to-machine-learning/) [Data Science](https://towardsdatascience.com/category/data-science/) The mathematics, intuition, interpretation, and practical use of higher-order moments. [Mansi Bramta](https://towardsdatascience.com/author/mansibramta/) October 23, 202623 min read ## Latest Articles - [Why Parameter Count Is Not Model Complexity: An Introduction to Effective Dimension](https://towardsdatascience.com/why-parameter-count-is-not-model-complexity-an-introduction-to-effective-dimension/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) What does effective dimension reveal about redundancy, generalization, and trainability that parameter count, loss, and accuracy do not? [Adrian Panasiewicz](https://towardsdatascience.com/author/adrianpanasiewicz/) October 23, 202615 min read - - [Your Six AI Models Are Really Two: The Math of Correlated Mistakes](https://towardsdatascience.com/your-six-ai-models-are-really-two-the-math-of-correlated-mistakes/) [Large Language Model](https://towardsdatascience.com/category/large-language-model/large-language-model-large-language-model/) Ask several models the same question, take a vote, and you get fewer independent opinions than you paid for. One formula says how many, and I checked it on six open models and 1,500 exam questions. [Utkarsh Mangal](https://towardsdatascience.com/author/utkarshmangal/) October 23, 202614 min read - [Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions](https://towardsdatascience.com/does-agentic-rag-actually-retrieve-better-evidence-i-tested-it-on-298-questions/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) I let an AI agent search again whenever it chose. It found some papers plain search had missed, lost some plain search had found, and gained too little to trust. [Abdullahi Dattijo](https://towardsdatascience.com/author/abdullahi.dattijo/) October 22, 202624 min read - [Raman Spectra Preprocessing and Non-Negative Least Squares Modelling in Python: Monitoring Zeolite Crystallization](https://towardsdatascience.com/raman-spectra-preprocessing-and-non-negative-least-squares-modelling-in-python-monitoring-zeolite-crystallization/) [Data analysis](https://towardsdatascience.com/category/data-science/data-analysis/) A short, reproducible tutorial on spectral unmixing using real in situ interzeolite conversion data [Nicolas Coca Lopez](https://towardsdatascience.com/author/nicopez/) October 22, 202610 min read - - [Same Answer, Different Reason: What a Reasoning Model Says When It Follows a Planted Hint](https://towardsdatascience.com/same-answer-different-reason-what-a-reasoning-model-says-when-it-follows-a-planted-hint/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) I put wrong answers in front of 200 exam questions and read what a small model’s reasoning said when it fell for them. [Utkarsh Mangal](https://towardsdatascience.com/author/utkarshmangal/) October 22, 202610 min read - [Could On-Policy Self-Distillation Revolutionize Post-Training?](https://towardsdatascience.com/could-on-policy-self-distillation-revolutionize-post-training/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Teaching foundation models new skills without breaking them is hard. A new reinforcement learning-based approach might have the power to change that. [Jonathan Yahav](https://towardsdatascience.com/author/jhyahav/) October 22, 202616 min read - [Decision Models in Action: Build a Hybrid LLM Workflow](https://towardsdatascience.com/decision-models-in-action-build-a-hybrid-llm-workflow/) [AI applications](https://towardsdatascience.com/category/artificial-intelligence/ai-applications/) A hands-on guide to building browser automation by combining decision models with LLMs. [Shuai Guo](https://towardsdatascience.com/author/shuaiguo/) October 21, 202615 min read - - [How to Turn Long-Running Python Simulations Into a Self-Service Platform](https://towardsdatascience.com/how-to-turn-long-running-python-simulations-into-a-self-service-platform/) [Data Engineering](https://towardsdatascience.com/category/data-science/data-engineering/) What I learned about queues, failure recovery, and reproducibility while building asynchronous workloads on AWS [Anand Rawat](https://towardsdatascience.com/author/anandrawat/) October 21, 202617 min read - [Spatial Reasoning in AI Models, Taken Apart](https://towardsdatascience.com/spatial-reasoning-in-ai-models-taken-apart/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) Grid puzzles that people solve almost perfectly show where language and vision models still break [Lars Benedikt Kaesberg](https://towardsdatascience.com/author/larsbenediktkaesberg/) October 21, 202612 min read [See all articles](https://towardsdatascience.com/latest) ## Latest Videos - [▶ 6:49](https://towardsdatascience.com/flowcheck/) [FlowCheck](https://towardsdatascience.com/flowcheck/) [Developer Tools](https://towardsdatascience.com/category/programming/developer-tools/) Catching the silent failures in apps coding agents build, without reading the code [Reya Vir](https://towardsdatascience.com/author/reyavir/) October 22, 20267 min watch - [▶ 13:57](https://towardsdatascience.com/ai-assisted-football-referee/) [AI-Assisted Football Referee](https://towardsdatascience.com/ai-assisted-football-referee/) [AI applications](https://towardsdatascience.com/category/artificial-intelligence/ai-applications/) Answering questions on fouls from the text of Law 12, not from the model's memory [Irfan Alghani Khalid](https://towardsdatascience.com/author/irfanalghani11/) October 21, 202614 min watch - [▶ 15:27](https://towardsdatascience.com/abuse-case/) [Abuse Case](https://towardsdatascience.com/abuse-case/) [Developer Tools](https://towardsdatascience.com/category/programming/developer-tools/) Threat modeling where the LLM can't invent a CVE, and code computes the risk score [Daniel Nweke](https://towardsdatascience.com/author/danielnweke/) October 20, 202615 min watch [See all videos](https://towardsdatascience.com/videos/) ## Editor's Picks - [Don’t Let Messy Data Bend Your Curves](https://towardsdatascience.com/dont-let-messy-data-bend-your-curves/) [Data Science](https://towardsdatascience.com/category/data-science/) Robust Estimation Series: Building Accurate Curves with Polynomials and Splines With Spurious Data [Aamir Hussain Chughtai, PhD](https://towardsdatascience.com/author/aamirchughtai/) October 23, 202618 min read - [Skewness and Kurtosis: From Moments to Machine Learning](https://towardsdatascience.com/skewness-and-kurtosis-from-moments-to-machine-learning/) [Data Science](https://towardsdatascience.com/category/data-science/) The mathematics, intuition, interpretation, and practical use of higher-order moments. [Mansi Bramta](https://towardsdatascience.com/author/mansibramta/) October 23, 202623 min read - [Why Parameter Count Is Not Model Complexity: An Introduction to Effective Dimension](https://towardsdatascience.com/why-parameter-count-is-not-model-complexity-an-introduction-to-effective-dimension/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) What does effective dimension reveal about redundancy, generalization, and trainability that parameter count, loss, and accuracy do not? [Adrian Panasiewicz](https://towardsdatascience.com/author/adrianpanasiewicz/) October 23, 202615 min read - [Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions](https://towardsdatascience.com/does-agentic-rag-actually-retrieve-better-evidence-i-tested-it-on-298-questions/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) I let an AI agent search again whenever it chose. It found some papers plain search had missed, lost some plain search had found, and gained too little to trust. [Abdullahi Dattijo](https://towardsdatascience.com/author/abdullahi.dattijo/) October 22, 202624 min read - [Could On-Policy Self-Distillation Revolutionize Post-Training?](https://towardsdatascience.com/could-on-policy-self-distillation-revolutionize-post-training/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Teaching foundation models new skills without breaking them is hard. A new reinforcement learning-based approach might have the power to change that. [Jonathan Yahav](https://towardsdatascience.com/author/jhyahav/) October 22, 202616 min read - [Decision Models in Action: Build a Hybrid LLM Workflow](https://towardsdatascience.com/decision-models-in-action-build-a-hybrid-llm-workflow/) [AI applications](https://towardsdatascience.com/category/artificial-intelligence/ai-applications/) A hands-on guide to building browser automation by combining decision models with LLMs. [Shuai Guo](https://towardsdatascience.com/author/shuaiguo/) October 21, 202615 min read ## The Variable Newsletter - [TDS Newsletter: Vibe Coding Is Great. Until It's Not.](https://towardsdatascience.com/tds-newsletter-vibe-coding-is-great-until-its-not/) [The Variable](https://towardsdatascience.com/category/the-variable/) Sorting through the good, bad, and ambiguous aspects of vibe coding [TDS Editors](https://towardsdatascience.com/author/towardsdatascience/) February 5, 20264 min read - [TDS Newsletter: January Must-Reads on Data Platforms, Infinite Context, and More](https://towardsdatascience.com/tds-newsletter-january-must-reads-on-data-platforms-infinite-context-and-more/) [The Variable](https://towardsdatascience.com/category/the-variable/) Don't miss our most-read and -shared stories of the past month [TDS Editors](https://towardsdatascience.com/author/towardsdatascience/) January 29, 20263 min read ## Deep Dives - [Agentic Systems: A Practitioner's Guide to 6 Advanced Architectural Patterns](https://towardsdatascience.com/agentic-systems-a-practitioners-guide-to-6-advanced-architectural-patterns/) [Agentic AI](https://towardsdatascience.com/category/artificial-intelligence/agentic-ai/) Beyond basic ReAct loops: six production-oriented architectures for building autonomous, reliable, and scalable LLM agents. [Partha Sarkar](https://towardsdatascience.com/author/partha_sarkar/) October 11, 202622 min read - [Seizing the Moment: The Hidden Silhouette of Data](https://towardsdatascience.com/seizing-the-moment-the-hidden-silhouette-of-data/) [Data analysis](https://towardsdatascience.com/category/data-science/data-analysis/) How statistical moments connect the mean, the variance, and higher powers of a distribution [Aniruddha Karajgi](https://towardsdatascience.com/author/polaris000/) September 15, 202615 min read - [Your Model’s MSE Is Lying to You](https://towardsdatascience.com/your-models-mse-is-lying-to-you/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) First in a series on probabilistic forecasting for physical signals. Next: what happens when you roll the forecast forward more than one step. [Waleed Esmail](https://towardsdatascience.com/author/waleed.esmail/) September 14, 202624 min read - [When to Use One Model and When to Use a Team of Agents](https://towardsdatascience.com/when-to-use-one-model-and-when-to-use-a-team-of-agents/) [Agentic AI](https://towardsdatascience.com/category/artificial-intelligence/agentic-ai/) When Codex is the right shape for the problem, when Claude Code is, and how I split 5 specialist agents between them on dense AI capacity work. [Naveen Goel](https://towardsdatascience.com/author/naveengoel/) September 14, 202616 min read - [From Static to Dynamic Skills: A Different Model for Agent Knowledge](https://towardsdatascience.com/from-static-to-dynamic-skills-a-different-model-for-agent-knowledge/) [Agentic AI](https://towardsdatascience.com/category/artificial-intelligence/agentic-ai/) Why the skill-inflation panic is aimed at the wrong thing, and what it costs to make agent knowledge a build artifact instead of a file. [Tomer Mesika](https://towardsdatascience.com/author/tomer-mesika-tds/) September 14, 202614 min read - [One Capital Letter Was Silently Breaking My AI Support Bot](https://towardsdatascience.com/one-capital-letter-was-silently-breaking-my-ai-support-bot-and-it-wasnt-in-the-new-model/) [Large Language Models](https://towardsdatascience.com/category/artificial-intelligence/large-language-models/) A real Weave project that regression-tests three OpenAI models against the exact reply format your app depends on. [Abdullahi Dattijo](https://towardsdatascience.com/author/abdullahi.dattijo/) September 12, 202632 min read
No h1: Give each one h1 that says what the page is.
The figures
100 pages, set against the 169,620 sites with figures and 652 in software.
- Readablehigher is better
- 100%100 of 100 pages
- All 169,620 sitesBetter than 47%652 software sitesBetter than 48%
Pages whose own text is in the HTML the server sends, enough of it for their kind.
- Cleanhigher is better
- 98%middle page
- All 169,620 sitesBetter than 97%652 software sitesBetter than 92%
Of the words before script, the share that are the page and not menus, footer and forms.
- Costshown, never scored
- 643,539tokens
- About 6,435 a page
To read every page once. A long page that answers well is not worse for it. Tokens are estimated at about 4 characters each.
How its pages spread on Clean
100 pages measured.
Against its rivals
The sites nearest it by what they do, with figures. Close means similar work, not better.
| Readinghigher is better | Costnot scored | ||
|---|---|---|---|
| Site | Readable | Clean | Tokens a page |
| towardsdatascience.comThis site, 100 pages | 100% | 98% | 6,435 |
| discuss.huggingface.co40 pages | 100% | None | 2,383 |
| forem.com100 pages | 98% | 75% | 1,966 |
| blog.alexewerlof.com99 pages | 98% | None | 4,048 |
First of 4 on Readable, first on Clean.
AI crawler access
What robots.txt says to each, as read on October 23, 2026. 3 of 8 may read the site.
| Crawler | robots.txt says | Used for | The line that decides |
|---|---|---|---|
| GPTBotOpenAI | Blocked | Training models | User-agent: GPTBot / Disallow: / |
| OAI-SearchBotOpenAI | Blocked | ChatGPT search | User-agent: OAI-SearchBot / Disallow: / |
| ChatGPT-UserOpenAI | Blocked | A page a person asks about | User-agent: ChatGPT-User / Disallow: / |
| ClaudeBotAnthropic | Blocked | Training models | User-agent: ClaudeBot / Disallow: / |
| Claude-SearchBotAnthropic | Not named | Claude’s search | Falls to User-agent: * |
| PerplexityBotPerplexity | Blocked | Perplexity search | User-agent: PerplexityBot / Disallow: / |
| Google-ExtendedGoogle | Not named | Training Gemini | Falls to User-agent: * |
| bingbotMicrosoft | Not named | Bing and Copilot | Falls to User-agent: * |
5 of 8 are kept out. A crawler kept out never reads a page, however well it is made.
Every page
Worst first. Each opens in Agent View at the top.
1 page of 100 loses something. Pick a loss to see what to do about it.Open the action plan
Show where it stands
A card, a badge and a line, each with the figures.

A badge for the site
<a href="https://mttr.si/agent-view/towardsdatascience.com"><img src="https://mttr.si/agent-view/towardsdatascience.com/badge.svg" alt="Agent View: 100% readable, 98% clean" height="20"></a>
The figures in one line
towardsdatascience.com in Agent View: 100% of pages readable, better than 48% of software sites; 98% clean; 643,539 tokens to read all 100 pages. mttr.si/agent-view/towardsdatascience.com
What to do next
Fix the pages, let an agent do it, and read the site again to see the figures move.
Connect your agent
Claude Code, Codex or Cursor reads the site’s fixes over MCP, fixes a page in your code, has mttr read it again, and checks the fix landed.
https://mttr.si/mcpSet up an agent
Claim the site
Prove it is yours with a DNS record or a file. Your agent can then reach its reads and jobs, and set each change to wait for your approval.
Claim the siteDrawn from the HTML mttr fetched on its read of October 23, 2026, not a screenshot, and only from the 100 pages read, where the read stopped at its page limit. An assistant’s own reader may differ. Tokens are estimated at about 4 characters each. A site an agent reads well is not a site an assistant cites.