Search Engine Optimization
27 views
Robots.txt
Quick Definition
Text file instructing crawlers which pages to crawl
Full Definition
A text file instructing web crawlers which pages or sections of a site to crawl or skip.
Examples
crawl budget management, blocking pages, search engine directives
Related Terms
technical-seo
xml-sitemap
crawlability
More Search Engine Optimization Terms
Topic Clusters
Content architecture with pillar pages linking to subtopic content
Search Intent
The underlying purpose behind a search query
Backlinks
Links from other websites signaling authority
Organic Traffic
Visitors from unpaid search engine results
Google Business Profile
Free tool managing business presence on Google
301 Redirect
Permanent redirect passing link equity between URLs