Search Engine Optimization
135 views
Robots.txt
Quick Definition
Text file instructing crawlers which pages to crawl
Full Definition
A text file instructing web crawlers which pages or sections of a site to crawl or skip.
Examples
crawl budget management, blocking pages, search engine directives
Related Terms
technical-seo
xml-sitemap
crawlability
More Search Engine Optimization Terms
Off-Page SEO
Activities outside the website to improve rankings
Structured Data
Standardized format for providing page info to search engines
Domain Authority
Metric predicting likelihood of ranking in search results
301 Redirect
Permanent redirect passing link equity between URLs
Technical SEO
Optimizing website infrastructure for search engines
Schema Markup
Structured data vocabulary for rich search results