How to Block Playwright
Complete guide to blocking Playwright (Microsoft) from crawling your website using robots.txt, server configuration, and Switch workflows.
Should You Block Playwright?
Playwright controls a real browser and interacts with your site like a human. It cannot be blocked via robots.txt because it doesn't use identifiable user-agent strings.
Use behavioral detection through Switch to identify and manage browser agent traffic.
Blocking Methods
1robots.txt
High for cooperative crawlersAdd a Disallow rule for Playwright's user-agent string in your robots.txt file. This is the standard, cooperative method that well-behaved crawlers respect.
2Server-side UA filtering
HighConfigure your web server (nginx, Apache, Cloudflare) to reject requests matching Playwright's user-agent patterns. This blocks at the network level before your application processes the request.
3Behavioral detection
Medium — requires specialized toolingPlaywright uses a real browser and doesn't identify itself via user-agent strings. Detection requires analyzing automation flags, interaction patterns, and JavaScript environment signals.
4Switch Content Gate
High for automated browsersSwitch's Content Gate uses document.write() to prevent headless browsers and automation frameworks from accessing your page content. Effective against Puppeteer, Playwright, and Selenium-based agents.
5Switch Journey Workflows
Highest — granular, real-time controlCreate a custom journey in Switch that detects Playwright and routes it to a block action, challenge, redirect, or modified content — without touching your server configuration.
robots.txt — Block Playwright
Add the following to your robots.txt file (at the root of your domain) to block Playwright:
User-agent: Playwright Disallow: / User-agent: HeadlessChrome/ Disallow: /
robots.txt — Allow with Restrictions
Alternatively, allow Playwright on most pages while blocking specific directories:
User-agent: Playwright Disallow: /private/ Allow: / User-agent: HeadlessChrome/ Disallow: /private/ Allow: /
Playwright User-Agent Strings
Use these patterns to identify Playwright in your server logs or firewall rules:
Frequently Asked Questions
Does blocking Playwright affect my Google search rankings?
No. Blocking Playwright does not affect your Google search rankings. Only blocking Googlebot impacts Google Search visibility.
Does Playwright respect robots.txt?
Yes, Playwright respects robots.txt directives. Adding a Disallow rule for its user-agent will prevent it from crawling blocked paths.
Can I allow Playwright on some pages but not others?
Yes. Use robots.txt to disallow specific directories, or use Switch journey workflows for granular page-level control with conditional logic.
Go beyond robots.txt
Switch detects Playwright in real-time and lets you build custom journey workflows — block, challenge, redirect, or serve modified content. No server changes required.
Get Started Free