A playwright enhancement with zyte smart proxy manager services
Use Playwright with
Smart Proxy Manager easily!
A wrapper over Playwright to provide Zyte Smart Proxy Manager specific functionalities.
1. Install Zyte SmartProxy Playwright
``
npm install zyte-smartproxy-playwright
2. Create a file sample.js with following content and replace with your SPM Apikey
` javascript
const { chromium } = require('zyte-smartproxy-playwright'); // Or 'firefox' or 'webkit'
(async () => {
const browser = await chromium.launch({
spm_apikey: '
headless: false,
static_bypass: false, // enable to save bandwidth (but may break some websites)
block_ads: false, // enable to save bandwidth (but may break some websites)
});
console.log('Before new page');
const page = await browser.newPage({ignoreHTTPSErrors: true});
console.log('Opening page ...');
try {
await page.goto('https://toscrape.com/', {timeout: 180000});
} catch(err) {
console.log(err);
}
console.log('Taking a screenshot ...');
await page.screenshot({path: 'screenshot.png'});
await browser.close();
})();
`
Make sure that you're able to make https requests using Smart Proxy Manager by following this guide Fetching HTTPS pages with Zyte Smart Proxy Manager
3. Run sample.js using Node
` bash`
node sample.js
launch accepts all the arguments accepted by firefox.launch or launch methods of other browser types
and some additional arguments defined below:
| Argument | Default Value | Description |
|----------|---------------|-------------|
| spm_apikey | undefined | Zyte Smart Proxy Manager API key that can be found on your zyte.com account. |spm_host
| | http://proxy.zyte.com:8011 | Zyte Smart Proxy Manager proxy host. |static_bypass
| | true | When true Zyte SmartProxy Playwright will skip proxy use for static assets defined by static_bypass_regex or pass false to use proxy. |static_bypass_regex
| | /.*?\.(?:txt\|json\|css\|less\|gif\|ico\|jpe?g\|svg\|png\|webp\|mkv\|mp4\|mpe?g\|webm\|eot\|ttf\|woff2?)$/ | Regex to use filtering URLs for static_bypass. |block_ads
| | true | When true Zyte SmartProxy Playwright will block ads defined by block_list using @cliqz/adblocker-playwright package. |block_list
| | ['https://secure.fanboy.co.nz/easylist.txt', 'https://secure.fanboy.co.nz/easyprivacy.txt'] | Block list to be used by Zyte SmartProxy Playwright in order to initiate blocker enginer using @cliqz/adblocker-playwright and block ads |headers
| | {'X-Crawlera-No-Bancheck': '1', 'X-Crawlera-Profile': 'pass', 'X-Crawlera-Cookies': 'disable'} | List of headers to be appended to requests |
and static_bypass enabled (default). Try to disable them if you encounter any issues.- When using the
headless: true mode, values generated for some browser-specific headers are a bit different, which may be detected by websites. Try using 'X-Crawlera-Profile': 'desktop' in that case:
` javascript
const browser = await chromium.launch({
spm_apikey: '',
headless: true,
headers: {'X-Crawlera-No-Bancheck': '1', 'X-Crawlera-Profile': 'desktop', 'X-Crawlera-Cookies': 'disable'}
});
`- When connecting to a remote Chrome browser instance, it should be launched with these arguments:
--proxy-server=http://proxy.zyte.com:8011 --disable-site-isolation-trials
- Consider our new zyte-smartproxy-plugin for playwright-extra
and puppeteer-extra frameworks.