Having some web scrapping fun with Puppeteer
npm install bkd-puppeteerWe use Puppeteer to initially try and fetch the first nav > ul > li > a of a given domain.
For every qualifying navigation item we find, we push the title and href values into an array.
Then we simply iterate through that array, goto that url and take both a default deview viewport screenshot as well as a full-page screenshot.
https://github.com/ScoutsOut/bkd-puppeteernpm i.env file in the root of the project's directory
HOST=https://npr.com
USER=yourUserName
PASS=yourPassword
DESTINATION=build-directory-of-choice
`Where generated screenshots are generated
- Whichever directory you've dicated within .evn, the Puppeteer bot will create directories by page and device.
- The file names have the following format: Page name - Device name - Height - width.jpg
- Realworld example: `Contact Us-iPad-768x1024.fullscreen.jpg`###Available Scripts
-
npm run puppet
- npm run puppet-dev` - only difference is the use of nodemon#####Hopefully someone out there will find this helpful. I'm going to continue to extend this (mostly for my own personal development).
If anyone has suggestions...feel free to throw ideas out there :)