npm install webform-privacy-consent-scanner

Webform Privacy Consent Scanner

![CI](https://github.com/c3nk/webform-privacy-consent-scanner/actions)
![License: MIT](https://opensource.org/licenses/MIT)
![Node.js Version](https://nodejs.org/)
![npm version](https://www.npmjs.com/package/webform-privacy-consent-scanner)

Advanced web form scanner detecting Google Forms, HubSpot Forms, Microsoft Forms, Formstack Forms with comprehensive CMP detection including Cookiebot, OneTrust, Efilli, and GDPR compliance auditing. Also detects 3rd party application links and embedded forms on websites.

!Webform Privacy Consent Scanner — detects Google/HubSpot/Microsoft forms and CMPs (Cookiebot, OneTrust, Efilli)

🚀 Quick Start

``bash

`Install`


npm install -g webform-privacy-consent-scanner
Basic scan with CMP detection

webform-scanner --input urls.txt --out results.csv --cmp
Full scan with dynamic rendering (default wait: 6000ms)

webform-scanner --input urls.txt --dynamic --cmp


📋 Table of Contents
- Features
- Installation
- Usage
- Supported Platforms
- Output Formats
- Examples
- Contributing
- License
- Security
✨ Features
- 🔍 Multi-Platform Form Detection: Google Forms, HubSpot Forms, Microsoft Forms, Formstack Forms, Target Website Forms
- 🔗 3rd Party Application Link Detection: Detects links to external applications and forms on websites
- 🍪 Comprehensive CMP Detection: Cookiebot, OneTrust, Efilli, GTM, Generic GDPR
- 🌐 Advanced Scanning: Static HTML + optional Playwright dynamic rendering
- 🔄 Smart Fallback: Automatic curl fallback for blocked requests
- 📊 Multiple Outputs: CSV, JSON, filtered text reports
- ⚡ High Performance: Concurrent scanning with configurable limits
- 🔒 Privacy Focused: Respects robots.txt, ethical scanning practices
- 🎯 CLI First: Powerful command-line interface with extensive options
📦 Installation
$3

- Node.js >= 18
- npm
$3

- Playwright (for dynamic scanning):

npm install -D playwright && npx playwright install

$3

bash
npm install -g webform-privacy-consent-scanner

$3

bash
git clone https://github.com/c3nk/webform-privacy-consent-scanner.git
cd webform-privacy-consent-scanner
npm install
Optional: Install Playwright for dynamic scanning

npm install -D playwright
npx playwright install


🎯 Usage
$3

bash
Scan URLs with CMP detection

webform-scanner --input urls.txt --cmp
Output: results_2025-01-15T10-30-00.csv

        results_2025-01-15T10-30-00.json

$3

bash
Dynamic scanning with custom wait

webform-scanner --input urls.txt --dynamic --wait 10000 --cmp
High concurrency for large lists

webform-scanner --input large-list.txt --concurrency 16 --timeout 20000
Custom output location

webform-scanner --input urls.txt --out my-scan-results.csv
Collector pattern detection (DOM analysis)

webform-scanner --input urls.txt --collectors "forms.example.com/,yourbrand.formstack.com/" --cmp

$3

bash
Filter Google Forms

node filter.mjs --attr is_google_form --value true
Filter by CMP vendor

node filter.mjs --attr cmp_vendor --value Cookiebot
Case-insensitive search

node filter.mjs --attr url --value example.com --ci --contains


🎪 Supported Platforms
$3

- Google Forms: Direct URLs, embedded iframes, form actions
- HubSpot Forms: Script detection, API endpoints, inline JavaScript
- Microsoft Forms: Response pages, short URLs, Office UI framework
- Formstack Forms: Hosted forms, embed scripts, iframe integration
$3

- Cookiebot: Popular EU CMP solution
- OneTrust: Enterprise-grade consent management
- Efilli: Turkish CMP platform
- Google Tag Manager: GTM-loaded CMP detection
- Generic: Standard GDPR/cookie consent banners
📊 Output Formats
$3

csv
url,method,status,is_target_website_form,is_yourcompany_form,is_example_form,is_google_form,is_hubspot_form,is_microsoft_form,is_formstack_form,detected_types,evidence,has_cmp,cmp_vendor,cmp_evidence,collectors_detected,collector_link_count,collector_embed_count,linked_forms_detected,linked_forms_count,note
https://www.c3nk.com/examples/forms/hubspot.html?cmp=cookiebot&mode=mock,dynamic,200,false,true,false,["hubspot"],"hbspt.forms.create(",true,"Cookiebot","Cookiebot",true,2,1,
https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock,dynamic,200,true,false,false,["google"],"docs.google.com/forms",true,"Cookiebot","cookiebot",false,0,0,
https://www.c3nk.com/examples/forms/microsoft.html?cmp=efilli&mode=mock,dynamic,200,false,false,true,["microsoft"],"forms.office.com",true,"Cookiebot","cookiebot",true,1,1,

$3

json
[
  {
    "url": "https://www.c3nk.com/examples/forms/hubspot.html?cmp=cookiebot&mode=mock",
    "method": "dynamic",
    "status": 200,
    "is_google_form": false,
    "is_hubspot_form": true,
    "is_microsoft_form": false,
    "detected_types": ["hubspot"],
    "evidence": "hbspt.forms.create(",
    "has_cmp": true,
    "cmp_vendor": "Cookiebot",
    "cmp_evidence": "Cookiebot",
    "collectors_detected": true,
    "collector_link_count": 2,
    "collector_embed_count": 1,
    "collectors": [
      {
        "target_pattern": "forms.example.com/*",
        "matched_url": "https://forms.example.com/contact",
        "relation": "link",
        "match_type": "wildcard",
        "text_or_context": "Contact us for more information"
      }
    ],
    "note": ""
  },
  {
    "url": "https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock",
    "method": "dynamic",
    "status": 200,
    "is_google_form": true,
    "is_hubspot_form": false,
    "is_microsoft_form": false,
    "detected_types": ["google"],
    "evidence": "docs.google.com/forms",
    "has_cmp": true,
    "cmp_vendor": "Cookiebot",
    "cmp_evidence": "cookiebot",
    "collectors_detected": false,
    "collector_link_count": 0,
    "collector_embed_count": 0,
    "collectors": [],
    "note": ""
  }
]

$3


FILTER REPORT
=============
Input file: test-results.json
Filter: is_google_form = true
Total results: 11
Filtered results: 1

RESULTS: -------- https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock`

`🔧 CLI Options`

| Option | Description | Default | |--------|-------------|---------| |--input | Input file with URLs | Required | |--out | Output file path | results_TIMESTAMP.csv| |--concurrency | Number of concurrent requests | 8| |--timeout | Request timeout | 15000| |--dynamic | Enable dynamic scanning | false| |--wait | Wait time for dynamic content (default: 6000ms) | 6000| |--cmp | Enable CMP detection | false| |--collectors | Comma-separated glob patterns for link/iframe/script detection | None |

`📈 Examples`

`$3`


- HubSpot + Cookiebot (mock signatures): https://www.c3nk.com/examples/forms/hubspot.html?cmp=cookiebot&mode=mock
- Google + OneTrust (mock signatures): https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock
- Microsoft + Efilli (mock signatures): https://www.c3nk.com/examples/forms/microsoft.html?cmp=efilli&mode=mock

> mock mode embeds only detection signatures (no third‑party requests). > Use&mode=live to actually load vendor scripts for visual checks.

`$3`


We ship three mock pages under

examples/forms/

 with the same signature logic.
$3

bash
echo "https://c3nk.com/" > urls.txt
webform-scanner --input urls.txt --cmp

$3

bash
webform-scanner --input company-websites.txt --concurrency 20 --cmp --dynamic

$3

bash
webform-scanner --input eu-websites.txt --cmp --out gdpr-audit.csv

$3

bash
Detect links to specific forms

webform-scanner --input urls.txt --collectors "forms.example.com/,yourbrand.formstack.com/"
Combine with form detection and CMP

webform-scanner --input urls.txt --collectors ".formstack.com/forms/" --cmp --dynamic
Multiple patterns for comprehensive analysis

webform-scanner --input company-sites.txt --collectors "contact.example.com/,forms.example.com/,apply.example.com/*"

$3

bash
npm run scan:static  # Uses examples/urls.sample.txt
npm run scan:full    # Dynamic scanning with examples


🤝 Contributing
We welcome contributions! Please see our Contributing Guide for details.
$3

bash
npm run start          # Show help
npm run scan:static    # Test static scanning
npm run scan:full      # Test dynamic scanning


📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
🔒 Security
Please see our Security Policy for responsible disclosure practices.
$3

- Only scan websites you own or have permission to test
- Respect robots.txt and website terms of service
- Use for legitimate privacy compliance auditing only
🗺️ Roadmap
- [ ] Additional CMP platform support
- [ ] Advanced filtering options
- [ ] Web interface
- [ ] API endpoints
- [ ] Docker containerization
- [ ] Integration with popular CI/CD platforms
📞 Support
- 📧 Email: me@c3nk.com
- 🐛 Issues: GitHub Issues
- 📖 Documentation: User Guide | User Guide (TR)
---
Built with ❤️ by c3nk.com
---
Webform Privacy Consent Scanner (Türkçe)
![CI](https://github.com/c3nk/webform-privacy-consent-scanner/actions)
![Lisans: MIT](https://opensource.org/licenses/MIT)
Google Forms, HubSpot Forms, Microsoft Forms tespit eden gelişmiş web form tarayıcı. Cookiebot, OneTrust, Efilli dahil kapsamlı CMP tespiti ve GDPR uyumluluk denetimi.
🚀 Hızlı Başlangıç

`bash

`Kurulum`


npm install -g webform-privacy-consent-scanner
Temel tarama

webform-scanner --input urls.txt --cmp
Tam tarama (dinamik, varsayılan bekleme: 6000ms)

webform-scanner --input urls.txt --dynamic --cmp


✨ Özellikler
- 🔍 Çoklu Platform Form Tespiti: Google Forms, HubSpot Forms, Microsoft Forms
- 🍪 Kapsamlı CMP Tespiti: Cookiebot, OneTrust, Efilli, GTM, Genel GDPR
- 🌐 Gelişmiş Tarama: Statik HTML + isteğe bağlı Playwright dinamik render
- 🔄 Akıllı Fallback: Engellenen istekler için otomatik curl fallback
- 📊 Çoklu Çıktı: CSV, JSON, filtrelenmiş metin raporları
- ⚡ Yüksek Performans: Yapılandırılabilir limitlerle eşzamanlı tarama
- 🔒 Gizlilik Odaklı: robots.txt'e saygı, etik tarama uygulamaları
📦 Kurulum

`bash

`Global kurulum`


npm install -g webform-privacy-consent-scanner
Yerel geliştirme

git clone https://github.com/c3nk/webform-privacy-consent-scanner.git
cd webform-privacy-consent-scanner
npm install


🎯 Kullanım
$3

bash
URL'leri CMP tespiti ile tara

webform-scanner --input urls.txt --cmp

$3

bash
Dinamik tarama

webform-scanner --input urls.txt --dynamic --wait 10000 --cmp
Yüksek eşzamanlılık

webform-scanner --input buyuk-liste.txt --concurrency 16 --timeout 20000

$3

bash
Google formlarını filtrele

node filter.mjs --attr is_google_form --value true
CMP sağlayıcısına göre filtrele

node filter.mjs --attr cmp_vendor --value Cookiebot

🎪 Desteklenen Platformlar

$3

- Google Forms: Doğrudan URL'ler, gömülü iframe'ler, form eylemleri
- HubSpot Forms: Script tespiti, API uç noktaları, satır içi JavaScript
- Microsoft Forms: Yanıt sayfaları, kısa URL'ler, Office UI framework

$3

- Cookiebot: Popüler AB CMP çözümü
- OneTrust: Kurumsal düzeyde onay yönetimi
- Efilli: Türk CMP platformu
- Google Tag Manager: GTM üzerinden yüklenen CMP tespiti
- Genel: Standart GDPR/çerez onay bannerları

🤝 Katkıda Bulunma

Katkılarınızı bekliyoruz! Detaylar için Katkıda Bulunma Kılavuzu'na bakın.

📄 Lisans

Bu proje MIT Lisansı altında lisanslanmıştır - detaylar için LICENSE dosyasına bakın.

🔒 Güvenlik

Sorumlu açıklama uygulamaları için Güvenlik Politikası'mıza bakın.

Sorumlu Kullanım:
- Sadece sahip olduğunuz veya test izniniz olan web sitelerini tarayın
- robots.txt'e ve web sitesi kullanım koşullarına saygı gösterin
- Sadece meşru gizlilik uyumluluk denetimi için kullanın

---

❤️ ile yapıldı: c3nk.com

npm install webform-privacy-consent-scanner

Webform Privacy Consent Scanner

!Webform Privacy Consent Scanner — detects Google/HubSpot/Microsoft forms and CMPs (Cookiebot, OneTrust, Efilli)

🚀 Quick Start

``bash

`Install`


npm install -g webform-privacy-consent-scanner
Basic scan with CMP detection

webform-scanner --input urls.txt --out results.csv --cmp
Full scan with dynamic rendering (default wait: 6000ms)

webform-scanner --input urls.txt --dynamic --cmp


📋 Table of Contents
- Features
- Installation
- Usage
- Supported Platforms
- Output Formats
- Examples
- Contributing
- License
- Security
✨ Features
- 🔍 Multi-Platform Form Detection: Google Forms, HubSpot Forms, Microsoft Forms, Formstack Forms, Target Website Forms
- 🔗 3rd Party Application Link Detection: Detects links to external applications and forms on websites
- 🍪 Comprehensive CMP Detection: Cookiebot, OneTrust, Efilli, GTM, Generic GDPR
- 🌐 Advanced Scanning: Static HTML + optional Playwright dynamic rendering
- 🔄 Smart Fallback: Automatic curl fallback for blocked requests
- 📊 Multiple Outputs: CSV, JSON, filtered text reports
- ⚡ High Performance: Concurrent scanning with configurable limits
- 🔒 Privacy Focused: Respects robots.txt, ethical scanning practices
- 🎯 CLI First: Powerful command-line interface with extensive options
📦 Installation
$3

- Node.js >= 18
- npm
$3

- Playwright (for dynamic scanning):

npm install -D playwright && npx playwright install

$3

bash
npm install -g webform-privacy-consent-scanner

$3

bash
git clone https://github.com/c3nk/webform-privacy-consent-scanner.git
cd webform-privacy-consent-scanner
npm install
Optional: Install Playwright for dynamic scanning

npm install -D playwright
npx playwright install


🎯 Usage
$3

bash
Scan URLs with CMP detection

webform-scanner --input urls.txt --cmp
Output: results_2025-01-15T10-30-00.csv

        results_2025-01-15T10-30-00.json

$3

bash
Dynamic scanning with custom wait

webform-scanner --input urls.txt --dynamic --wait 10000 --cmp
High concurrency for large lists

webform-scanner --input large-list.txt --concurrency 16 --timeout 20000
Custom output location

webform-scanner --input urls.txt --out my-scan-results.csv
Collector pattern detection (DOM analysis)

webform-scanner --input urls.txt --collectors "forms.example.com/,yourbrand.formstack.com/" --cmp

$3

bash
Filter Google Forms

node filter.mjs --attr is_google_form --value true
Filter by CMP vendor

node filter.mjs --attr cmp_vendor --value Cookiebot
Case-insensitive search

node filter.mjs --attr url --value example.com --ci --contains


🎪 Supported Platforms
$3

- Google Forms: Direct URLs, embedded iframes, form actions
- HubSpot Forms: Script detection, API endpoints, inline JavaScript
- Microsoft Forms: Response pages, short URLs, Office UI framework
- Formstack Forms: Hosted forms, embed scripts, iframe integration
$3

- Cookiebot: Popular EU CMP solution
- OneTrust: Enterprise-grade consent management
- Efilli: Turkish CMP platform
- Google Tag Manager: GTM-loaded CMP detection
- Generic: Standard GDPR/cookie consent banners
📊 Output Formats
$3

csv
url,method,status,is_target_website_form,is_yourcompany_form,is_example_form,is_google_form,is_hubspot_form,is_microsoft_form,is_formstack_form,detected_types,evidence,has_cmp,cmp_vendor,cmp_evidence,collectors_detected,collector_link_count,collector_embed_count,linked_forms_detected,linked_forms_count,note
https://www.c3nk.com/examples/forms/hubspot.html?cmp=cookiebot&mode=mock,dynamic,200,false,true,false,["hubspot"],"hbspt.forms.create(",true,"Cookiebot","Cookiebot",true,2,1,
https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock,dynamic,200,true,false,false,["google"],"docs.google.com/forms",true,"Cookiebot","cookiebot",false,0,0,
https://www.c3nk.com/examples/forms/microsoft.html?cmp=efilli&mode=mock,dynamic,200,false,false,true,["microsoft"],"forms.office.com",true,"Cookiebot","cookiebot",true,1,1,

$3

json
[
  {
    "url": "https://www.c3nk.com/examples/forms/hubspot.html?cmp=cookiebot&mode=mock",
    "method": "dynamic",
    "status": 200,
    "is_google_form": false,
    "is_hubspot_form": true,
    "is_microsoft_form": false,
    "detected_types": ["hubspot"],
    "evidence": "hbspt.forms.create(",
    "has_cmp": true,
    "cmp_vendor": "Cookiebot",
    "cmp_evidence": "Cookiebot",
    "collectors_detected": true,
    "collector_link_count": 2,
    "collector_embed_count": 1,
    "collectors": [
      {
        "target_pattern": "forms.example.com/*",
        "matched_url": "https://forms.example.com/contact",
        "relation": "link",
        "match_type": "wildcard",
        "text_or_context": "Contact us for more information"
      }
    ],
    "note": ""
  },
  {
    "url": "https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock",
    "method": "dynamic",
    "status": 200,
    "is_google_form": true,
    "is_hubspot_form": false,
    "is_microsoft_form": false,
    "detected_types": ["google"],
    "evidence": "docs.google.com/forms",
    "has_cmp": true,
    "cmp_vendor": "Cookiebot",
    "cmp_evidence": "cookiebot",
    "collectors_detected": false,
    "collector_link_count": 0,
    "collector_embed_count": 0,
    "collectors": [],
    "note": ""
  }
]

$3


FILTER REPORT
=============
Input file: test-results.json
Filter: is_google_form = true
Total results: 11
Filtered results: 1

RESULTS: -------- https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock`

`🔧 CLI Options`

`📈 Examples`

`$3`


- HubSpot + Cookiebot (mock signatures): https://www.c3nk.com/examples/forms/hubspot.html?cmp=cookiebot&mode=mock
- Google + OneTrust (mock signatures): https://www.c3nk.com/examples/forms/google.html?cmp=onetrust&mode=mock
- Microsoft + Efilli (mock signatures): https://www.c3nk.com/examples/forms/microsoft.html?cmp=efilli&mode=mock

> mock mode embeds only detection signatures (no third‑party requests). > Use&mode=live to actually load vendor scripts for visual checks.

`$3`


We ship three mock pages under

examples/forms/

 with the same signature logic.
$3

bash
echo "https://c3nk.com/" > urls.txt
webform-scanner --input urls.txt --cmp

$3

bash
webform-scanner --input company-websites.txt --concurrency 20 --cmp --dynamic

$3

bash
webform-scanner --input eu-websites.txt --cmp --out gdpr-audit.csv

$3

bash
Detect links to specific forms

webform-scanner --input urls.txt --collectors "forms.example.com/,yourbrand.formstack.com/"
Combine with form detection and CMP

webform-scanner --input urls.txt --collectors ".formstack.com/forms/" --cmp --dynamic
Multiple patterns for comprehensive analysis

webform-scanner --input company-sites.txt --collectors "contact.example.com/,forms.example.com/,apply.example.com/*"

$3

bash
npm run scan:static  # Uses examples/urls.sample.txt
npm run scan:full    # Dynamic scanning with examples


🤝 Contributing
We welcome contributions! Please see our Contributing Guide for details.
$3

bash
npm run start          # Show help
npm run scan:static    # Test static scanning
npm run scan:full      # Test dynamic scanning


📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
🔒 Security
Please see our Security Policy for responsible disclosure practices.
$3

- Only scan websites you own or have permission to test
- Respect robots.txt and website terms of service
- Use for legitimate privacy compliance auditing only
🗺️ Roadmap
- [ ] Additional CMP platform support
- [ ] Advanced filtering options
- [ ] Web interface
- [ ] API endpoints
- [ ] Docker containerization
- [ ] Integration with popular CI/CD platforms
📞 Support
- 📧 Email: me@c3nk.com
- 🐛 Issues: GitHub Issues
- 📖 Documentation: User Guide | User Guide (TR)
---
Built with ❤️ by c3nk.com
---
Webform Privacy Consent Scanner (Türkçe)
![CI](https://github.com/c3nk/webform-privacy-consent-scanner/actions)
![Lisans: MIT](https://opensource.org/licenses/MIT)
Google Forms, HubSpot Forms, Microsoft Forms tespit eden gelişmiş web form tarayıcı. Cookiebot, OneTrust, Efilli dahil kapsamlı CMP tespiti ve GDPR uyumluluk denetimi.
🚀 Hızlı Başlangıç

`bash

`Kurulum`


npm install -g webform-privacy-consent-scanner
Temel tarama

webform-scanner --input urls.txt --cmp
Tam tarama (dinamik, varsayılan bekleme: 6000ms)

webform-scanner --input urls.txt --dynamic --cmp


✨ Özellikler
- 🔍 Çoklu Platform Form Tespiti: Google Forms, HubSpot Forms, Microsoft Forms
- 🍪 Kapsamlı CMP Tespiti: Cookiebot, OneTrust, Efilli, GTM, Genel GDPR
- 🌐 Gelişmiş Tarama: Statik HTML + isteğe bağlı Playwright dinamik render
- 🔄 Akıllı Fallback: Engellenen istekler için otomatik curl fallback
- 📊 Çoklu Çıktı: CSV, JSON, filtrelenmiş metin raporları
- ⚡ Yüksek Performans: Yapılandırılabilir limitlerle eşzamanlı tarama
- 🔒 Gizlilik Odaklı: robots.txt'e saygı, etik tarama uygulamaları
📦 Kurulum

`bash

`Global kurulum`


npm install -g webform-privacy-consent-scanner
Yerel geliştirme

git clone https://github.com/c3nk/webform-privacy-consent-scanner.git
cd webform-privacy-consent-scanner
npm install


🎯 Kullanım
$3

bash
URL'leri CMP tespiti ile tara

webform-scanner --input urls.txt --cmp

$3

bash
Dinamik tarama

webform-scanner --input urls.txt --dynamic --wait 10000 --cmp
Yüksek eşzamanlılık

webform-scanner --input buyuk-liste.txt --concurrency 16 --timeout 20000

$3

bash
Google formlarını filtrele

node filter.mjs --attr is_google_form --value true
CMP sağlayıcısına göre filtrele

node filter.mjs --attr cmp_vendor --value Cookiebot

🎪 Desteklenen Platformlar

$3

🤝 Katkıda Bulunma

Katkılarınızı bekliyoruz! Detaylar için Katkıda Bulunma Kılavuzu'na bakın.

📄 Lisans

Bu proje MIT Lisansı altında lisanslanmıştır - detaylar için LICENSE dosyasına bakın.

🔒 Güvenlik

Sorumlu açıklama uygulamaları için Güvenlik Politikası'mıza bakın.

---

❤️ ile yapıldı: c3nk.com