High-performance SFCC catalog, inventory, and pricebook reducer for sandbox optimization
npm install sfcc-sandbox-reducerHigh-performance SFCC catalog reducer and image downloader for sandbox optimization. This toolkit analyzes SFCC catalogs, pricebooks, and inventories to create reduced datasets suitable for development and testing sandboxes.
- Node.js 18.0.0 or higher
- SFCC WebDAV access credentials
``bash`
npm install -g sfcc-sandbox-reducer
`bash`
npm install sfcc-sandbox-reducer
`bash`
git clone
cd sfcc-sandbox-reducer
npm install
npm link # Makes commands available globally
1. Create configuration files in your project directory:
`bashInteractive configuration wizard
sfcc-sandbox-reducer init
2. Edit the configuration files with your specific values.
3. Place your SFCC export files in the appropriate directories:
-
input/catalogs/ - Master and navigation catalogs
- input/pricebooks/ - Pricebook XML files
- input/inventory-lists/ - Inventory XML files4. Run the reducer:
`bash
sfcc-reduce --config reducer-config.json --output output
`5. Download images (optional):
`bash
sfcc-download-images
`Configuration
$3
`json
{
"webdav_base": "https://YOUR-SANDBOX.demandware.net/on/demandware.servlet/webdav/Sites/Catalogs/BRAND_master/default",
"input": {
"master_catalog": "./input/catalogs/BRAND_master/catalog.xml",
"navigation_catalogs": "./input/catalogs/BRAND_*_navigation/catalog.xml",
"pricebooks": "./input/pricebooks/*.xml",
"inventories": "./input/inventory-lists/*.xml"
},
"output": {
"directory": "./output"
},
"filters": {
"keep_online_products": true,
"sites_to_check": ["SITE_FR", "SITE_UK", "SITE_DE"],
"always_keep": [],
"always_remove": [],
"max_products": 10000
}
}
`$3
| Option | Description |
|--------|-------------|
|
webdav_base | Base URL for WebDAV image downloads |
| input.master_catalog | Path to master catalog (glob pattern) |
| input.navigation_catalogs | Path to navigation catalogs (glob pattern) |
| input.pricebooks | Path to pricebook files (glob pattern) |
| input.inventories | Path to inventory files (glob pattern) |
| filters.keep_online_products | Only keep products with online-flag=true |
| filters.sites_to_check | Site IDs to check for online status |
| filters.always_keep | Product IDs to always keep |
| filters.always_remove | Product IDs to always remove |
| filters.max_products | Maximum number of products (proportional reduction) |$3
`json
{
"username": "your-webdav-username",
"password": "your-webdav-password"
}
`CLI Usage
$3
`bash
Using npx
npx sfcc-sandbox-reducer Or add scripts to your package.json
`$3
Add these scripts to your project's
package.json:`json
{
"scripts": {
"reduce": "sfcc-sandbox-reducer reduce",
"download-images": "sfcc-sandbox-reducer download-images"
}
}
`$3
#### Catalog Reducer
`bash
sfcc-sandbox-reducer reduce [options]Options:
-c, --config Path to configuration file (default: "reducer-config.json")
-o, --output Output directory (default: "output")
-V, --verbose Enable verbose logging
--dry-run Analyze only, don't write files
-h, --help Display help
`Examples:
`bash
Run with default config
sfcc-sandbox-reducer reduceRun with custom config and output
sfcc-sandbox-reducer reduce --config my-config.json --output reducedPreview without writing files
sfcc-sandbox-reducer reduce --dry-runShortcut command (if installed globally)
sfcc-reduce --config reducer-config.json
`#### Image Downloader
`bash
sfcc-sandbox-reducer download-images [options]Options:
-c, --category Category ID to download images for
--catalog Navigation catalog name
-o, --output Output directory (default: "downloaded-images")
--concurrency Number of parallel downloads (default: 10)
-V, --verbose Enable verbose logging
-h, --help Display help
`Examples:
`bash
Interactive mode (prompts for catalog and category)
sfcc-sandbox-reducer download-imagesNon-interactive mode
sfcc-sandbox-reducer download-images --catalog BRAND_FR_navigation --category mens-clothingWith higher concurrency
sfcc-sandbox-reducer download-images --concurrency 20Shortcut command (if installed globally)
sfcc-download-images
`#### Configuration Init
`bash
sfcc-sandbox-reducer init [options]Options:
-y, --yes Skip prompts and use defaults (non-interactive)
-f, --force Overwrite existing configuration files
--hostname SFCC sandbox hostname
--catalog Master catalog name
--sites Comma-separated list of site IDs
--max-products Maximum number of products to keep (default: 10000)
--with-credentials Also create dw.json credentials file
-h, --help Display help
`Examples:
`bash
Interactive configuration wizard
sfcc-sandbox-reducer initCreate config with all defaults
sfcc-sandbox-reducer init --yesNon-interactive with specific options
sfcc-sandbox-reducer init --hostname xxxx.dx.commercecloud.salesforce.com --catalog brand_master --sites FR,UKForce overwrite existing config
sfcc-sandbox-reducer init --forceShortcut command (if installed globally)
sfcc-init
`$3
| Command | Aliases |
|---------|---------|
|
reduce | r |
| download-images | dl, images |
| init | i, create, setup |How It Works
$3
1. Phase 1: Analysis
- Parses master catalogs using streaming XML
- Builds product dependency graph (masters → variants → variation-groups)
- Identifies online products per site
- Scans navigation catalogs for category assignments
- Scans pricebooks and inventories for product presence
2. Phase 2: Filtering
- Keeps products that are:
- Online (have online-flag=true for specified sites)
- Have a variation-group assigned to navigation categories
- Have at least one variant with both price AND inventory
- Applies proportional reduction if max_products is specified
3. Phase 3: Output
- Writes reduced XML files with only qualifying products
- Removes deprecated SFCC elements
- Preserves directory structure
$3
SFCC products have a hierarchical structure:
- Master Product - The parent product
- Variation Groups - Appear in navigation (PLPs)
- Variants - The actual sellable SKUs
The reducer keeps entire product groups where:
- At least one variation-group has a navigation category assignment
- At least one variant has both a price record AND an inventory record
Example Output
`
SFCC Catalog Reducer v1.0.0
==================================================
[OK] Loaded configuration from reducer-config.json[PHASE 1] Collecting product IDs...
Found 15 files
Scanning master catalogs...
catalog.xml (450.32 MB)
-> 50000 online products (masters + variants)
-> 5000 master products with variants
-> 2500 variation groups
Scanning navigation catalogs...
-> 3000 products in navigation
Scanning pricebooks...
-> 45000 products with prices
Scanning inventories...
-> 40000 products with inventory records
Computing product groups...
-> 2000 product groups kept, 500 removed
[OK] Phase 1 complete in 12.34s
Keeping 25000 products (50.0% reduction)
[PHASE 2] Reducing files...
Master catalogs: . 5000 kept, 45000 removed
Navigation catalogs: ...... 3000 kept, 0 removed
Inventories: ...... 25000 kept, 15000 removed
Pricebooks: .......... 25000 kept, 20000 removed
[OK] Phase 2 complete in 45.67s
==================================================
REDUCTION SUMMARY
==================================================
Product Filtering:
Total in master catalog: 50000
Online products: 50000
In navigation catalogs: 3000
With price records: 45000
With inventory records: 40000
Final kept: 25000
File Statistics:
Files processed: 23
Records kept: 58000
Records removed: 80000
Size Reduction:
Original size: 1.25 GB
Reduced size: 625.00 MB
Saved: 625.00 MB (50.0%)
[DONE] Total processing time: 58.01s
`Programmatic Usage
`javascript
import { CatalogReducer } from 'sfcc-sandbox-reducer/lib/reducer.js';
import { loadConfig } from 'sfcc-sandbox-reducer/lib/config.js';const config = await loadConfig('reducer-config.json');
const reducer = new CatalogReducer(config, 'output', { verbose: true });
// Phase 1: Collect and analyze
await reducer.collectProductIds();
// Phase 2: Write reduced files
await reducer.reduceAll();
// Get statistics
reducer.printSummary();
`File Structure
`
your-project/
├── dw.json # WebDAV credentials (git-ignored)
├── reducer-config.json # Configuration (git-ignored)
├── input/
│ ├── catalogs/
│ │ ├── BRAND_master/
│ │ │ └── catalog.xml
│ │ ├── BRAND_FR_navigation/
│ │ │ └── catalog.xml
│ │ └── ...
│ ├── inventory-lists/
│ │ └── *.xml
│ └── pricebooks/
│ └── *.xml
├── output/ # Reduced files (git-ignored)
│ ├── catalogs/
│ ├── inventory-lists/
│ └── pricebooks/
└── downloaded-images/ # Downloaded images (git-ignored)
`Tips
- First run is slower - Subsequent runs use cached analysis
- Use
--dry-run to preview reduction without writing files
- Adjust max_products to control output size
- Check .reducer-cache/ if you need to force re-analysis (delete it)Troubleshooting
$3
Make sure you've run the reducer first to generate output catalogs.$3
Create a dw.json file with your WebDAV credentials.$3
The tool uses streaming XML parsing, so memory shouldn't be an issue. If you encounter problems, try reducing concurrency: CONCURRENCY=5 sfcc-download-images`AGPL-3.0-or-later