A TypeScript library for scraping game data from itch.io with a clean, scalable architecture
npm install chimi-scraperChimi Scraper is a professional-grade TypeScript library designed for extracting comprehensive game data from itch.io. Built with a robust, scalable architecture, it enables developers to efficiently scrape game information and build powerful APIs for indie game aggregation platforms, analytics tools, and data-driven applications.
Chimi Scraper provides programmatic access to itch.io's extensive game catalog through web scraping techniques. The library abstracts the complexity of HTML parsing and data extraction, offering a clean, type-safe interface for accessing game metadata, pricing information, developer details, and media assets.
- Comprehensive Game Discovery: Access curated game collections including New & Popular, Top Sellers, Top Rated, and newest releases
- Advanced Search: Query games by keywords with pagination support
- Rich Metadata Extraction: Retrieve detailed game information including descriptions, screenshots, videos, pricing, platform compatibility, and developer information
- Production-Ready Architecture: Provider-based design with abstract base classes for easy extensibility
- Type Safety: Full TypeScript support with comprehensive interface definitions
- Performance Optimized: Minimal dependencies with efficient scraping algorithms
``bash`
npm install chimi-scraper
`typescript
import { GAMES } from 'chimi-scraper';
const scraper = new GAMES.ItchIO();
// Fetch trending games
const trending = await scraper.fetchNewAndPopular(1);
console.log(Found ${trending.results.length} trending games);
// Search for specific games
const searchResults = await scraper.search('puzzle platformer', 1);
console.log(searchResults.results);
// Get detailed game information
const gameDetails = await scraper.fetchGameInfo(searchResults.results[0].url);
console.log(gameDetails);
`
`javascript
const { GAMES } = require('chimi-scraper');
const scraper = new GAMES.ItchIO();
async function fetchGames() {
try {
const topSellers = await scraper.fetchTopSellers(1);
return topSellers.results;
} catch (error) {
console.error('Failed to fetch games:', error);
throw error;
}
}
`
Chimi Scraper is designed to be the foundation for building robust game data APIs. Here's how to create a RESTful API:
`typescript
import express from 'express';
import { GAMES } from 'chimi-scraper';
const app = express();
const scraper = new GAMES.ItchIO();
// Get trending games
app.get('/api/games/trending', async (req, res) => {
try {
const page = parseInt(req.query.page as string) || 1;
const games = await scraper.fetchNewAndPopular(page);
res.json(games);
} catch (error) {
res.status(500).json({ error: 'Failed to fetch trending games' });
}
});
// Search games
app.get('/api/games/search', async (req, res) => {
try {
const query = req.query.q as string;
const page = parseInt(req.query.page as string) || 1;
if (!query) {
return res.status(400).json({ error: 'Query parameter required' });
}
const results = await scraper.search(query, page);
res.json(results);
} catch (error) {
res.status(500).json({ error: 'Search failed' });
}
});
// Get detailed game information
app.get('/api/games/:id', async (req, res) => {
try {
const gameUrl = decodeURIComponent(req.params.id);
const gameInfo = await scraper.fetchGameInfo(gameUrl);
res.json(gameInfo);
} catch (error) {
res.status(500).json({ error: 'Failed to fetch game details' });
}
});
app.listen(3000, () => {
console.log('Game API running on port 3000');
});
`
RESTful Endpoints Structure:
``
GET /api/games/trending?page=1
GET /api/games/top-sellers?page=1
GET /api/games/top-rated?page=1
GET /api/games/newest?page=1
GET /api/games/search?q=horror&page=1
GET /api/games/details/:gameId
Response Caching:
`typescript
import NodeCache from 'node-cache';
const cache = new NodeCache({ stdTTL: 600 }); // 10 minute cache
app.get('/api/games/trending', async (req, res) => {
const cacheKey = trending-${req.query.page || 1};`
const cached = cache.get(cacheKey);
if (cached) {
return res.json(cached);
}
const games = await scraper.fetchNewAndPopular(parseInt(req.query.page) || 1);
cache.set(cacheKey, games);
res.json(games);
});
#### GAMES.ItchIO
The primary scraper class for itch.io game data extraction.
Constructor:
`typescript`
const scraper = new GAMES.ItchIO();
Methods:
| Method | Parameters | Returns | Description |
|--------|------------|---------|-------------|
| fetchNewAndPopular | page?: number | Promise | Retrieves games from itch.io's "New & Popular" section |fetchTopSellers
| | page?: number | Promise | Retrieves top-selling games with sales data |fetchTopRated
| | page?: number | Promise | Retrieves highest-rated games by user ratings |fetchNewest
| | page?: number | Promise | Retrieves recently published games |search
| | query: string, page?: number | Promise | Searches games by keyword with pagination |fetchGameInfo
| | gameUrl: string | Promise | Retrieves comprehensive game metadata |
#### IGameResult
Basic game information returned from list operations:
`typescript`
interface IGameResult {
id: string; // Unique game identifier
title: string; // Game title
url: string; // Direct link to game page
image?: string; // Thumbnail image URL
price?: IPrice; // Pricing information
isFree: boolean; // Free game indicator
platforms?: string[]; // Supported platforms
developer?: string; // Developer/publisher name
}
#### IGameInfo
Comprehensive game metadata from detailed scraping:
`typescript`
interface IGameInfo {
id: string;
title: string;
url: string;
image?: string;
cover?: string; // High-resolution cover image
description?: string; // Full game description
genres?: string[]; // Game categories/genres
tags?: string[]; // User-generated tags
price?: IPrice;
isFree: boolean;
isOnSale?: boolean; // Sale status indicator
originalPrice?: IPrice; // Pre-sale pricing
platforms?: string[]; // Platform compatibility
releaseDate?: string; // Publication date
rating?: number; // Average user rating
ratingCount?: number; // Number of ratings
developer?: string;
publisher?: string;
screenshots?: string[]; // Game screenshot URLs
videos?: string[]; // Trailer/video URLs
}
#### ISearch
Paginated search results container:
`typescript`
interface ISearch
currentPage?: number; // Current page number
hasNextPage?: boolean; // Next page availability
totalPages?: number; // Total page count (when available)
totalResults?: number; // Total result count (when available)
results: T[]; // Array of results
}
#### IPrice
Pricing information structure:
`typescript`
interface IPrice {
amount: number; // Numeric price value
currency: string; // Currency code (typically "USD")
formatted: string; // Display-formatted price string
}
Chimi Scraper employs a robust, extensible architecture designed for enterprise-grade applications:
- Provider Pattern: Platform-specific implementations (itch.io) are encapsulated as providers, enabling easy extension to additional gaming platforms
- Abstract Base Classes: Common scraping functionality is abstracted into base classes, promoting code reuse and consistency
- Type Safety: Comprehensive TypeScript definitions ensure compile-time error detection and enhanced developer experience
- Separation of Concerns: Clear separation between data models, HTTP utilities, parsing logic, and provider implementations
``
chimi-scraper/
├── src/
│ ├── models/ # Core type definitions and interfaces
│ │ ├── types.ts # Data structure definitions
│ │ ├── base-parser.ts # Abstract scraper foundation
│ │ └── game-parser.ts # Game-specific scraping contracts
│ ├── providers/ # Platform implementations
│ │ └── games/
│ │ └── itch.ts # itch.io scraping implementation
│ ├── utils/ # Shared utilities
│ │ ├── http.ts # HTTP client with retry logic
│ │ └── parser.ts # HTML parsing helpers
│ └── index.ts # Public API exports
├── examples/ # Implementation examples
├── dist/ # Compiled JavaScript output
└── README.md
`typescript
import { GAMES } from 'chimi-scraper';
const scraper = new GAMES.ItchIO();
async function safelyFetchGames(category: string, page: number = 1) {
try {
switch (category) {
case 'trending':
return await scraper.fetchNewAndPopular(page);
case 'top-sellers':
return await scraper.fetchTopSellers(page);
default:
throw new Error(Unknown category: ${category});Failed to fetch ${category} games:
}
} catch (error) {
console.error(, error);`
return { results: [], hasNextPage: false, currentPage: page };
}
}
For high-traffic applications, implement proper rate limiting and caching:
`typescript
import rateLimit from 'express-rate-limit';
// Rate limiting middleware
const limiter = rateLimit({
windowMs: 15 60 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per windowMs
message: 'Too many requests from this IP'
});
app.use('/api/', limiter);
`
Optimize API responses for bandwidth efficiency:
`typescript`
app.get('/api/games/trending', async (req, res) => {
const games = await scraper.fetchNewAndPopular(1);
// Return only essential fields for list views
const optimizedResults = games.results.map(game => ({
id: game.id,
title: game.title,
url: game.url,
image: game.image,
isFree: game.isFree,
platforms: game.platforms
}));
res.json({
...games,
results: optimizedResults
});
});
- Concurrent Requests: Implement request queuing for bulk operations
- Caching Strategy: Cache responses for frequently requested data
- Connection Pooling: Reuse HTTP connections for efficiency
- Error Recovery: Implement exponential backoff for failed requests
We welcome contributions from the developer community. Please follow these guidelines:
bash
git clone https://github.com/97x/chimi.git
cd chimi
npm install
npm run build
``MIT License - see the LICENSE file for details.
- Documentation: GitHub Repository
- Issues: Report bugs and request features
- itch.io Platform: Official itch.io website
---
Note: This library is designed for legitimate data access and should be used in compliance with itch.io's terms of service and robots.txt guidelines. Always implement appropriate rate limiting and respect the platform's resources.