Command-line interface for SkyFetch - Stream terminal logs and command outputs to your centralized logging platform
npm install @skyfetch/clibash
npm install -g @skyfetch/cli
`
$3
If you want to contribute or run the latest development version:
`bash
Clone the repository
git clone https://github.com/RajneeshOjha0/sky-fetch.git
cd sky-fetch
Install dependencies
npm install
Navigate to CLI package
cd packages/cli
npm install
Link globally for development
npm link
Verify installation
skyfetch --version
`
š Quick Start
$3
Before using SkyFetch CLI, you need:
1. API URL - Your SkyFetch server endpoint
2. API Key - Authentication key from your SkyFetch dashboard
#### Running SkyFetch Server Locally
If you want to run the SkyFetch server locally:
`bash
Clone the repository
git clone https://github.com/RajneeshOjha0/sky-fetch.git
cd sky-fetch
Install dependencies
npm install
Set up environment variables
Create .env file in the root directory with:
MONGODB_URI=your_mongodb_connection_string
JWT_SECRET=your_jwt_secret
SMTP_HOST=your_smtp_host (for email)
SMTP_USER=your_smtp_user
SMTP_PASS=your_smtp_password
Start the API server
npm run dev:api
In another terminal, start the web dashboard
npm run dev:web
`
The API will be available at http://localhost:3000
#### Create an Account and Get API Key
1. Open the web dashboard at http://localhost:5173 (or your deployed URL)
2. Sign up for an account
3. Verify your email with the OTP sent
4. Create an organization
5. Create a project within your organization
6. Go to the API Keys section
7. Generate a new API key for your project
8. Copy the API key (you'll only see it once!)
$3
Set up your API endpoint and authentication:
`bash
Set API endpoint
skyfetch config set apiUrl http://localhost:3000
Set your API key (get this from your SkyFetch dashboard)
skyfetch config set apiKey sk_your_api_key_here
`
For production/deployed server:
`bash
skyfetch config set apiUrl https://your-skyfetch-server.com
skyfetch config set apiKey sk_your_production_api_key
`
$3
#### Option A: Start Background Agent (Shell History)
Capture all your shell commands automatically:
`bash
skyfetch start
`
This will:
- ā
Detect your shell (bash/zsh/fish)
- ā
Find your shell history file
- ā
Start streaming new commands to SkyFetch
- ā
Run in the background
#### Option B: Run Specific Commands
Capture output from specific commands:
`bash
Run a command and capture its output
skyfetch run "npm run dev"
Run tests and capture results
skyfetch run "npm test"
Run with log filtering
skyfetch run "npm run dev" --exclude "nodemon|vite|debug"
`
$3
Open your SkyFetch dashboard to view, search, and analyze your logs:
- Local: http://localhost:5173
- Production: https://your-skyfetch-server.com
š Complete Usage Guide
$3
#### View All Configuration
`bash
skyfetch config list
`
Output:
`
Current Configuration:
apiUrl: http://localhost:3000
apiKey: sk_**
bufferSize: 100
flushInterval: 10000
`
#### Set Configuration Values
`bash
Set API URL
skyfetch config set apiUrl http://localhost:3000
Set API Key
skyfetch config set apiKey sk_your_api_key_here
Set global log filter
skyfetch config set exclude "nodemon|vite|webpack"
Set buffer size (number of logs to buffer)
skyfetch config set bufferSize 100
Set flush interval (milliseconds)
skyfetch config set flushInterval 10000
`
#### Get Specific Configuration
`bash
skyfetch config get apiUrl
skyfetch config get apiKey
`
#### Delete Configuration
`bash
skyfetch config delete exclude
`
$3
The background agent monitors your shell history and streams commands to SkyFetch.
`bash
Start with auto-detected shell
skyfetch start
Start with specific shell
skyfetch start --shell zsh
skyfetch start --shell bash
skyfetch start --shell fish
Start with custom history file
skyfetch start --history-file ~/.custom_history
`
What happens when you start:
1. CLI detects your shell type
2. Locates your shell history file
3. Starts watching for new commands
4. Buffers logs locally
5. Sends batches to your SkyFetch server
6. Retries on failure
$3
Capture stdout and stderr from any command:
`bash
Basic usage
skyfetch run "npm test"
skyfetch run "python script.py"
skyfetch run "docker-compose up"
With log filtering
skyfetch run "npm run dev" --exclude "nodemon|vite|debug"
Long-running processes
skyfetch run "npm start"
`
The --exclude flag:
- Accepts regex patterns
- Filters out matching log lines
- Reduces noise in your logs
Examples:
`bash
Exclude build tool output
skyfetch run "npm run build" --exclude "webpack|babel"
Exclude debug logs
skyfetch run "node app.js" --exclude "debug|trace"
Exclude HTTP request logs
skyfetch run "npm start" --exclude "GET|POST|PUT|DELETE"
`
$3
Set a filter once, use it everywhere:
`bash
Set global exclusion pattern
skyfetch config set exclude "nodemon|vite|webpack|debug"
Now all commands use this filter automatically
skyfetch run "npm run dev"
skyfetch run "npm test"
`
Common filter patterns:
- nodemon|vite - Exclude build tool output
- debug|trace - Exclude debug logs
- webpack|babel - Exclude bundler output
- GET|POST - Exclude HTTP logs
- \[HMR\] - Exclude hot module replacement logs
$3
Send system metrics to SkyFetch:
`bash
This is typically called automatically by the CLI
But you can also run it manually for testing
node src/scripts/audit.js
`
This sends:
- CPU usage percentage
- Memory usage percentage
- Disk usage percentage
- Timestamp
š§ Advanced Configuration
$3
Configuration is stored in: ~/.skyfetch/config.json
Example configuration file:
`json
{
"apiUrl": "http://localhost:3000",
"apiKey": "sk_your_api_key_here",
"exclude": "nodemon|vite|debug",
"bufferSize": 100,
"flushInterval": 10000
}
`
$3
Logs are buffered locally in: ~/.skyfetch/buffer.log
This file stores logs when:
- You're offline
- The API is unreachable
- There's a network error
Logs are automatically synced when connection is restored.
$3
You can also configure using environment variables:
`bash
Set environment variables
export SKYFETCH_API_URL=http://localhost:3000
export SKYFETCH_API_KEY=sk_your_api_key_here
Run CLI (will use environment variables)
skyfetch start
`
Priority order:
1. Command-line arguments
2. Environment variables
3. Config file (~/.skyfetch/config.json)
$3
For automatic startup, add to your shell profile:
Bash (~/.bashrc or ~/.bash_profile):
`bash
Auto-start SkyFetch agent
if command -v skyfetch &> /dev/null; then
skyfetch start &
fi
`
Zsh (~/.zshrc):
`bash
Auto-start SkyFetch agent
if (( $+commands[skyfetch] )); then
skyfetch start &
fi
`
Fish (~/.config/fish/config.fish):
`fish
Auto-start SkyFetch agent
if type -q skyfetch
skyfetch start &
end
`
šļø Architecture
$3
`
SkyFetch CLI
āāā Config Manager ā Handles configuration storage
āāā History Watcher ā Monitors shell history files
āāā Command Runner ā Executes commands and captures output
āāā Log Buffer ā Buffers logs locally for batch sending
āāā API Client ā Communicates with SkyFetch API
āāā Retry Logic ā Ensures reliable log delivery
āāā Monitoring Service ā Tracks system metrics
`
$3
`
Shell Command
ā
History File
ā
Watcher Service ā Detects new commands
ā
Log Buffer ā Stores locally
ā
API Client ā Sends batch to server
ā
SkyFetch API ā Stores in MongoDB
ā
Web Dashboard ā View and search logs
`
š Security
- API Keys: Stored securely in ~/.skyfetch/config.json with restricted permissions
- HTTPS: All communication with the API uses HTTPS (in production)
- Local Buffering: Logs buffered locally if API is unreachable
- No Sensitive Data: Passwords and secrets are not logged by default
- Tenant Isolation: Logs are isolated by organization and project
š Troubleshooting
$3
Problem: skyfetch: command not found
Solution:
`bash
Check if npm global bin is in PATH
npm config get prefix
Add to PATH (Linux/Mac)
export PATH="$(npm config get prefix)/bin:$PATH"
Add to PATH (Windows)
Add %APPDATA%\npm to your PATH environment variable
Or reinstall globally
npm install -g @skyfetch/cli
`
$3
Problem: Logs sent but not visible in dashboard
Solution:
1. Check your API configuration:
`bash
skyfetch config list
`
2. Verify API key is valid:
- Go to your SkyFetch dashboard
- Check API Keys section
- Ensure the key is active and not revoked
3. Check network connectivity:
`bash
curl http://localhost:3000/health
`
4. View local buffer to see if logs are being captured:
`bash
cat ~/.skyfetch/buffer.log
`
5. Check API server logs for errors
$3
Problem: Permission denied errors
Solution:
`bash
Fix permissions on config directory (Linux/Mac)
chmod 755 ~/.skyfetch
chmod 644 ~/.skyfetch/config.json
On Windows, run as administrator if needed
`
$3
Problem: start command doesn't capture new commands
Solution:
1. Verify your shell:
`bash
echo $SHELL
`
2. Check history file location:
`bash
# Bash
echo $HISTFILE
# Zsh
echo $HISTFILE
`
3. Manually specify history file:
`bash
skyfetch start --history-file ~/.bash_history
`
4. Ensure history is being written:
`bash
# Bash - add to ~/.bashrc
shopt -s histappend
PROMPT_COMMAND="history -a; $PROMPT_COMMAND"
# Zsh - add to ~/.zshrc
setopt INC_APPEND_HISTORY
`
$3
Problem: ECONNREFUSED or connection timeout
Solution:
1. Check if API server is running:
`bash
curl http://localhost:3000/health
`
2. Verify API URL is correct:
`bash
skyfetch config get apiUrl
`
3. Check firewall settings
4. For local development, ensure MongoDB is running
$3
Problem: CLI using too many resources
Solution:
1. Increase flush interval (reduce frequency):
`bash
skyfetch config set flushInterval 30000 # 30 seconds
`
2. Increase buffer size (send larger batches):
`bash
skyfetch config set bufferSize 200
`
3. Add more aggressive filtering:
`bash
skyfetch config set exclude "debug|trace|verbose|nodemon|vite"
`
š” Examples & Use Cases
$3
`bash
Start dev server with log capture, excluding build tool noise
skyfetch run "npm run dev" --exclude "webpack|hot-update|vite"
`
$3
`bash
Run tests and stream results to SkyFetch
skyfetch run "npm test"
Run tests with coverage
skyfetch run "npm run test:coverage"
`
$3
`bash
Capture deployment logs
skyfetch run "npm run deploy"
Monitor production app
skyfetch run "pm2 start app.js"
`
$3
`bash
Run with verbose logging
skyfetch run "node --trace-warnings app.js"
Then search in dashboard for specific errors
`
$3
GitHub Actions:
`yaml
name: Test with SkyFetch
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
- name: Install SkyFetch CLI
run: npm install -g @skyfetch/cli
- name: Configure SkyFetch
run: |
skyfetch config set apiUrl ${{ secrets.SKYFETCH_API_URL }}
skyfetch config set apiKey ${{ secrets.SKYFETCH_API_KEY }}
- name: Run tests with logging
run: skyfetch run "npm test"
`
GitLab CI:
`yaml
test:
script:
- npm install -g @skyfetch/cli
- skyfetch config set apiUrl $SKYFETCH_API_URL
- skyfetch config set apiKey $SKYFETCH_API_KEY
- skyfetch run "npm test"
`
$3
`bash
Monitor Docker container logs
skyfetch run "docker-compose up"
Stream specific container logs
skyfetch run "docker logs -f my-container"
`
š Related Projects
- SkyFetch API - Backend server for log aggregation
- SkyFetch Web - Web dashboard for viewing and searching logs
- SkyFetch Shared - Shared types and utilities
š API Reference
$3
| Command | Description |
|---------|-------------|
| skyfetch config list | View all configuration |
| skyfetch config set | Set configuration value |
| skyfetch config get | Get configuration value |
| skyfetch config delete | Delete configuration value |
| skyfetch start [options] | Start background agent |
| skyfetch run | Run command with capture |
| skyfetch --version | Show version |
| skyfetch --help | Show help |
$3
| Key | Description | Default |
|-----|-------------|---------|
| apiUrl | SkyFetch API endpoint | - |
| apiKey | Authentication API key | - |
| exclude | Global regex filter pattern | - |
| bufferSize | Number of logs to buffer | 100 |
| flushInterval | Flush interval (ms) | 10000 |
$3
skyfetch start options:
- --shell - Specify shell type (bash/zsh/fish)
- --history-file - Custom history file path
skyfetch run options:
- --exclude - Regex pattern to exclude logs
š¤ Contributing
Contributions are welcome! Here's how to contribute:
1. Fork the repository
`bash
# Fork on GitHub, then clone your fork
git clone https://github.com/YOUR_USERNAME/sky-fetch.git
cd sky-fetch
`
2. Create a feature branch
`bash
git checkout -b feature/amazing-feature
`
3. Make your changes
`bash
cd packages/cli
# Make your changes
`
4. Test your changes
`bash
npm link
skyfetch --version
# Test your feature
`
5. Commit and push
`bash
git add .
git commit -m "Add amazing feature"
git push origin feature/amazing-feature
`
6. Create a Pull Request
- Go to GitHub
- Create a PR from your branch to main
š License
MIT Ā© SkyFetch Team
See LICENSE file for details.
š Links
- GitHub Repository: https://github.com/RajneeshOjha0/sky-fetch
- npm Package: https://www.npmjs.com/package/@skyfetch/cli
- Issues: https://github.com/RajneeshOjha0/sky-fetch/issues
- Documentation: https://github.com/RajneeshOjha0/sky-fetch#readme
š¬ Support
- Issues: GitHub Issues
- Discussions: GitHub Discussions
š Acknowledgments
Built with:
- Commander.js - CLI framework
- Chalk - Terminal styling
- Chokidar - File watching
- Axios - HTTP client
- Inquirer - Interactive prompts
---
Made with ā¤ļø by the SkyFetch Team
Ready to get started? Install now:
`bash
npm install -g @skyfetch/cli
``