Stock Crypto Sentiment Analyzer

Sentiment-regime overlay for algorithmic trading bots. This package ingests tariff-related tweets, policy news, and market data; classifies narratives; scores crash/surge probability; detects market regimes; and exposes guardrail middleware so technical-analysis (TA) strategies can derisk automatically.

Features

- Feed ingestion: Twitter/X filtered stream, NewsAPI-compatible poller, and market-data sampler auto-register via environment variables.
- Narrative classification: Keyword pre-filter + optional transformer (via @xenova/transformers) label events as threat, escalation, de-escalation, or noise.
- Narrative taxonomy & overrides: Six-category taxonomy assets (assets/taxonomy/**) with risk-profile overrides keep recall ≥90% per storyline while enabling ticker/sector-specific tuning.
- Ticker-aware sentiment: Entity extraction + rolling symbol store expose intraday crash/surge/confidence snapshots via overlay.getSymbolSentiment(symbol) and the yarn symbol:sentiment CLI.
- Catalyst ingestors: Earnings calendar, pre/post-market gapper feeds, and the macro calendar adapter normalize catalysts into sentiment events so guardrails can differentiate planned beats/misses from CPI/FOMC/jobs shocks.
- Transformer presets: Optional FinBERT/DeBERTa/FinGPT pipelines download into a managed cache (SENTIMENT_MODEL_PATH) and blend with the heuristic scorer when SENTIMENT_SCORING_ENABLE=true.
- Custom feed hooks: Template adapters (Alpaca/Polygon) plus logging guidance let you plug proprietary catalysts directly into the overlay (docs/feeds/custom-feed-hooks.md).
- Symbol metrics & alerts: /metrics exports per-symbol threat/escalation/de-escalation counters with Alertmanager/Grafana recipes (docs/alerts/symbol-metrics.md, docs/dashboards/symbol-heatmap.json).
- Sentiment scoring: Computes (crashProb, surgeProb, confidence), persists scores with TTL, and logs context for audits.
- Regime detection: Rolling volatility/liquidity metrics classify regimes (fragile, trend, range, calm).
- Guardrail engine: Middleware that modulates TA signals, blocks high-risk trades, caps leverage after de-escalations, and emits structured decision logs.
- Auditability & governance: Encrypted decision evidence vault with role-gated exports ensures every trade links back to raw sentiment events and executed orders.
- Tooling: Annotation CLI, backtest helper, feed connectivity scripts, and staging configuration guide.

Installation

``bash
yarn add @developerzava/stock-crypto-sentiment-analyzer

or

yarn add ../path/to/stock-crypto-sentiment-analyzer/tmp/analyzer.tgz
`

Environment Variables

Copy .env.example and fill in credentials:
`
SENTIMENT_TWITTER_BEARER=...
SENTIMENT_NEWS_API_KEY=...
SENTIMENT_NEWS_MAX_AGE_DAYS=2
SENTIMENT_OFFICIAL_SOCIAL_ENABLED=true
SENTIMENT_OFFICIAL_SOCIAL_REQUIRED=true
SENTIMENT_MARKET_DATA_URL=https://...
SENTIMENT_CLASSIFIER_ENABLE=true
SENTIMENT_CLASSIFIER_MODEL=Xenova/distilbert-base-uncased-finetuned-sst-2-english
SENTIMENT_SCORING_ENABLE=true
SENTIMENT_TRANSFORMER_PRESET=finbert
SENTIMENT_TRANSFORMER_CACHE=./data/models/transformers
SENTIMENT_SCORING_MODEL=Xenova/distilbert-base-uncased-finetuned-sst-2-english
SENTIMENT_ALERT_WEBHOOK=https://discord.com/api/webhooks/.../...
SENTIMENT_ALERT_MAX_PER_MINUTE=30
SENTIMENT_ALERT_HISTORICAL_LOOKBACK_MINUTES=60
SENTIMENT_ALERT_SUPPRESS_HISTORICAL_SLA=true
SENTIMENT_ALERT_MAX_RETRIES=3
SENTIMENT_REDIS_URL=redis://127.0.0.1:6379/0
SENTIMENT_REDIS_DB=0
SENTIMENT_REDIS_PREFIX=sentiment:
SENTIMENT_MODEL_PATH=./models/latest
SENTIMENT_GUARDRAIL_CONFIG=./config/guardrails.default.json
SENTIMENT_STRATEGY_ID=spy-0-dte
SENTIMENT_EVIDENCE_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
SENTIMENT_EVIDENCE_ROLE=compliance
SENTIMENT_EVIDENCE_PATH=./data/evidence/decisions
SENTIMENT_SYMBOL_SENTIMENT_PATH=./data/symbol-sentiment.json
SENTIMENT_EARNINGS_API_URL=https://api.example.com/earnings
SENTIMENT_EARNINGS_API_KEY=abc123
SENTIMENT_EARNINGS_POLL_INTERVAL_MS=300000
SENTIMENT_GAPPERS_API_URL=https://api.example.com/gappers
SENTIMENT_GAPPERS_API_KEY=abc123
SENTIMENT_GAPPERS_SESSION=pre
SENTIMENT_GAPPERS_POLL_INTERVAL_MS=60000
SENTIMENT_GAPPERS_MIN_GAP_PCT=3
SENTIMENT_MACRO_CALENDAR_URL=https://api.example.com/macro
SENTIMENT_MACRO_CALENDAR_API_KEY=abc123
SENTIMENT_MACRO_CALENDAR_POLL_INTERVAL_MS=300000
`
When running the overlay in Docker while Redis lives on the host, point SENTIMENT_REDIS_URL at redis://host.docker.internal:6379 (or any hostname reachable from both host and container) and pair it with SENTIMENT_REDIS_DB to pick a DB index that does not collide with other apps. Use SENTIMENT_REDIS_PREFIX to namespace keys (defaults to sentiment:). Leave SENTIMENT_GUARDRAIL_CONFIG unset unless you supply a real override file; the bundled default loads automatically.
See docs/staging-config.md for provisioning details and docs/integration.md for full variable descriptions.

SENTIMENT_SCORING_ENABLE=true enables the transformer blend; pick a preset with SENTIMENT_TRANSFORMER_PRESET (finbert, deberta-v3-large, or fingpt). Cache downloads under SENTIMENT_MODEL_PATH by default, or override with SENTIMENT_TRANSFORMER_CACHE if the bots mount a different volume.

Quick Start


`ts
import {
createSentimentOverlay,
DecisionAuditTrail,
GuardrailEngine,
SentimentGuardrail,
} from '@developerzava/stock-crypto-sentiment-analyzer';
import { createStructuredLogger } from '@developerzava/stock-crypto-sentiment-analyzer/logging';

const logger = createStructuredLogger({ serviceName: 'spy-0-dte' });
const overlay = await createSentimentOverlay({ env: process.env, logger });
const guardrails = new GuardrailEngine({ overlay, logger, earningsWindowMinutes: 120 });
const encryptionKey = process.env.SENTIMENT_EVIDENCE_ENCRYPTION_KEY;
if (!encryptionKey) throw new Error('SENTIMENT_EVIDENCE_ENCRYPTION_KEY must be set');
const auditTrail = new DecisionAuditTrail({
strategyId: process.env.SENTIMENT_STRATEGY_ID ?? 'spy-0-dte',
encryptionKey,
logger,
});

export async function executeTaSignal(signal: TradeSignal) {
const decision = await guardrails.evaluate(signal);
if (!decision.allowed) {
logger.warn({
category: LogCategory.DECISION,
action: 'trade-blocked',
message: 'Guardrail blocked TA signal',
details: { reason: decision.reason },
});
return;
}

const execution = await orderRouter.submit(decision.adjustedSignal);
await auditTrail.record({
decision,
requestedSignal: signal,
executedOrders: execution.orders, // array of { orderId, symbol, side, quantity, executedAt }
orderBatchId: execution.batchId,
});
}

const tslaSentiment = overlay.getSymbolSentiment('TSLA');
logger.info({
category: LogCategory.DECISION,
action: 'symbol-sentiment-snapshot',
message: 'Latest symbol sentiment snapshot',
details: { symbol: 'TSLA', sentiment: tslaSentiment },
});
`
The overlay automatically:
1. Registers feeds (Twitter, news, market data) if credentials exist.
2. Buffers events, classifies narratives, runs sentiment scoring, and emits structured events.
3. Updates overlay context (crash/surge probability, confidence, regime, recent events).
4. GuardrailEngine pulls that context for every TA decision.

$3


- The package ships both ESM and CJS bundles; asset resolution (FinBERT model config, taxonomy) uses Node's module resolver via require.resolve(), ensuring reliable path resolution in all installation scenarios (npm/yarn/pnpm, monorepos, Docker, etc.).
- Logs must include action, category, and message. The provided logger wrapper will map legacy msg to action/message, but downstream bots should emit structured events directly to avoid validator rejections.

CLI & Scripts

- yarn label:events — append labeled rows to data/annotations.csv.
- yarn evaluate:keywords — enforce ≥90% recall per taxonomy category via data/taxonomy-annotations.csv.
- yarn backtest:sentiment — run a simple crash-detection backtest using data/backtest/sample-events.json.
- yarn verify:feeds — ensure required environment variables are set.
- yarn ping:twitter|news|market — smoke test external APIs.
- yarn export:evidence --from=2025-03-01 --to=2025-03-07 --role=compliance — decrypts decision logs for investigations (requires SENTIMENT_EVIDENCE_ENCRYPTION_KEY + authorized role).
- yarn symbol:sentiment --symbol=TSLA — print the latest crash/surge/confidence snapshots for a ticker from the persisted store.

Development

`bash
yarn install
yarn lint
yarn test
yarn build
`
Publishing (dry-run):
`bash
npm publish --dry-run
`
prepublishOnly runs lint→test→build automatically.

Architecture Overview

`
Feeds (Twitter/News/Market) -> NarrativeClassifier -> SentimentScoring -> ScoreStore
\-> AlertDispatcher (threat webhooks)
Overlay Context (crashProb, surgeProb, confidence, regime, latest events)
^
|
RegimeDetector <- MarketDataPoller
|
GuardrailEngine + SentimentGuardrail -> Adjusted TA Signals -> Order Router
`

Testing

- Unit tests with Vitest live under tests/unit/**. Run yarn test for the full suite.
- Mock overlay (createMockOverlay) and GuardrailEngine tests ensure risk rules work without live feeds.

Documentation

- docs/integration.md — installation, env vars, guardrail usage, logging/metrics.
- docs/staging-config.md — credential provisioning checklist.
- internal/stories/user-stories.md (repo-only) — roadmap and status of major features.
- docs/data-lineage.md / docs/security/data-access.md` — evidence lineage diagram plus access-control policy for decision logs.

License

MIT © developerzava