IoC extractor

!Node.js CI

IoC extractor is an npm package for extracting common IoC (Indicator of Compromise) from a block of text.

Note: the package is highly influenced by cacador.

Installation

``sh
npm install -g ioc-extractor


or if you want to use ioc-extractor as a library in your JS/TS project


npm install ioc-extractor
`

Usage

$3

`bash
$ ioc-extractor --help
Usage: ioc-extractor [options]

Options:
--no-strict Disable strict option
--no-refang Disable refang option
--no-sort Disable sort option
-p, --punycode Enable punycode option
-o, --only Show only specific IoC types
-h, --help display help for command
`

`bash
$ echo "1.1.1.1 8.8.8.8 example.com" | ioc-extractor | jq
{
"asns": [],
"btcs": [],
"cves": [],
"domains": [
"example.com"
],
"emails": [],
"eths": [],
"gaPubIDs": [],
"gaTrackIDs": [],
"ipv4s": [
"1.1.1.1",
"8.8.8.8"
],
"ipv6s": [],
"macAddresses": [],
"md5s": [],
"sha1s": [],
"sha256s": [],
"sha512s": [],
"ssdeeps": [],
"urls": [],
"xmrs": []
}

$ echo "1.1.1.1 8.8.8.8" | ioc-extractor --only ipv4s | jq
{
"ipv4s": [
"1.1.1.1",
"8.8.8.8"
]
}
`

$3

`ts
import { extractIOC } from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = extractIOC(input);
console.log(ioc.md5s);
// => ['f6f8179ac71eaabff12b8c024342109b']
console.log(ioc.ipv4s);
// => ['1.1.1.1']
console.log(ioc.domains);
// => ['google.com']
`

extractIOC takes the following options:

- punycode
- refang
- sort
- strict

If you want to extract a specific type of IoC, you can use an extract function by IoC type.

`ts
import {
refang,
extractDomains,
extractIPv4s,
extractMD5s,
} from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const refanged = refang(input);
// => 1.1.1.1 google.com f6f8179ac71eaabff12b8c024342109b

const ipv4s = extractIPv4s(refanged);
// => ['1.1.1.1']

const domains = extractDomains(refanged);
// => ['google.com']

const md5s = extractMD5s(refanged);
// => ['f6f8179ac71eaabff12b8c024342109b']
`

Network related extract functions (e.g. extractDomains) can take the following options:

- sort
- strict

See docs for more details.

Alternatively, if you want to extract a list of specific IoC types at once, you can use partialExtractIOC.

`ts
import { partialExtractIOC } from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = partialExtractIOC(input, ["ipv4s", "domains"]);
console.log(ioc);
// => {"ipv4s":["1.1.1.1"],"domains":["google.com"]}
`

IoC Types

This package supports the following IoCs:

- Hashes: MD5, SHA1, SHA256, SHA512, SSDEEP
- Networks: domain, email, IPv4, IPv6, URL, ASN
- Hardwares: MAC address
- Utilities: CVE (CVE ID)
- Cryptocurrencies: BTC (BTC address), ETH (ETH address), XMR (XMR address)
- Trackers: GA track ID (Google Analytics tracking ID), GA pub ID (Google Adsense Publisher ID)

Refang Techniques

For Networks IoCs, the following refang techniques are supported:

| Techniques | Defanged | Refanged |
| ------------------------------------ | -------------------------------------- | ------------------------------- |
| . in spaces | 1.1.1 . 1 | 1.1.1.1 |
| . in brackets, parentheses, etc. | 1.1.1[.]1 | 1.1.1.1 |
| dot in brackets, parentheses, etc. | example[dot]com | example.com |
| Back slash before . | example\.com | example.com |
| / in brackets, parentheses, etc. | http://example.com[/]path | http://example.com/path |
| :// in brackets, parentheses, etc. | http[://]example.com | http://example.com |
| : in brackets, parentheses, etc. | http[:]//example.com | http://example.com |
| @ in brackets, parentheses, etc. | test[@]example.com | test@example.com |
| at in brackets, parentheses, etc. | test[at]example.com | test@example.com |
| hxxp | hxxps://example.com | https://example.com |
| Partial | 1.1.1[.1 | 1.1.1.1 |
| Any combination | hxxps[:]//test\.example[.)com[/]path | https://test.example.com/path |

Options

$3

Whether to do strict TLD matching or not. Defaults to true.

$3

Whether to do refang or not. Defaults to false.

$3

Whether to do Punycode conversion or not. Defaults to false.

$3

Whether to sort values or not. Defaults to true`.

Alternatives

- cmu-sei/cyobstract
- fhightower/ioc-finder
- InQuest/python-iocextract
- sroberts/cacador