Graphite Feeder

   


Introduction

Feed metrics from several sources to graphite.

Installation

No node package deployed to npm yet. Please see section about development to build app yourself.

Configuration

node-config is used for managing configurations.
>Configurations are stored in configuration files within your application, and can be overridden and extended by environment variables, command line parameters, or external sources.

Development

Setup docker container running graphite docker-graphite-statsd
``bash
sudo docker run -d \
--name graphite \
-p 80:80 \
-p 2003:2003 \
-p 8125:8125/udp \
hopsoft/graphite-statsd
`

If you are developing on MacOS you need to have boot2docker installed and running to be able to use docker.

To access the graphite web UI go to http://localhost/

On MacOS you need to use the proper IP of the boot2docker container (boot2docker ip) instead.

Clone repo and change to directory
`bash
git clone git@github.com:commercetools/graphite-feeder.git graphite-feeder && cd $_
`

Create a configuration file config/local.yaml and adjust missing values.
`yaml


configuration for graphite/carbon

graphite:
host: localhost # use boot2docker IP on MacOS
port: 2003 # carbon port

configuration for Salesforce feeder

salesforce:
username: # required when feeding from Salesforce
password: # required when feeding from Salesforce
token: # required when feeding from Salesforce
`

Resolve and install required dependencies.
`bash
npm install
`

Build and run salesforce feeder (using DEBUG level for logging)
`bash
grunt && DEBUG=* ./bin/feeder-salesforce -d
`

Release npm package using the predefined grunt task.
`bash
grunt release
`

Docker

There is also a docker container for easy setup/execution of the CLI.

Run docker container:
`bash
docker run -v /path/to/config/:/config -e NODE_CONFIG_DIR=/config/ sphereio/graphite-feeder
`

Set an alias for repeated calls:
`bash
alias feeder='docker run -v /path/to/config/:/config -e NODE_CONFIG_DIR=/config/ sphereio/graphite-feeder'
`

After setting the alias you can simply use:
`bash
feeder
`

Feeders

Feeders are used for quering data from different sources and feeding the data into graphite. The following section describes the available feeders.

$3

The Salesforce feeder queries data using Salesforce API and feeds the metric to graphite.

Sample configuration to push a metric to graphite. The configuration defines for each data you want to query a SOQL (Salesforce Object Query Language) as well as the mapping to resulting graphite metric.

`yaml


configuration for Salesforce feeder


salesforce:
prefix: salesforce # prefix for graphite path
datasources: # sources
revenue: # query invoiced opportunities
query: SELECT Account.name, Amount, CurrencyIsoCode,Type FROM Opportunity WHERE FiscalYear=2015 AND (StageName='Invoice' OR StageName='Receipt of payment')
mapping: # configure mapping salesforce result -> graphite metric
path: de.commercetools.opportunities.#{Account.Name}.#{Type}.amount.#{CurrencyIsoCode} # define path (field names from result allowed)
value: Amount # value field in result
timestamp: CloseDate # date field in result
`

You can define multiple data sources if you like to query and push different metrics from Salesforce to graphite.
`yaml


configuration for Salesforce feeder


salesforce:
prefix: salesforce # prefix for graphite path
datasources: #
: ...
: ...
``