NETANOS: Named entity-based Text Anonymization for Open Science
npm install netanos- go to the directory netanos/libs/stanford-ner/ and run the following command (in Terminal) after unzipping the downloaded Stanford NER file:
``bash`
$ java -mx1000m -cp "./stanford-ner.jar:lib/*" edu.stanford.nlp.ie.NERServer -loadClassifier classifiers/english.muc.7class.distsim.crf.ser.gz -port 8080 -outputFormat inlineXML
- once you started the Java-Server, keep it running and navigate to the directory ./netanos to run your anonymization script with node run.js
- after you've finished, you can end the server with
Furthermore, NETANOS relies on the following node.js-dependencies:
* ner (https://github.com/niksrc/ner)
* promise (https://github.com/then/promise)
You can use npm install or compile NETANOS from source. For both installation types, make sure you've got the Node.js dependencies installed:
- Install the Stanford NER Java distribution (see above for the exact command)
- Open terminal and install the dependencies.
`javascript`
npm install ner
npm install promise
#### 1. npm install
NETANOS can easily be installed via npm.
`
$ npm install netanos
The integration is illustrated below. The anonymization function takes the input string and a callback function as arguments and returns the anonymized string via the callback.
`javascript
var netanos = require("netanos"); //note that this is different from the filepath in the from-source installation
var input = "Max and Ben spent more than 1000 hours on writing the software. They started in August 2016 in Amsterdam.";
netanos.ner(input, function(output) {
console.log(output);
});
/*
"Barry and Rick spent more than 997 hours on writing the software. They started in January 14 2016 in Odessa."
*/
`
#### 2. Compile from source
Alternatively, the NETANOS source-code can be integrated manually with the Netanos.js file as user endpoint.
1. Access run.js to set the input of your string.
`javascript
var netanos = require("./Netanos.js");
var input = "Max and Ben spent more than 1000 hours on writing the software. They started in August 2016 in Amsterdam.";
netanos.ner(input, function(output) {
console.log(output);
});
/*
"Barry and Rick spent more than 997 hours on writing the software. They started in January 14 2016 in Odessa."
*/
`
2. In Terminal, run the run.js script:
`
node run.js
Once the Java server is running, you can test the functionality of NETANOS with npm test. To do this, make sure you've got the JavaScript test framework mocha.js installed (use npm install mocha).
The tests will run for all four core methods of NETANOS.
NETANOS offers the following functionality:
1. Context-preserving anonymization (netanos.anon): each identified entity is replaced with an indexed generic replacement of the entity type (e.g. Peter -> [PERSON_1], Chicago -> [LOCATION\_1]).
`javascript
var input = "Max and Ben spent more than 1000 hours on writing the software. They started in August 2016 in Amsterdam.";
netanos.anon(input, function(output) {
console.log(output);
});
/*
"[PERSON_1] and [PERSON_2] spent more than [DATE/TIME_1] on writing the software. They started in [DATE/TIME_2] in [LOCATION_1]."
*/
`
2. Named entity-based replacement (): each identified entity will be replaced with a different entity of the same type (e.g. Peter -> Alfred, Chicago -> London).
`javascript
var input = "Max and Ben spent more than 1000 hours on writing the software. They started in August 2016 in Amsterdam.";
netanos.ner(input, function(output) {
console.log(output);
});
/*
“Barry and Rick spent more than 997 hours on writing the software. They started in January 14 2016 in Odessa.”
*/
`
3. Non-context preserving anonymization (): this approach is not based on named entities and replaces every word starting with a capital letter and every numeric value with "XXX".
`javascript
var input = "Max and Ben spent more than 1000 hours on writing the software. They started in August 2016 in Amsterdam.";
/*
Note that the non-context preserving replacement is not asynchronous as it does not rely on the named entitiy recognition.
*/
var anonymized = netanos.noncontext(input);
console.log(anonymized);
/*
“XXX and XXX spent more than XXX hours on writing the software. XXX started in XXX XXX in XXX.”
*/
`
4. Combined, non-context preserving anonymization (): the non-context preserving replacement and the named entity-based replacement are combined such that each word starting with a capital letter, each numeric value and all identified named entities are replaced with "XXX".
`javascript
var input = "Max and Ben spent more than 1000 hours on writing the software. They started in August 2016 in Amsterdam.";
netanos.combined(input, function(output) {
console.log(output);
});
/*
“XXX and XXX spent more than XXX XXX on writing the software. XXX started in XXX XXX in XXX.”
*/
``
#### Contact
projectnetanos@gmail.com
MIT © Bennett Kleinberg & Maximilian Mozes