Text summarizer using Node.js
npm install @trashhalo/node-summarizer
!npm
!NPM
!npm
node-summarizer is a Node.js module that summarizes text into a specified number of sentences. This module uses two
different extractive summarization techniques: frequency based and textrank based. It also provides sentiment analysis on the
given text, reduction percentage, as well as other information about the text and generated summary. Read the How it Works
section for more information about the two different approaches used.
1. Installation
2. Usage
* Initialization
* Getting a summary
* Getting a reduction percentage
* Sentiment Analysis
* Usage Notes
3. Method Details
4. How it Works
5. Dependencies
6. License
To install using NPM
``
npm i node-summarizer2. Usage
To get started, initialize a SummaryManager object.
`
let SummarizerManager = require("node-summarizer").SummarizerManager;
let Summarizer = new SummarizerManager(text_to_summarize,number_of_sentences);
`
Params
- text_to_summarize: is a String of the text you want summarized.
- number_of_sentences: is an Int of how many sentences you want in the summary.
There are two different approaches taken for summarization. Read the How it Works for more information on each of them.
To get a frequency summary:
`
let summary = Summarizer.getSummaryByFrequency().summary;
- The getSummaryByFrequency() method returns as object with summary as one of the properties. More information on it in the Method Details section.
To get a TextRank summary:
`
let summary = Summarizer.getSummaryByRank().then((summary_object)=>{
return summary_object.summary
})
- The getSummaryByRank() method returns a Promise. More information on it in the Method Details section.
To get the reduction percentage as a decimal:
`
//If you want the reduction percentage of a frequency summary.
let reduction_percentage = Summarizer.getFrequencyReductionAsDec().dec_reduction;
//If you want the reduction percentage of a TextRank summary. Returns a promise.
let reduction_percentage = Summarizer.getRankReductionAsDec().then((reduction_obj)=>{
return reduction_obj.dec_reduction;
})
`
To get the reduction percentage as a string:
`
//If you want the reduction percentage of a frequency summary.
let reduction_percentage = Summarizer.getFrequencyReduction().reduction;
//If you want the reduction percentage of a TextRank summary. Returns a promise.
let reduction_percentage = Summarizer.getRankReduction().then((reduction_obj)=>{
return reduction_obj.reduction;
})
`
To get the sentiment value of the string to be summarized:
`
let sentiment = Summarizer.getSentiment();
-This will return a Float.
- The easisest way to use this is to initialize the SummaryManager object => Get a summary using one or both of the summarization methods => Get reduction percentage based on the summarization method used.
- The alternative way is to simply initialize the SummaryManager object => call one of the reduction methods which automatically creates a summary if one doesn't exist and returns both the reduction percentage and new summary as an object. This is fine if all you need is the reduction percentage and summary.
- If a sentence cannot be split into sentences or if there are not enough sentences, the summary will be an Error().
More details on the available methods of the SummarizerManager class.
Once a SummarizerManager object has been initialized, calling this method will return an object with:
`
{
summary: "", //String of the summary
sentence_list: [], //List of all of the tokenized sentences in the given text
weighted_map: Map, //Map of all of the tokenized words with their frequencies.
sorted_sentences: [] //A list of all of the sentences sorted by weights.
}
This method returns a Promise. The result of the Promise is an object with:
`
{
summary: "", //String of the summary
sentence_list: [], //List of all of the tokenized sentences in the given text
nouns_and_adjactive_map: Map //Map of all of the sentences with the values being a list of nouns and adjactives in the sentence
}
Once a SummarizerObject has been initialized calling this method will return an object with:
`
{
reduction: "", //A String of the percentage of reduction ex. "50.1%"
summary: "" //Current frequency summary
}
-Calling this method without first calling the getSummaryByFrequency() method will still work. It will automatically create
a frequency summary.
This method returns a Promise. The result of the Promise is an object with:
`
{
reduction: "", //A String of the percentage of reduction ex. "50.1%"
summary: "" //Current frequency summary
}
-Calling this method without first calling the getSummaryByRank() method will still work. It will just automatically create
a rank summary.
This method works the same way as getFrequencyReduction and returns an object with:
`
{
dec_reduction: Float, //A float of the reduction ex. .50192
summary: "" //Current frequency summary
}
This method works the same way as getRankReductionAsDec and returns a Promise. The result of the Promise is an object with:
`
{
dec_reduction: "", //A String of the percentage of reduction ex. .50192
summary: "" //Current frequency summary
}
This method returns a Float of the sentiment value.
How the text is summarized.
This type of summary works best for text that is not too complicated. The advantage of this approach is that it more efficient than the textrank implementation. It was heavily inspired by this post.
- Split the given text into sentences.
- Preprocess the sentences by removing all punctuation and making all letters lowercase.
- Make a list of all the words that occur in the text and find the frequency of the words.
- Take the calculated frequencies of the words and calculate the total weight of the original sentences.
While this approach costs more in terms of time complexity, it is better for getting the summary of things like newspaper articles and essays. Read more about it here.
- Split the given text into sentences.
- Preprocess the sentences by removing all punctuation and making all letters lowercase.
- Make a map of all of the sentences with the key being the sentences themselves and the values being an array of the nouns and adjactives in the sentence.
ex. {"The Detroit Pistons are a good basketball team" => [detroit, pistons, good, basketball, team], "A basketball is round"=>[basketball, round]}
- Make a weighted graph with edges that connect sentences with matching nouns or adjectives. The weight of each edge is the number of matching nouns and adjactives. In the example above, there would be two nodes connected with an edge of 1 for "basketball"
- Choose a random starting point in the graph and "walk" through it many times using the weight of each edge as a probability of which next vertex to go to. For example, a node with two vertices with an edge of weight 4 and an edge of weight 3 would have a 4/7 chance to go to the first vertex and 3/7 chance to go to the other vertex.
- A count is kept of how many times a node is walked on.
- Sort the sentences.
This project is licensed under the terms of the ISC license.