How To Analyse Paragraphs vs Sentences

When working with free text (comments, reviews, etc.) we face a variety of text sizes; from a single word to a sentence, or to larger chunks such as paragraphs. Analysing all of these is possible under Relevance AI. However, there exist differences in the results.

Imagine the following free-text. The reviewer is clearly stating two action points.

I want to see better-designed programs. People need more practice time.

The text can be analysed as a whole or be broken into two sentences where each one gets analysed separately.

It is important to note, that under analysis like one-to-one clustering, one piece of text (independent of the number of composing sentences/topics) is assigned to only one category/cluster. This means performing a more fine-grained analysis requires breaking larger pieces of text into sentences or what we call sentence splitting at Relevance AI.

As a result of sentence splitting:

• Pieces of text will be broken into their composing sentences
• The dataset will grow in size
• Each sentence will be analysed independently of its surrounding sentences

Sentiment

Sentiment analysis is to identify the polarity of text (i.e. positive, negative or neutral). Under free-text data, imagine a paragraph of 5 sentences (e.g. two positives, two negatives and one neutral).

I like the neighbourhood. I am happy with the purchase. However, shops are not as advanced as I hoped for. And buses usually run late. That's the way it is I guess.Ana

Analysing this text as a whole will result in one output (positive, negative or neutral). It is decided by the AI model depending on which sentiment is the dominant one in the whole text. Similar to what was explained above for one-to-one clustering, to achieve a more fine-grained analysis (i.e. sentiment for each sentence) we recommend sentence splitting before running sentiment analysis, however, keep in mind that it results in a much larger dataset (i.e. more rows in your dataset).