The goal of the text classification tasks is to assign labels or classes to text.

Applications of classifying text:

• sentiment analysis.
  
• entities extraction.
  
• ...
  

Text classification can be used with:

• representation models: task-sepecific models and embedding models
  
• generative models
  

There are two types of representation models for text classification:

• Task-specific models: they are trained for specific tasks (sentiment analysis, ...).
  
• Embedding models: they generate embeddings that can be used for text classification tasks, among other tasks.
  

These models are created by fine-tuning a base representation models (like BERT).

Example: task-specific model (sentiment analysis)

from transformers import AutoModelForSequenceClassification from transformers import TFAutoModelForSequenceClassification from transformers import AutoTokenizer, AutoConfig import numpy as np from scipy.special import softmax MODEL = f"cardiffnlp/twitter-roberta-base-sentiment-latest" model = AutoModelForSequenceClassification.from_pretrained(MODEL) tokenizer = AutoTokenizer.from_pretrained(MODEL) config = AutoConfig.from_pretrained(MODEL) encoded_input = tokenizer("The weather today is great!", return_tensors='pt') output = model(**encoded_input) scores = output[0][0].detach().numpy() scores = softmax(scores) ranking = np.argsort(scores) ranking = ranking[::-1] for i in range(scores.shape[0]): l = config.id2label[ranking[i]] s = scores[ranking[i]] print(f"{i+1}) {l} {np.round(float(s), 4)}")
Output:
1) positive 0.9899 2) neutral 0.0068 3) negative 0.0033
We can also perform text classification with generative models, such as OpenAI's GPT models:

from openai import OpenAI client = OpenAI( api_key="YOUR_API_KEY" ) prompt = """Predict whether the following sentence is a positive or negative statement: The weather today is great! If it is positive return 1 otherwise return 0. """ sentence = "The weather today is great!" messages=[ { "role": "user", "content": prompt } ] output = client.chat.completions.create( model="gpt-4o-mini", messages=messages, temperature=0 ) print(output.choices[0].message.content) # 1