Text classification is a fundamental natural language processing task that assigns predefined labels or categories to text documents. This supervised learning technique enables machines to automatically categorize text based on its content.

Modern text classification leverages two primary approaches with Large Language Models:

• Representation Models
  These models convert text into numerical representations (embeddings) that capture semantic meaning:
  
  • Task-Specific Models: Fine-tuned for particular classification tasks (e.g., sentiment analysis, spam detection). These models are trained on domain-specific datasets and optimized for specific use cases.
    
  • Embedding Models: Generate general-purpose text embeddings that can be used with traditional machine learning classifiers or similarity-based approaches. Examples include sentence-transformers, OpenAI's text-embedding models, and Google's Universal Sentence Encoder.
    
  
  Both types typically start with pre-trained transformer models like BERT, RoBERTa, DeBERTa, or DistilBERT, which are then fine-tuned on task-specific datasets.
  
  
• Generative Models
  Large language models like GPT-4, Claude, Gemini, or LLaMA can perform text classification through:
  
  • Zero-shot Classification: Classifying text without task-specific training using natural language instructions.
    
  • Few-shot Learning: Providing a few examples to guide the model's classification behavior.
    
  • Prompt Engineering: Crafting effective prompts to elicit accurate classifications.
    

Advantages of Task-Specific Models

• High Accuracy: Optimized for specific tasks with domain-relevant training data.
  
• Fast Inference: Efficient processing with smaller model sizes and lower computational requirements.
  
• Consistent Performance: Reliable results for the trained task with predictable behavior.
  

Advantages of Generative Models

• Flexibility: Handle diverse classification tasks without retraining.
  
• Zero-shot Capability: Classify into new categories without examples.
  
• Reasoning: Provide explanations for classifications and handle complex reasoning tasks.
  

Common Applications:

• Sentiment Analysis: Determining emotional tone (positive, negative, neutral) in reviews, social media posts, or customer feedback.
  
• Topic Classification: Categorizing documents by subject matter (sports, politics, technology, etc.).
  
• Spam Detection: Filtering unwanted emails or messages.
  
• Content Moderation: Identifying inappropriate or harmful content.
  
• Document Classification: Organizing legal documents, research papers, or business reports.
  
• Language Detection: Identifying the language of a given text.
  
• Intent Classification: Understanding user intentions in chatbots and virtual assistants.
  

Task-specific models offer high accuracy and efficiency for well-defined classification tasks. Here's an example using a fine-tuned RoBERTa model for sentiment analysis.

Python code:

$ vi representation-sentiment.py

from transformers import AutoTokenizer, AutoConfig, AutoModelForSequenceClassification
import numpy as np
from scipy.special import softmax
import torch

MODEL = "cardiffnlp/twitter-roberta-base-sentiment-latest"

try:
  # load the pre-trained sentiment analysis model, tokenizer, and configuration
  model = AutoModelForSequenceClassification.from_pretrained(MODEL)
  tokenizer = AutoTokenizer.from_pretrained(MODEL)
  config = AutoConfig.from_pretrained(MODEL)

  # Tokenize input text
  encoded_input = tokenizer("The weather today is great!", return_tensors='pt', truncation=True, padding=True)

  # analyze sentiment of input text and return predictions
  with torch.no_grad(): # disable gradient computation for inference
    output = model(**encoded_input)

  # extract and normalize scores using softmax
  scores = output.logits[0].detach().numpy()
  scores = softmax(scores)

  # rank predictions by confidence (highest to lowest)
  ranking = np.argsort(scores)[::-1]

  for i in range(len(scores)):
    label = config.id2label[ranking[i]]
    score = scores[ranking[i]]
    print(f"{i+1}) {label.capitalize()}: {np.round(float(score), 4)}")

except Exception as e:
  print(f"Error: {e}")

Run the Python script:

$ python3 representation-sentiment.py

Output:

1) Positive: 0.9899
2) Neutral: 0.0068
3) Negative: 0.0033

Generative models provide flexibility and can handle diverse classification tasks without task-specific training. Here's an implementation using OpenAI's GPT models.

Python code:

$ vi generative-sentiment.py

import openai

openai.api_key = "YOUR_API_KEY"

prompt = """Can you tell if the following sentence is a positive, negative, or neutral statement:

The weather today is great!

If it is positive return Positive. If it is negative return Negative. Otherwise return Neutral.
Also return the confidence score of your prediction.
"""

messages=[
    { "role": "user", "content": prompt}
]

output = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    temperature=0
)

print(output.choices[0].message.content)

Run the Python script:

$ python3 generative-sentiment.py

Output:

The sentence "The weather today is great!" is a positive statement. 

Prediction: Positive  
Confidence Score: 0.95