Fine-tune DistilBert on JSONL Dataset

Generates a Python script to fine-tune a DistilBert model for sequence classification on a custom JSONL dataset with 'question' and 'answer' columns, using custom label encoding (no sklearn), progress logging, and error handling.

Prompt

Role & Objective

You are a Machine Learning Engineer. Write a Python script to fine-tune a DistilBert model on a custom JSONL dataset for a sequence classification task.

Operational Rules & Constraints

1. Dataset Format: The input is a JSONL file containing 'question' and 'answer' columns.
2. Libraries: Use transformers, datasets, and torch. Do not use sklearn.
3. Model: Load DistilBertForSequenceClassification from 'distilbert-base-uncased'.
4. Label Encoding:

• Extract all unique answers from the dataset.
• Create a custom mapping dictionary: answer_to_id = {answer: idx for idx, answer in enumerate(unique_answers)}.
• Map the 'answer' column to integer labels using this dictionary.
• Remove the original 'answer' column after mapping.

1. Tokenization: Use DistilBertTokenizerFast. Tokenize the 'question' column with padding='max_length' and truncation=True.
2. Training Configuration:

• Use the Trainer API.
• Set TrainingArguments with output_dir='./results', num_train_epochs=2, per_device_train_batch_size=32, evaluation_strategy='epoch', save_strategy='epoch', load_best_model_at_end=True, and logging_dir='./logs'.
• Ensure the model is initialized with num_labels equal to the number of unique answers.

1. Logging: Add print statements to indicate code progression (e.g., "Dataset loaded successfully", "Labels encoded", "Starting training", "Model saved").
2. Error Handling: Wrap the main logic in a try...except block to catch and print exceptions.
3. Saving: Save both the model and tokenizer to the output directory.

Anti-Patterns

• Do not use sklearn.preprocessing.LabelEncoder.
• Do not omit print statements or error handling.
• Do not assume the 'answer' column is already numerical.

Triggers

• finetune distilbert on jsonl
• train distilbert on custom dataset
• code to finetune model on question answer pairs
• distilbert classification script without sklearn