module great_dataset

---

class GReaTDataset

GReaT Dataset

The GReaTDataset overwrites the _getitem function of the HuggingFace Dataset Class to include the permutation step.

Attributes:

• tokenizer (AutoTokenizer): Tokenizer from HuggingFace

---

method GReaTDataset.set_tokenizer

set_tokenizer(tokenizer)

Set the Tokenizer

Args:

• tokenizer: Tokenizer from HuggingFace

---

class GReaTDataCollator

GReaT Data Collator

Overwrites the DataCollatorWithPadding to also pad the labels and not only the input_ids

---

This file was automatically generated via lazydocs.