<aside> <img src="/icons/hexagon-two-sixths_blue.svg" alt="/icons/hexagon-two-sixths_blue.svg" width="40px" /> Dataset Used: https://www.kaggle.com/datasets/danofer/sarcasm

</aside>

The dataset used is derived from Reddit comments which have the Sarcasm tag.

Since this is a Reddit Dataset, there are lots of things to consider. The columns include the Label (Sarcasm or not), Comment, Author, Score, Ups, Downs, Date, Created At and Parent_Comment.

Most of the Columns are not needed for the training so we only choose the Label, Comment and Parent Comment.

Why did we decide to include the Parent Comment ? As we know Sarcasm is pretty difficult to understand even by human standards. Since context is really important we decided to include the Parent Comment as it will greatly help us in understanding the Context of the Sarcasm itself.

<aside> <img src="/icons/book-closed_blue.svg" alt="/icons/book-closed_blue.svg" width="40px" /> Colab Notebook: https://colab.research.google.com/drive/1oBN5OWBBn51VmQN9tt_NOkOI6NoPH5zA?usp=sharing

</aside>

Steps necessary to employ are Preprocessing then EDA as the sentences have lots of stops words.

We need to remove the stop words first and lower all the words to make it even before doing EDA to get a clearer analysis on the data.

def remove_stopwords(text):
    text = text.lower()
    tokens = re.findall(r'\b\w+\b', text)
    final_text = [word for word in tokens if word not in stop]
    return " ".join(final_text)

stop = set(stopwords.words('english'))

df['comment'] = df['comment'].apply(remove_stopwords)
df['parent_comment'] = df['parent_comment'].apply(remove_stopwords)

After that we plot the graph for the Sentence Lengths of both the Comment and Parent Comment

df['sentence_length'] = df['comment'].apply(lambda x: len(x.split()))
df['sentence_length_parent'] = df['parent_comment'].apply(lambda x: len(x.split()))

bins = 50
length_range = (0, 50)

plt.figure(figsize=(24, 6))

plt.subplot(1, 2, 1)
plt.hist(df['sentence_length'], bins=bins, range=length_range, color='skyblue', edgecolor='black')
plt.title('Distribution of Comment Sentence Lengths')
plt.xlabel('Sentence Length (Number of Words)')
plt.ylabel('Frequency')

plt.subplot(1, 2, 2)
plt.hist(df['sentence_length_parent'], bins=bins, range=length_range, color='green', edgecolor='black')
plt.title('Distribution of Parent Comment Sentence Lengths')
plt.xlabel('Sentence Length (Number of Words)')
plt.ylabel('Frequency')

plt.show()

To save some time in the selection of the Data, we decided to only take the sentence with less than 50 words. After that we combine the sentences and filter again.

df = df[(df['sentence_length'] <= 50) & (df['sentence_length_parent'] <= 50)]

df['combined_comments'] = df['parent_comment'].astype(str) + ' // ' + df['comment'].astype(str)

df['len'] = df['combined_comments'].apply(lambda x: len(x))
df = df[df['len'] <= 200]

In between the parent comment and the comment itself, we gave it a separator in form of ‘ // ‘.

Lastly, we select randomly 10k data size for both the labels to make the training faster.