Name: Anand Surendra Masurkar¶
Predicting Suicidal Ideation Using Machine learning¶
RESEARCH QUESTION¶
The aim of this project is to perform data analysis and predict suicidal ideation using Machine learning models. It focuses on developing models that can classify unseen records either having suicidal ideation or not (labeled as yes or no respectively). Dataset used for this study is from kaggle website.¶
Importing Necessary Libraries (Required)¶
In [0]:
import pandas as pd
import csv
import json
from matplotlib import pyplot as plt
import numpy as np
import plotly.graph_objs as go
import plotly as plty
from sklearn import preprocessing
import sklearn.feature_extraction.text as tfidf
from sklearn.model_selection import train_test_split
from sklearn import datasets,linear_model, preprocessing,utils
from sklearn.metrics import mean_squared_error,r2_score
from sklearn import metrics
from sklearn.metrics import roc_curve, auc
from sklearn.metrics import confusion_matrix, classification_report
from sklearn.utils.multiclass import unique_labels
import itertools
from sklearn import svm
import collections
from sklearn.naive_bayes import MultinomialNB
Function to Plot ROC curve¶
In [0]:
# Plot an ROC. pred - the predictions, y - the expected output.
def plot_roc(pred,y):
fpr, tpr, thresholds = roc_curve(y, pred)
roc_auc = auc(fpr, tpr)
plt.figure()
plt.plot(fpr, tpr, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], 'k--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic (ROC)')
plt.legend(loc="lower right")
plt.show()
Exploratory Data Analysis¶
In [0]:
from google.colab import drive
drive.mount('/content/drive')
Read CSV file from drive¶
In [0]:
df = pd.read_csv("drive/My Drive/foreveralone.csv")
Print head of the dataframe obtained by reading csv using pandas library¶
In [0]:
df.head()
Out[0]:
In [0]:
df.info()
In [0]:
df.describe()
Out[0]:
Check for any duplicates present in the dataFrame¶
In [0]:
df.duplicated().any()
Out[0]:
False indicates no duplicate records present¶
Check for any null values in the dataFrame¶
In [0]:
df.isna().any()
Out[0]:
Count of each distinct job_title¶
In [0]:
df.job_title.value_counts()
Out[0]:
Check info about type of each feature¶
In [0]:
df.info()
We observe that datatype of friends is in float. number of friends cannot be in float. Therefore converting it to int data type is necessary.¶
In [0]:
# change dataype to int
df['friends'] = df['friends'].astype(np.int64)
Check again if the modification done on friends column is successfully done¶
In [0]:
df.info()
Droping rows which consists of null values¶
In [0]:
df.dropna(inplace=True)
Conforming again if any null values are present¶
In [0]:
df.isna().any()
Out[0]:
In [0]:
# strip stings with white space
df['job_title'] = df.job_title.str.strip()
Function to replace job_title values to make them consistent and standardizing values for example student as Student for every record. Othervise two unique values will be used like student and Student which are same but model will consider them as two distinct values.¶
In [0]:
# Function to replace job_title values
def replace_text(what, to):
df.replace(what, to, inplace= True)
Call replace_text function tostandardize values across multiple columns¶
In [0]:
replace_text('student', 'Student')
replace_text('none', 'None')
replace_text("N/a", 'None')
replace_text('na', 'None')
replace_text('-', 'None')
replace_text('.', 'None')
replace_text('*', 'None')
replace_text('ggg', 'None')
In [0]:
df.job_title.value_counts()
Out[0]:
Statistics of Gender column¶
In [0]:
df.gender.value_counts()
Out[0]:
Visualizing Gender Frequency¶
In [0]:
import matplotlib.pyplot as plt; plt.rcdefaults()
import numpy as np
import matplotlib.pyplot as plt
objects = ('Male', 'Female', 'Transgender Male', 'Transgender Female')
y_pos = np.arange(len(objects))
performance = df.gender.value_counts()
plt.bar(y_pos, performance, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.ylabel('Count')
plt.title('Gender Frequency')
plt.show()
Statistics of sexuality column¶
In [0]:
# sexuality freqency
df.sexuallity.value_counts()
Out[0]:
Visualizing satistics of column sexuality¶
In [0]:
objects = ('Straight', 'Bisexual','Gay/Lesbian')
y_pos = np.arange(len(objects))
performance = df.sexuallity.value_counts()
plt.bar(y_pos, performance, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.ylabel('Count')
plt.title('Sexuality Frequency')
plt.show()
BodyWeight Column Statistics¶
In [0]:
# body weight
df.bodyweight.value_counts()
Out[0]:
Visualizing the Bodyweigt column frequency¶
In [0]:
objects = ('Normal weight', 'Overweight','Underweight','Obese')
y_pos = np.arange(len(objects))
performance = df.bodyweight.value_counts()
plt.bar(y_pos, performance, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.ylabel('Count')
plt.title('bodyweight Frequency')
plt.show()
In [0]:
# depressed
df.depressed.value_counts()
Out[0]:
In [0]:
objects = ('Yes', 'No')
y_pos = np.arange(len(objects))
performance = df.depressed.value_counts()
plt.bar(y_pos, performance, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.ylabel('Count')
plt.title('depressed Frequency')
plt.show()
In [0]:
df.social_fear.value_counts()
Out[0]:
In [0]:
objects = ('Yes', 'No')
y_pos = np.arange(len(objects))
performance = df.depressed.value_counts()
plt.bar(y_pos, performance, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.ylabel('Count')
plt.title('social_fear Frequency')
plt.show()
Attempt Suicide Frequency statistics¶
In [0]:
df.attempt_suicide.value_counts()
Out[0]:
Attempt_suicide column frequency statistics¶
In [0]:
objects = ('No', 'Yes')
y_pos = np.arange(len(objects))
performance = df.attempt_suicide.value_counts()
plt.bar(y_pos, performance, align='center', alpha=0.5)
plt.xticks(y_pos, objects)
plt.ylabel('Count')
plt.title('attempt_suicide Frequency')
plt.show()
Visualizing Histogram of various columns in data frame¶
In [0]:
df['age'].hist()
Out[0]:
In [0]:
df['friends'].hist()
Out[0]:
In [0]:
male = df[df['gender'] == 'Male' ]
female = df[df['gender'] == 'Female' ]
In [0]:
df.head()
Out[0]:
Checking datatypes for each column feature¶
In [0]:
df.dtypes
Out[0]:
Determining unique entries in features of dataframe¶
This will allow us to conform no discrepancy among records is present¶
In [0]:
df['sexuallity'].unique()
Out[0]:
In [0]:
df['gender'].unique()
Out[0]:
In [0]:
df['race'].unique()
Out[0]:
In [0]:
df['bodyweight'].unique()
Out[0]:
In [0]:
df['virgin'].unique()
Out[0]:
In [0]:
df['income'].unique()
Out[0]:
In [0]:
df['prostitution_legal'].unique()
Out[0]:
In [0]:
df['pay_for_sex'].unique()
Out[0]:
In [0]:
df['depressed'].unique()
Out[0]:
In [0]:
df['employment'].unique()
Out[0]:
In [0]:
df['edu_level'].unique()
Out[0]:
In [0]:
df['job_title'].unique()
Out[0]:
In [0]:
df_new = df
In [0]:
df_new.head()
Out[0]:
In [0]:
df['what_help_from_others'].unique()
Out[0]:
In [0]:
df_new = df_new.drop(columns=['time','what_help_from_others'])
In [0]:
df_new = df_new.drop(df_new.index[445])
In [0]:
df_new.head()
Out[0]:
In [0]:
df_new.info()
In [0]:
location = df_new.loc[df_new['race'] == 'First two answers. Gender is androgyne, not male; sexuality is asexual, not bi.']
In [0]:
df_new = df_new.drop(location.index)
In [0]:
df_new.loc[df_new['race'] == 'First two answers. Gender is androgyne, not male; sexuality is asexual, not bi.']
Out[0]:
In [0]:
df_new = df_new.drop(columns=['improve_yourself_how'])
In [0]:
df_for_sampling = df_new.copy(deep=True) #Important
In [0]:
df_new.head()
Out[0]:
Function to Label encode Y (Output)¶
In [0]:
# Label Encoding Only for Y (output)
def encode_text_index(df_new, name):
le = preprocessing.LabelEncoder()
df_new[name] = le.fit_transform(df_new[name])
return le.classes_
Encoding attempt_suicide column.¶
In [0]:
my_list = ['attempt_suicide']
for i in my_list:
encode_text_index(df_new,i)
Function to one hot encode categorical values in the dataframe¶
In [0]:
# Encode text values to dummy variables(i.e. [1,0,0],[0,1,0],[0,0,1] for red,green,blue)
# One Hot encoding
def encode_text_dummy(df_new, name):
dummies = pd.get_dummies(df_new[name])
for x in dummies.columns:
dummy_name = "{}-{}".format(name, x)
df_new[dummy_name] = dummies[x]
df_new.drop(name, axis=1, inplace=True)
In [0]:
mylist_2 = ["gender","age","sexuallity","income","race","bodyweight","virgin","prostitution_legal","pay_for_sex","social_fear","depressed","employment","job_title","edu_level"]
for t in mylist_2:
print(t)
encode_text_dummy(df_new,t)
In [0]:
df_new.head()
Out[0]:
Creating Numpy arrays for both x and y¶
In [0]:
Y_numpy_array = df_new['attempt_suicide'].values # Y is ready for model train test split
In [0]:
df_for_x = df_new.drop(columns=['attempt_suicide'])
In [0]:
X_numpy_array = df_for_x.values # X is ready for model train test split
In [0]:
Y_numpy_array[:2]
Out[0]:
Train test split data into 70% training and 30% testing¶
In [0]:
x_train,x_test,y_train,y_test = train_test_split(X_numpy_array,Y_numpy_array, test_size=0.3,random_state=42)
Determining shape of x_train, y_train, x_test,y_test¶
In [0]:
type(y_train)
print(utils.multiclass.type_of_target(y_train))
In [0]:
x_train.shape
Out[0]:
In [0]:
x_test.shape
Out[0]:
In [0]:
y_train.shape
Out[0]:
In [0]:
y_test.shape
Out[0]:
Logistic Regression¶
In [0]:
logreg = linear_model.LogisticRegression()
In [0]:
logreg.fit(x_train, y_train)
Out[0]:
In [0]:
y_pred_logistic = logreg.predict(x_test)
print(y_pred_logistic[:5])
print(y_test[:5])
In [0]:
# The coefficients
#print('Coefficients: \n', logreg.coef_)
# The mean squared error
print("Mean squared error: %.2f"
% mean_squared_error(y_test, y_pred_logistic))
In [0]:
score = metrics.accuracy_score(y_test, y_pred_logistic)
print("Accuracy score: {}".format(score))
In [0]:
score_tf_precision_stopping = metrics.precision_score(y_test, y_pred_logistic, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(y_test, y_pred_logistic, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(y_test, y_pred_logistic, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
len(y_test)
Out[0]:
In [0]:
len(y_pred_logistic)
Out[0]:
In [0]:
y_test[:5]
Out[0]:
In [0]:
y_pred_logistic[:5]
Out[0]:
In [0]:
y_test.shape
Out[0]:
In [0]:
y_pred_logistic.shape
Out[0]:
In [0]:
cf =confusion_matrix(y_test,y_pred_logistic)
In [0]:
plt.imshow(cf,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(y_test))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf.max() / 2.
for i,j in itertools.product(range(cf.shape[0]),range(cf.shape[1])):
plt.text(j,i,format(cf[i,j],'d'),horizontalalignment='center',color='white' if cf[i,j] >thresh else 'black')
plt.show();
In [0]:
df['attempt_suicide'].unique()
Out[0]:
In [0]:
type(y_train)
Out[0]:
In [0]:
print(utils.multiclass.type_of_target(y_train))
In [0]:
type(y_pred_logistic)
Out[0]:
In [0]:
print(utils.multiclass.type_of_target(y_pred_logistic))
SVM¶
In [0]:
SVM_classifier = svm.SVC()
In [0]:
SVM_classifier.fit(x_train, y_train)
Out[0]:
In [0]:
y_pred_SVM = SVM_classifier.predict(x_test)
In [0]:
# The coefficients
#print('Coefficients: \n', logreg.coef_)
# The mean squared error
print("Mean squared error: %.2f"
% mean_squared_error(y_test, y_pred_SVM))
# Explained variance score: 1 is perfect prediction
print('R2 score: %.2f' % r2_score(y_test, y_pred_SVM))
In [0]:
cf =confusion_matrix(y_test,y_pred_SVM)
In [0]:
plt.imshow(cf,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(y_test))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf.max() / 2.
for i,j in itertools.product(range(cf.shape[0]),range(cf.shape[1])):
plt.text(j,i,format(cf[i,j],'d'),horizontalalignment='center',color='white' if cf[i,j] >thresh else 'black')
plt.show();
Naive Bayes model¶
In [0]:
nb = MultinomialNB()
nb.fit(x_train,y_train)
y_nb_predict = nb.predict(x_test)
In [0]:
score = metrics.accuracy_score(y_test, y_nb_predict)
print("Accuracy score: {}".format(score))
In [0]:
score_tf_precision_stopping = metrics.precision_score(y_test, y_nb_predict, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(y_test, y_nb_predict, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(y_test, y_nb_predict, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(y_test, y_nb_predict)
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(y_test.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
KNN¶
In [0]:
#knn
from sklearn.neighbors import KNeighborsClassifier
knn_extra = KNeighborsClassifier(n_neighbors=10)
knn_extra.fit(x_train,y_train)
y_pred_knn_e = knn_extra.predict(x_test)
In [0]:
score = metrics.accuracy_score(y_test, y_pred_knn_e)
print("Accuracy score: {}".format(score))
score_tf_precision_stopping = metrics.precision_score(y_test, y_pred_knn_e, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(y_test, y_pred_knn_e, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(y_test,y_pred_knn_e, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(y_test,y_pred_knn_e)
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(y_test.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
TensorFlow Classification¶
In [0]:
import keras
import keras
from keras.models import Sequential
from keras import regularizers
from keras.layers.core import Dense, Activation
from keras.layers import Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.callbacks import EarlyStopping, ModelCheckpoint
In [0]:
num_classes = 2
#Convert class vectors to one hot format
y_train1 = keras.utils.to_categorical(y_train,num_classes)
y_test1 = keras.utils.to_categorical(y_test,num_classes)
print(x_train.shape)
print(y_test1.shape)
In [0]:
checkpointer2 = ModelCheckpoint(filepath="./best1.hdf5", verbose=0, save_best_only=True) # save best model
In [0]:
for i in range(5):
print(i)
model = Sequential()
model.add(Dense(25, input_dim=x_train.shape[1], activation='relu')) # Hidden 1
model.add(Dense(10, activation='relu')) # Hidden 2
model.add(Dense(y_train1.shape[1], activation='softmax'))
#adam = optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=6, verbose=2, mode='auto')
model.compile(loss="categorical_crossentropy", optimizer='adam')
model.fit(x_train,y_train1,callbacks=[monitor,checkpointer2],validation_data=(x_test, y_test1),epochs=10)
In [0]:
model.load_weights('./best1.hdf5')
y_pred_c = model.predict(x_test)
y_pred_c = np.argmax(y_pred_c,axis=1)
y_test_c= np.argmax(y_test1,axis=1)
score = metrics.accuracy_score(y_test_c, y_pred_c)
print("Accuracy score: {}".format(score))
In [0]:
score = metrics.accuracy_score(y_test_c, y_pred_c)
print("Accuracy score: {}".format(score))
score_tf_precision_stopping = metrics.precision_score(y_test_c, y_pred_c, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(y_test_c, y_pred_c, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(y_test_c, y_pred_c, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(y_test_c, y_pred_c)
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(y_test_c.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
Testing with over sampling data Using SMOTENC¶
In [0]:
y_df_oversampling = df_for_sampling['attempt_suicide']
In [0]:
y_df_oversampling_array = y_df_oversampling.values
In [0]:
y_df_oversampling_array
Out[0]:
In [0]:
x_df_oversampling = df_for_sampling.drop(columns="attempt_suicide")
In [0]:
x_df_oversampling_array = x_df_oversampling.values
In [0]:
x_df_oversampling_array[:1]
Out[0]:
In [0]:
y_df_oversampling
Out[0]:
In [0]:
x_df_oversampling
Out[0]:
In [0]:
x_train_over_sampling,x_test_over_sampling,y_train_over_sampling,y_test_oversampling = train_test_split(x_df_oversampling_array,y_df_oversampling_array, test_size=0.3,random_state=42)
In [0]:
x_train_over_sampling.shape
Out[0]:
In [0]:
x_test_over_sampling.shape
Out[0]:
In [0]:
y_train_over_sampling.shape
Out[0]:
In [0]:
y_test_oversampling.shape
Out[0]:
OverSampling Technique¶
In [0]:
>>> from imblearn.over_sampling import SMOTENC
>>> smote_nc = SMOTENC(categorical_features=[0,1,2,3,4,5,6,7,8,10,11,12,13,14], random_state=0)
>>> x_resampled, y_resampled = smote_nc.fit_resample(x_train_over_sampling, y_train_over_sampling)
>>> print(sorted(collections.Counter(y_resampled).items()))
In [0]:
y_column = ['attempt_suicide']
x_columns = ["gender","sexuallity","age","income","race","bodyweight","virgin","prostitution_legal","pay_for_sex","friends","social_fear","depressed","employment","job_title","edu_level"]
In [0]:
x_resampled_dataframe = pd.DataFrame(x_resampled, columns=x_columns)
x_resampled_dataframe.shape
Out[0]:
In [0]:
x_resampled_dataframe[:1]
Out[0]:
In [0]:
y_resampled_dataframe = pd.DataFrame(y_resampled, columns=y_column)
y_resampled_dataframe.shape
Out[0]:
In [0]:
x_test_dataframe = pd.DataFrame(x_test_over_sampling, columns=x_columns)
x_test_dataframe.shape
Out[0]:
In [0]:
y_test_dataframe = pd.DataFrame(y_test_oversampling, columns=y_column)
y_test_dataframe.shape
Out[0]:
In [0]:
# Label Encoding Only for Y (output)
def encode_text_index1(df_arg1, name):
le = preprocessing.LabelEncoder()
df_arg1[name] = le.fit_transform(df_arg1[name])
return le.classes_
In [0]:
my_list = ['attempt_suicide']
for i in my_list:
encode_text_index1(y_resampled_dataframe,i)
encode_text_index1(y_test_dataframe,i)
In [0]:
def encode_text_dummy1(df_arg, name):
dummies = pd.get_dummies(df_arg[name])
print(dummies)
for x in dummies.columns:
dummy_name = "{}-{}".format(name, x)
df_arg[dummy_name] = dummies[x]
df_arg.drop(name, axis=1, inplace=True)
for x in dummies.columns:
dummy_name = "{}-{}".format(name, x)
x_test_dataframe[dummy_name] = dummies[x]
x_test_dataframe.drop(name, axis=1, inplace=True)
In [0]:
print(x_resampled_dataframe.shape)
print(x_test_dataframe.shape)
mylist_3 = ["gender","age","sexuallity","income","race","bodyweight","virgin","prostitution_legal","pay_for_sex","social_fear","depressed","employment","job_title","edu_level"]
for t in mylist_3:
encode_text_dummy1(x_resampled_dataframe,t)
#encode_text_dummy1(x_test_dataframe,t)
print(x_resampled_dataframe.shape)
print(x_test_dataframe.shape)
In [0]:
final_x_train_resampled = x_resampled_dataframe.values
final_y_train_resampled = y_resampled_dataframe.values
final_x_test_resampled = x_test_dataframe.values
final_y_test_resampled = y_test_dataframe.values
final_y_train_resampled = final_y_train_resampled.flatten()
final_y_test_resampled = final_y_test_resampled.flatten()
Logistic Regression¶
In [0]:
logreg_resampled = linear_model.LogisticRegression()
In [0]:
final_y_train_resampled[:5]
Out[0]:
In [0]:
logreg_resampled.fit(final_x_train_resampled, final_y_train_resampled.flatten())
Out[0]:
In [0]:
y_pred_logistic_resampled = logreg_resampled.predict(final_x_test_resampled)
In [0]:
# The coefficients
#print('Coefficients: \n', logreg.coef_)
# The mean squared error
print("Mean squared error: %.2f"
% mean_squared_error(final_y_test_resampled, y_pred_logistic_resampled))
In [0]:
score = metrics.accuracy_score(final_y_test_resampled, y_pred_logistic_resampled)
print("Accuracy score: {}".format(score))
In [0]:
score_tf_precision_stopping = metrics.precision_score(final_y_test_resampled, y_pred_logistic_resampled, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(final_y_test_resampled, y_pred_logistic_resampled, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(final_y_test_resampled, y_pred_logistic_resampled, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(final_y_test_resampled,y_pred_logistic_resampled)
In [0]:
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(final_y_test_resampled.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
Naive Bayes model¶
In [0]:
nb = MultinomialNB()
nb.fit(final_x_train_resampled,final_y_train_resampled)
y_nb_predict = nb.predict(final_x_test_resampled)
In [0]:
score = metrics.accuracy_score(final_y_test_resampled, y_nb_predict)
print("Accuracy score: {}".format(score))
In [0]:
score_tf_precision_stopping = metrics.precision_score(final_y_test_resampled, y_nb_predict, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(final_y_test_resampled, y_nb_predict, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(final_y_test_resampled,y_nb_predict, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(final_y_test_resampled,y_nb_predict)
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(final_y_test_resampled.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
KNN¶
In [0]:
#knn
from sklearn.neighbors import KNeighborsClassifier
knn_extra = KNeighborsClassifier(n_neighbors=10)
knn_extra.fit(final_x_train_resampled,final_y_train_resampled)
y_pred_knn_e = knn_extra.predict(final_x_test_resampled)
In [0]:
score = metrics.accuracy_score(final_y_test_resampled, y_pred_knn_e)
print("Accuracy score: {}".format(score))
score_tf_precision_stopping = metrics.precision_score(final_y_test_resampled, y_pred_knn_e, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(final_y_test_resampled, y_pred_knn_e, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(final_y_test_resampled,y_pred_knn_e, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(final_y_test_resampled,y_pred_knn_e)
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(final_y_test_resampled.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
Tensorflow classification¶
In [0]:
import keras
import keras
from keras.models import Sequential
from keras import regularizers
from keras.layers.core import Dense, Activation
from keras.layers import Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.callbacks import EarlyStopping, ModelCheckpoint
In [0]:
num_classes = 2
#Convert class vectors to one hot format
final_y_train1_resampled = keras.utils.to_categorical(final_y_train_resampled,num_classes)
final_y_test1_resampled = keras.utils.to_categorical(final_y_test_resampled,num_classes)
print(final_x_train_resampled.shape)
print(final_y_test1_resampled.shape)
In [0]:
checkpointer2 = ModelCheckpoint(filepath="./best1.hdf5", verbose=0, save_best_only=True) # save best model
In [0]:
for i in range(5):
print(i)
model = Sequential()
model.add(Dense(25, input_dim=final_x_train_resampled.shape[1], activation='relu')) # Hidden 1
model.add(Dense(10, activation='relu')) # Hidden 3
model.add(Dense(final_y_train1_resampled.shape[1], activation='softmax'))
#adam = optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=6, verbose=2, mode='auto')
model.compile(loss="categorical_crossentropy", optimizer='adam')
model.fit(final_x_train_resampled,final_y_train1_resampled,callbacks=[monitor,checkpointer2],validation_data=(final_x_test_resampled, final_y_test1_resampled),epochs=10)
In [0]:
model.load_weights('./best1.hdf5')
y_pred_c = model.predict(final_x_test_resampled)
y_pred_c = np.argmax(y_pred_c,axis=1)
y_test_c= np.argmax(final_y_test1_resampled,axis=1)
score = metrics.accuracy_score(y_test_c, y_pred_c)
print("Accuracy score: {}".format(score))
In [0]:
score = metrics.accuracy_score(y_test_c, y_pred_c)
print("Accuracy score: {}".format(score))
score_tf_precision_stopping = metrics.precision_score(y_test_c, y_pred_c, average= "weighted")
print("Precision score: {}".format(score_tf_precision_stopping))
score_tf_recall_stopping = metrics.recall_score(y_test_c, y_pred_c, average= "weighted")
print("Recall score: {}".format(score_tf_recall_stopping))
score_tf_f1_stopping = metrics.f1_score(y_test_c, y_pred_c, average= "weighted")
print("F1 score: {}".format(score_tf_f1_stopping))
In [0]:
cf1 =confusion_matrix(y_test_c, y_pred_c)
plt.imshow(cf1,cmap=plt.cm.Blues,interpolation='nearest')
plt.colorbar()
plt.title('Confusion Matrix without Normalization')
plt.xlabel('Predicted')
plt.ylabel('Actual')
tick_marks = np.arange(len(set(y_test_c.flatten()))) # length of classes
class_labels = ['0','1']
tick_marks
plt.xticks(tick_marks,class_labels)
plt.yticks(tick_marks,class_labels)
# plotting text value inside cells
thresh = cf1.max() / 2.
for i,j in itertools.product(range(cf1.shape[0]),range(cf1.shape[1])):
plt.text(j,i,format(cf1[i,j],'d'),horizontalalignment='center',color='white' if cf1[i,j] >thresh else 'black')
plt.show();
Discussion¶
Based on the exploratory data analysis records which are classified as suicide attempt as positive, counts to 85 and records which are classified as never attempt a suicide is 372. Data is biased because one class has more number of records as compared to other. Therefore, accuracy won't be the best measure to predict on such biased data.¶
Over sampling is done to the records of class which has minimum number of records. This helped to make number of records in both classes to be equal. The technique used to do oversampling is by using SMOTE-NC (Synthetic Minority Over-sampling Technique for Nominal and Continuous) as dataset contains both numerical and categorical data. Important thing to note here is that oversampling is performed only on training set after train test split to avoid synthetic records to be added to test set which does not represent the actual test set and real world recordings/readings.¶
After oversampling, confusion matrix shows that correct prediction on class which had less records has increased but not to a substantial amount. Moreover incorrect classification also increased where majority class is present. Hence we cannot rely completely on accuracy metric here again and need more evidence on this behavior of this model.¶
Conclusion¶
Accuracy for various developed models are in range from 54% to 77%. But accuracy is not the best measure to predict the model performance/ quality when we look at the confusion matrix.¶
Moreover having more number of records might help Neural network models to learn features of classes, due to behavior observed in confusion matrix.¶
References¶
Dataset¶
https://www.kaggle.com/kingburrito666/the-demographic-rforeveralone-dataset/home
Synthetic Minority Over-sampling Technique for Nominal and Continuous¶
https://imbalancedlearn.readthedocs.io/en/stable/generated/imblearn.over_sampling.SMOTENC.html
Scikit Learn Official Website for documentation¶
https://scikit-learn.org/stable/
Pandas Documentation¶
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html