Types of Activation Functions: Sigmoid tanh, ReLU, Softmax. Part 2

Here’s a Python program that demonstrates a basic Artificial Neural Network (ANN) using Keras, a high-level API in TensorFlow. We’ll build an ANN to classify whether a person survived or not based on features from the Titanic dataset, a popular dataset for binary classification tasks.

For this example, we will use the following steps:

1. Import Libraries
2. Prepare Data (Load and preprocess data)
3. Define the ANN model with activation functions
4. Compile and Train the Model
5. Evaluate the Model

Data Source: We’ll use the Titanic dataset from Kaggle, which you can download and load into this program.

Step 1: Import Libraries

import numpy as np 
import pandas as pd from sklearn.model_selection 
import train_test_split from sklearn.preprocessing 
import StandardScaler from tensorflow.keras.models 
import Sequential from tensorflow.keras.layers 
import Dense from tensorflow.keras.utils 
import to_categorical

Step 2: Load and Preprocess Data

Download the Titanic dataset, load it into a Pandas DataFrame, and preprocess it.

data = pd.read_csv("titanic.csv") 
# Selecting features and target 
X = data[['Pclass', 'Age', 'SibSp', 'Parch', 'Fare']].fillna(0) # Fill missing values with 0 
y = data['Survived'] 

# Split data into train and test sets 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) 
# Normalize the data 
scaler = StandardScaler() 
X_train = scaler.fit_transform(X_train) 
X_test = scaler.transform(X_test)

Step 3: Define the ANN Model

Define a simple ANN with multiple layers. We’ll use:

• Input layer: Based on the input features
• Hidden layers: With ReLU activation function
• Output layer: Sigmoid activation function for binary classification

model = Sequential()

# Input layer with 5 neurons (features) and first hidden layer with 16 neurons
model.add(Dense(units=16, activation='relu', input_shape=(X_train.shape[1],)))
# Second hidden layer with 8 neurons
model.add(Dense(units=8, activation='relu'))
# Output layer with sigmoid activation for binary classification
model.add(Dense(units=1, activation='sigmoid'))

Step 4: Compile and Train the Model

Compile the model with the Adam optimizer and binary cross-entropy loss function, then train it.

# Compile the model 
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) 
# Train the model 
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.2)

Step 5: Evaluate the Model

Evaluate the model on the test set to check its performance.

# Evaluate the model on test data 
test_loss, test_accuracy = model.evaluate(X_test, y_test) print("Test Accuracy:", test_accuracy)

Full Code Recap

Here’s the entire code for the program:

import numpy as np 
import pandas as pd from sklearn.model_selection 
import train_test_split from sklearn.preprocessing 
import StandardScaler from tensorflow.keras.models 
import Sequential from tensorflow.keras.layers 
import Dense 

# Load and preprocess data 
data = pd.read_csv("titanic.csv") 
X = data[['Pclass', 'Age', 'SibSp', 'Parch', 'Fare']].fillna(0) 
y = data['Survived'] 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) scaler = StandardScaler() 
X_train = scaler.fit_transform(X_train) 
X_test = scaler.transform(X_test) 
# Define the ANN model 
model = Sequential() 
model.add(Dense(units=16, activation='relu', input_shape=(X_train.shape[1],))) model.add(Dense(units=8, activation='relu')) 
model.add(Dense(units=1, activation='sigmoid')) 
# Compile the model 
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) 
# Train the model 
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.2) 
# Evaluate the model 
test_loss, test_accuracy = model.evaluate(X_test, y_test) 
print("Test Accuracy:", test_accuracy)

Explanation

• Layers:
• Training: We train the model on 50 epochs, adjusting weights each time for better accuracy.
• Evaluation: The evaluate function provides the test accuracy, indicating how well the model performs on unseen data.

This example demonstrates a complete workflow for building, training, and evaluating an ANN on a binary classification problem!