Document Type : Original Article

Authors

• Najmeh Hosseinipour-Mahani ¹
• Amirreza Jahantab ²

¹ Department of Applied Mathematics, Graduate University of Advanced Technology, Kerman, Iran

² Department of Computer science, Shahid Bahonar University of Kerman Kerman, Iran

Abstract

Abstract— Activation functions are essential for extracting meaningful relationships from real-world data in deep learning models. The design of activation functions is critical, as they directly influence the performance of these models. Nonlinear activation functions are commonly preferred since linear functions can limit a model’s learning capacity. Nonlinear activation functions can either have fixed parameters, which are predefined before training, or adjustable ones that modify during training. Fixed-parameter activation functions require the user to set the parameter values prior to model training. However, finding suitable parameters can be time-consuming and may slow down the convergence of the model. In this study, a novel fixed-parameter activation function is proposed and its performance is evaluated using benchmark MNIST datasets, demonstrating improvements in both accuracy and convergence speed.

Abstract— Activation functions are essential for extracting meaningful relationships from real-world data in deep learning models. The design of activation functions is critical, as they directly influence the performance of these models. Nonlinear activation functions are commonly preferred since linear functions can limit a model’s learning capacity. Nonlinear activation functions can either have fixed parameters, which are predefined before training, or adjustable ones that modify during training. Fixed-parameter activation functions require the user to set the parameter values prior to model training. However, finding suitable parameters can be time-consuming and may slow down the convergence of the model. In this study, a novel fixed-parameter activation function is proposed and its performance is evaluated using benchmark MNIST datasets, demonstrating improvements in both accuracy and convergence speed.

Keywords

• Keywords—Activation Function
• Deep Learning
• Fixed-Parameter
• Neural Networks
• MNIST Dataset
• Nonlinear Function
• Gradient Optimization
• Vanishing Gradient Problem

Main Subjects

• Artificial Intelligence, Robotics, Cognitive Computing