A collection of various gradient descent algorithms implemented in Python from scratch

[ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.

NAG-GS: Nesterov Accelerated Gradients with Gauss-Siedel splitting

Simple Document Classification using Multi Class Logistic Regression & SVM Soft Margin from scratch

Using Matrix Factorization/Probabilistic Matrix Factorization to solve Recommendation。矩阵分解进行推荐系统算法。

Python code for Gradient Descent, Momentum, and Adam optimization methods. Train neural networks efficiently.

Numerical Optimization for Machine Learning & Data Science

In this project it is used a Machine Learning model based on a method called Extreme Learning, with the employment of L2-regularization. In particular, a comparison was carried out between: (A1) which is a variant of incremental extreme learning machine that is QRIELM and (A2) which is a standard momentum descent approach, applied to the ELM.

This is an implementation of different optimization algorithms such as: - Gradient Descent (stochastic - mini-batch - batch) - Momentum - NAG - Adagrad - RMS-prop - BFGS - Adam Also, most of them are implemented in vectorized form for multi-variate problems

Generic L-layer 'straight in Python' fully connected Neural Network implementation using numpy.

EE456 2022 mini project implementation of two-moons problem using multi-layer-perceptron with back-propagation with analyzing performance of initializing methods and momentum rule

Machine Learning, Deep Learning Implementations

This repository contains a python implementation of Feed Forward Neural Network with Backpropagation, along with the example scripts for training the network to classify images from mnist and fashion_mnist datasets from keras.

Comparsion of Machine Learning optimization algorithms with MNIST dataset