ddbourgin /numpy-ml

Machine learning, in numpy

★ 16,322⑂ 3,730👁 445⑃ 72● 53▤ 0
PythonGPL-3.0Created 2019-04-06numpy-ml.readthedocs.io/
+0
Daily
16,322
stars
3,730
forks
445
watchers
72
prs
53
issues
0
releases

README

numpy-ml
Ever wish you had an inefficient but somewhat legible collection of machine
learning algorithms implemented exclusively in NumPy? No?

 Installation

 For rapid experimentation
To use this code as a starting point for ML prototyping / experimentation, just clone the repository, create a new virtualenv, and start hacking:

sh
$ git clone https://github.com/ddbourgin/numpy-ml.git
$ cd numpy-ml && virtualenv npml && source npml/bin/activate
$ pip3 install -r requirements-dev.txt


 As a package
If you don't plan to modify the source, you can also install numpy-ml as a
Python package: pip3 install -u numpy_ml.

The reinforcement learning agents train on environments defined in the OpenAI
gym. To install these alongside numpy-ml, you
can use pip3 install -u 'numpy_ml[rl]'.

 Documentation
For more details on the available models, see the project documentation.

 Available models

  Click to expand!

1. Gaussian mixture model
    - EM training

2. Hidden Markov model
    - Viterbi decoding
    - Likelihood computation
    - MLE parameter estimation via Baum-Welch/forward-backward algorithm

3. Latent Dirichlet allocation (topic model)
    - Standard model with MLE parameter estimation via variational EM
    - Smoothed model with MAP parameter estimation via MCMC

4. Neural networks
     Layers / Layer-wise ops
        - Add
        - Flatten
        - Multiply
        - Softmax
        - Fully-connected/Dense
        - Sparse evolutionary connections
        - LSTM
        - Elman-style RNN
        - Max + average pooling
        - Dot-product attention
        - Embedding layer
        - Restricted Boltzmann machine (w. CD-n training)
        - 2D deconvolution (w. padding and stride)
        - 2D convolution (w. padding, dilation, and stride)
        - 1D convolution (w. padding, dilation, stride, and causality)
     Modules
        - Bidirectional LSTM
        - ResNet-style residual blocks (identity and convolution)
        - WaveNet-style residual blocks with dilated causal convolutions
        - Transformer-style multi-headed scaled dot product attention
     Regularizers
        - Dropout
     Normalization
        - Batch normalization (spatial and temporal)
        - Layer normalization (spatial and temporal)
     Optimizers
        - SGD w/ momentum
        - AdaGrad
        - RMSProp
        - Adam
     Learning Rate Schedulers
        - Constant
        - Exponential
        - Noam/Transformer
        - Dlib scheduler
     Weight Initializers
        - Glorot/Xavier uniform and normal
        - He/Kaiming uniform and normal
        - Standard and truncated normal
     Losses
        - Cross entropy
        - Squared error
        - Bernoulli VAE loss
        - Wasserstein loss with gradient penalty
        - Noise contrastive estimation loss
     Activations
        - ReLU
        - Tanh
        - Affine
        - Sigmoid
        - Leaky ReLU
        - ELU
        - SELU
        - GELU
        - Exponential
        - Hard Sigmoid
 

Top issues

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON