-1
Daily
+6
Weekly
+13
Monthly
16,345
stars
1,542
forks
115
watchers
98
prs
464
issues
17
releases
README
Kolmogorov-Arnold Networks (KANs) This is the github repo for the paper "KAN: Kolmogorov-Arnold Networks" and "KAN 2.0: Kolmogorov-Arnold Networks Meet Science". You may want to quickstart with hellokan, try more examples in tutorials, or read the documentation here. Kolmogorov-Arnold Networks (KANs) are promising alternatives of Multi-Layer Perceptrons (MLPs). KANs have strong mathematical foundations just like MLPs: MLPs are based on the universal approximation theorem, while KANs are based on Kolmogorov-Arnold representation theorem. KANs and MLPs are dual: KANs have activation functions on edges, while MLPs have activation functions on nodes. This simple change makes KANs better (sometimes much better!) than MLPs in terms of both model accuracy and interpretability. A quick intro of KANs here. Installation Pykan can be installed via PyPI or directly from GitHub. Pre-requisites: Python 3.9.7 or higher pip For developers git clone https://github.com/KindXiaoming/pykan.git cd pykan pip install -e . Installation via github pip install git+https://github.com/KindXiaoming/pykan.git Installation via PyPI: pip install pykan Requirements python python==3.9.7 matplotlib==3.6.2 numpy==1.24.4 scikit_learn==1.1.3 setuptools==65.5.0 sympy==1.11.1 torch==2.2.2 tqdm==4.66.2 pandas==2.0.1 seaborn pyyaml After activating the virtual environment, you can install specific package requirements as follows: python pip install -r requirements.txt Optional: Conda Environment Setup For those who prefer using Conda: conda create --name pykan-env python=3.9.7 conda activate pykan-env pip install git+https://github.com/KindXiaoming/pykan.git For GitHub installation or pip install pykan For PyPI installation Efficiency mode For many machine-learning users, when (1) you need to write the training loop yourself (instead of using model.fit()); (2) you never use the symbolic branch, it is important to call model.speed() before training! Otherwise, the symbolic branch is on, which is super slow because the symbolic computations are not parallelized! Computation requirements Examples in tutorials are runnable on a single CPU typically less than 10 minutes. All examples in the paper are runnable on a single CPU in less than one day. Training KANs for PDE is the most expensive and may take hours to days on a single CPU. We use CPUs to train our models because we carried out parameter sweeps (both for MLPs and KANs) to obtain Pareto Frontiers. There are thousands of small models which is why we use CPUs rather than GPUs. Admittedly, our problem scales are smaller than typical machine learning tasks but are typical for science-related tasks. In case the scale of your task is large, it is advisable to use GPUs. Documentation The documentation can be found here. Tutorials Quickstart Get started with hellokan.ipynb notebook. More demos More Notebook tutorials can be found in tutorials. Advice on hyperparameter tuning Many intuition about
Top issues
#199When running on Apple GPU (MPS), the loss is always nan.28 comments · 2024-05-15#376Any step-by-step guide on KAN with code?15 comments · 2024-07-26#151Running Example 2: Deep Formulas on my computer yielded inconsistent results15 comments · 2024-05-10#107M1 runtime fails with "AssertionError: Torch not compiled with CUDA enabled"14 comments · 2024-05-07#379HelloKAN error: train_loss: nan | test_loss: nan | reg: nan 10 comments · 2024-07-28#336Trouble Running "Hello, KAN!" Code: SyntaxError in MultKAN.py10 comments · 2024-07-14#225The loss did not decrease when training on the observations 10 comments · 2024-05-21#172How to use KAN to fit a implicit function?9 comments · 2024-05-12
Latest release
v0.2.8 v0.2.8 2024-11-14Full Changelog: https://github.com/KindXiaoming/pykan/compare/v0.2.7...v0.2.8 update grid extension to avoid loss spikes