tensorflow /tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

★ 17,471⑂ 3,829👁 9⑃ 671● 1,245▤ 75
PythonApache-2.0Created 2017-06-15
More from this author
Download: v1.15.7v1.15.7 ↓1.15.72020-06-17
+0
Daily
17,471
stars
3,829
forks
9
watchers
671
prs
1,245
issues
75
releases

README

Tensor2Tensor









Tensor2Tensor, or
T2T for short, is a library
of deep learning models and datasets designed to make deep learning more
accessible and accelerate ML
research.


T2T was developed by researchers and engineers in the
Google Brain team and a community
of users. It is now deprecated — we keep it running and welcome
bug-fixes, but encourage users to use the successor library Trax.

 Quick Start

This iPython notebook
explains T2T and runs in your browser using a free VM from Google,
no installation needed. Alternatively, here is a one-command version that
installs T2T, downloads MNIST, trains a model and evaluates it:


pip install tensor2tensor && t2t-trainer \
  --generate_data \
  --data_dir=~/t2t_data \
  --output_dir=~/t2t_train/mnist \
  --problem=image_mnist \
  --model=shake_shake \
  --hparams_set=shake_shake_quick \
  --train_steps=1000 \
  --eval_steps=100


 Contents

 Suggested Datasets and Models
   Mathematical Language Understanding
   Story, Question and Answer
   Image Classification
   Image Generation
   Language Modeling
   Sentiment Analysis
   Speech Recognition
   Summarization
   Translation
 Basics
   Walkthrough
   Installation
   Features
 T2T Overview
   Datasets
   Problems and Modalities
   Models
   Hyperparameter Sets
   Trainer
 Adding your own components
 Adding a dataset
 Papers
 Run on FloydHub

 Suggested Datasets and Models

Below we list a number of tasks that can be solved with T2T when
you train the appropriate model on the appropriate problem.
We give the problem and model below and we suggest a setting of
hyperparameters that we know works well in our setup. We usually
run either on Cloud TPUs or on 8-GPU machines; you might need
to modify the hyperparameters if you run on a different setup.

 Mathematical Language Understanding

For evaluating mathematical expressions at the character level involving addition, subtraction and multiplication of both positive and negative decimal numbers with variable digits assigned to symbolic variables, use

 the MLU data-set:
 --problem=algorithmic_math_two_variables

You can try solving the problem with different transformer models and hyperparameters as described in the paper:
 Standard transformer:
--model=transformer
--hparams_set=transformer_tiny
 Universal transformer:
--model=universal_transformer
--hparams_set=universal_transformer_tiny
 Adaptive universal transformer:
--model=universal_transformer
--hparams_set=adaptive_universal_transformer_tiny

 Story, Question and Answer

For answering questions based on a story, use

 the bAbi data-set:
 --problem=babi_qa_concat_task1_1k

You can choose the bAbi task from the range [1,20] and the subset from 1k or
10k. To combine test data from all tasks into a single test set, use
--problem=babi_qa_concat_all_tasks_10k

 Image Classification

For image classification, we have a number of standard data-sets:

 ImageNet (a large data-set): --problem=image_imagenet, or one
   of the re-scaled versio

Top issues

Latest release

v1.15.7 v1.15.7 2020-06-17
Multistep Adam Optimizer many thanks to @AgoloCuongHoang for contributing in 1773 !
  Residual Shuffle-Exchange Network thanks to @EmilsOzolins in 1805 !
  Not pinning the gym version.

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON