v0.1 release

* Update README
* Update version to `v0.1` in version.py, setup.py, CHANGELOG.md

Author: ZhitingHu

CHANGELOG.md

@@ -1,11 +1,7 @@
 
-## [Unreleased]
+## [v0.1.0](https://github.com/asyml/texar-pytorch/releases/tag/v0.1.0) (2019-10-15)
 
-### New features
-
-### Feature improvements
-
-### Fixes
+The first formal release of Texar-PyTorch
 
 ## [v0.0.1](https://github.com/asyml/texar-pytorch/releases/tag/v0.0.1) (2019-08-02)
 

README.md

@@ -9,87 +9,105 @@
 [![Documentation Status](https://readthedocs.org/projects/texar-pytorch/badge/?version=latest)](https://texar-pytorch.readthedocs.io/en/latest/?badge=latest)
 [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/asyml/texar-pytorch/blob/master/LICENSE)
 
-*(Note: This is the alpha release of Texar-PyTorch.)* 
- 
-**Texar-PyTorch** is an open-source toolkit based on PyTorch, aiming to support a broad set of machine learning, especially text generation tasks, such as machine translation, dialog, summarization, content manipulation, language modeling, and so on. Texar is designed for both researchers and practitioners for fast prototyping and experimentation.
 
-*If you work with TensorFlow, be sure to check out **[Texar (TensorFlow)](https://github.com/asyml/texar)** which has (mostly) the **same functionalities and interfaces**.*
+**Texar-PyTorch** is a toolkit aiming to support a broad set of machine learning, especially natural language processing and text generation tasks. Texar provides a library of easy-to-use ML modules and functionalities for composing whatever models and algorithms. The tool is designed for both researchers and practitioners for fast prototyping and experimentation.
 
-With the design goals of **modularity, versatility, and extensibility** in mind, Texar extracts the common patterns underlying the diverse tasks and methodologies, creates a library of highly reusable modules and functionalities, and facilitates **arbitrary model architectures and algorithmic paradigms**, e.g., 
-   * encoder(s) to decoder(s), sequential- and self-attentions, memory, hierarchical models, classifiers, ... 
-   * maximum likelihood learning, reinforcement learning, adversarial learning, probabilistic modeling, ... 
-   * **pre-trained models** such as **BERT**, **GPT2**, **XLNet**, ...
+Texar-PyTorch integrates many of the best features of TensorFlow into PyTorch, delivering highly usable and customizable modules superior to PyTorch native ones. 
 
-With Texar, cutting-edge complex models can be easily constructed, freely enriched with best modeling/training practices, readily fitted into standard training/evaluation pipelines, and quickly experimented and evolved by, e.g., plugging in and swapping out different modules.
+### Key Features
+* **Two Versions, (Mostly) Same Interfaces**. Texar-PyTorch (this repo) and **[Texar-TF](https://github.com/asyml/texar)** have mostly the same interfaces. Both further combine the best design of TF and PyTorch:
+  - Interfaces and variable sharing in *PyTorch convention*
+  - Excellent factorization and rich functionalities in *TF convention*.
+* **Versatile** to support broad needs:
+   - data processing, model architectures, loss functions, training and inference algorithms, evaluation, ...
+   - encoder(s) to decoder(s), sequential- and self-attentions, memory, hierarchical models, classifiers, ... 
+   - maximum likelihood learning, reinforcement learning, adversarial learning, probabilistic modeling, ... 
+* **Fully Customizable** at multiple abstraction level -- both novice-friendly and expert-friendly. 
+  - Free to plug in whatever external modules, since Texar is fully compatible with the native PyTorch APIs. 
+* **Modularized** for maximal re-use and clean APIs, based on principled decomposition of *Learning-Inference-Model Architecture*. 
+* **Rich Pre-trained Models, Rich Usage with Uniform Interfaces**. BERT, GPT2, XLNet, etc, for encoding, classification, generation, and composing complex models with other Texar components!
+* Clean, detailed [documentation](https://texar-pytorch.readthedocs.io) and rich [examples](./examples).
 
 <div align="center">
    <img src="./docs/_static/img/texar_stack.png"><br><br>
 </div> 
 
-
-### Key Features
-* **Versatility**. Texar contains a wide range of modules and functionalities for composing arbitrary model architectures and implementing various learning algorithms, as well as for data processing, evaluation, prediction, etc.
-* **Modularity**. Texar decomposes diverse, complex machine learning models and algorithms into a set of highly-reusable modules. In particular, model **architecture, losses, and learning processes** are fully decomposed.  
-Users can construct their own models at a high conceptual level, just like assembling building blocks. It is convenient to plug in or swap out modules, and configure rich options for each module. For example, switching between maximum-likelihood learning and reinforcement learning involves only changing several lines of code.
-* **Extensibility**. It is straightforward to integrate any user-customized, external modules. Also, Texar is fully compatible with the native PyTorch interfaces and can take advantage of the rich PyTorch features, and resources from the vibrant open-source community.
-* Interfaces with different functionality levels. Users can customize a model through 1) simple **Python/YAML configuration files** of provided model templates/examples; 2) programming with **Python Library APIs** for maximal customizability.
-* Easy-to-use APIs; rich configuration options for each module, all with default values.
-* Well-structured high-quality code of uniform design patterns and consistent styles. 
-* Clean, detailed [documentation](https://texar-pytorch.readthedocs.io) and rich [examples](./examples).
+<div align="center">
+   <img src="./docs/_static/img/texar_modules_big.png"><br><br>
+</div> 
 
 
 ### Library API Example
-A code portion that builds a (self-)attentional sequence encoder-decoder model:
+A code example that builds and trains a **Conditional GPT2** model (e.g., for machine translation and text summarization):
 ```python
 import texar.torch as tx
-
-class Seq2Seq(tx.ModuleBase):
-  def __init__(self, data):
-    self.embedder = tx.modules.WordEmbedder(
-        data.target_vocab.size, hparams=hparams_emb)
-    self.encoder = tx.modules.TransformerEncoder(
-        hparams=hparams_encoder)  # config through `hparams`
-    self.decoder = tx.modules.AttentionRNNDecoder(
-        token_embedder=self.embedder,
-        input_size=self.embedder.dim,
-      	encoder_output_size=self.encoder.output_size,
-      	vocab_size=data.target_vocab.size,
-        hparams=hparams_decoder)
-
-  def forward(self, batch): 
-    outputs_enc = self.encoder(
-        inputs=self.embedder(batch['source_text_ids']),
-        sequence_length=batch['source_length'])
-     
-    outputs, _, _ = self.decoder(
-        memory=outputs_enc, 
-        memory_sequence_length=batch['source_length'],
-        helper=self.decoder.get_helper(decoding_strategy='train_greedy'), 
-        inputs=batch['target_text_ids'],
-        sequence_length=batch['target_length']-1)
-
-    # Loss for maximum likelihood learning
-    loss = tx.losses.sequence_sparse_softmax_cross_entropy(
-        labels=batch['target_text_ids'][:, 1:],
-        logits=outputs.logits,
-        sequence_length=batch['target_length']-1)  # Automatic masking
-
-    return loss
-
-
-data = tx.data.PairedTextData(hparams=hparams_data) 
-iterator = tx.data.DataIterator(data)
-
-model = Seq2seq(data)
-for batch in iterator:
-    loss = model(batch)
-    # ...
+from texar.torch.run import *
+
+# (1) Modeling
+class ConditionalGPT2Model(nn.Module):
+  """An encoder-decoder model with GPT-2 as the decoder."""
+  def __init__(self, vocab_size):
+    super().__init__()
+    # Use hyperparameter dict for model configuration
+    self.embedder = tx.modules.WordEmbedder(vocab_size, hparams=emb_hparams)
+    self.encoder = tx.modules.TransformerEncoder(hparams=enc_hparams)
+    self.decoder = tx.modules.GPT2Decoder("gpt2-small")  # With pre-trained weights
+
+  def _get_decoder_output(self, batch, train=True):
+    """Perform model inference, i.e., decoding."""
+    enc_states = self.encoder(inputs=self.embedder(batch['source_text_ids']),
+                              sequence_length=batch['source_length'])
+    if train:  # Teacher-forcing decoding at training time
+      return self.decoder(
+          inputs=batch['target_text_ids'], sequence_length=batch['target_length'] - 1,
+          memory=enc_states, memory_sequence_length=batch['source_length'])
+    else:      # Beam search decoding at prediction time
+      start_tokens = torch.full_like(batch['source_text_ids'][:, 0], BOS)
+      return self.decoder(
+          beam_width=5, start_tokens=start_tokens,
+          memory=enc_states, memory_sequence_length=batch['source_length'])
+
+  def forward(self, batch):
+    """Compute training loss."""
+    outputs = self._get_decoder_output(batch)
+    loss = tx.losses.sequence_sparse_softmax_cross_entropy(  # Sequence loss
+        labels=batch['target_text_ids'][:, 1:], logits=outputs.logits,
+        sequence_length=batch['target_length'] - 1)  # Automatic masking
+    return {"loss": loss}
+
+  def predict(self, batch):
+    """Compute model predictions."""
+    sequence, _ = self._get_decoder_output(batch, train=False)
+    return {"gen_text_ids": sequence}
+
+  
+# (2) Data
+# Create dataset splits using built-in data loaders
+datasets = {split: tx.data.PairedTextData(hparams=data_hparams[split])
+            for split in ["train", "valid", "test"]}
+
+model = ConditionalGPT2Model(datasets["train"].target_vocab.size)
+
+# (3) Training
+# Manage the train-eval loop with the Executor API
+executor = Executor(
+  model=model, datasets=datasets,
+  optimizer={"type": torch.optim.Adam, "kwargs": {"lr": 5e-4}},
+  stop_training_on=cond.epoch(20),
+  log_every=cond.iteration(100),
+  validate_every=cond.epoch(1),
+  train_metric=("loss", metric.RunningAverage(10, pred_name="loss")),
+  valid_metric=metric.BLEU(pred_name="gen_text_ids", label_name="target_text_ids"),
+  save_every=cond.validation(better=True),
+  checkpoint_dir="outputs/saved_models/")
+executor.train()
+executor.test(datasets["test"])
 ```
 Many more examples are available [here](./examples).
 
 
 ### Installation
-Texar-PyTorch requires PyTorch 1.0 or higher. Please follow the [official instructions](https://pytorch.org/get-started/locally/#start-locally) to install the appropriate version.
+Texar-PyTorch requires `PyTorch>=1.0`. Please follow the [official instructions](https://pytorch.org/get-started/locally/#start-locally) to install the appropriate version.
 
 After PyTorch is installed, please run the following commands to install Texar-PyTorch:
 ```
@@ -114,16 +132,15 @@ If you use Texar, please cite the [tech report](https://arxiv.org/abs/1809.00794
 ```
 Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
 Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan and Eric Xing
-2018
+ACL 2019
 
-@article{hu2018texar,
+@inproceedings{hu2019texar,
   title={Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation},
   author={Hu, Zhiting and Shi, Haoran and Tan, Bowen and Wang, Wentao and Yang, Zichao and Zhao, Tiancheng and He, Junxian and Qin, Lianhui and Wang, Di and others},
-  journal={arXiv preprint arXiv:1809.00794},
-  year={2018}
+  booktitle={ACL 2019, System Demonstrations},
+  year={2019}
 }
 ```
 
-
 ### License
 [Apache License 2.0](./LICENSE)

docs/_static/img/texar_modules_big.png

@@ -17,7 +17,7 @@
 
 setuptools.setup(
     name="texar-pytorch",
-    version="0.0.1",
+    version="0.1.0",
     url="https://github.com/asyml/texar-pytorch",
 
     description="Toolkit for Machine Learning and Text Generation",

setup.py

@@ -13,8 +13,8 @@
 # limitations under the License.
 
 _MAJOR = "0"
-_MINOR = "0"
-_REVISION = "1-unreleased"
+_MINOR = "1"
+_REVISION = "0"
 
 VERSION_SHORT = "{0}.{1}".format(_MAJOR, _MINOR)
 VERSION = "{0}.{1}.{2}".format(_MAJOR, _MINOR, _REVISION)