in ,

Write With Transformer, Hacker News

Write With Transformer, Hacker News

This web app, built by the Hugging Face team, is the official demo of thepytorch-transformersrepository’s text generation capabilities.


🤓 Arxiv-NLP

Built on the OpenAI GPT-2 model, the Hugging Face team has fine-tuned the small version on a tiny dataset (60 MB of text) of Arxiv papers. The targeted subject is Natural Language Processing, resulting in a very Linguistics / Deep Learning oriented generation.


🦄 GPT-2

The almighty king of text generation, GPT-2 comes in four available sizes, only three of which have been publicly made available. Feared for its fake news generation capabilities, it currently stands as the most syntactically coherent model. A direct successor to the original GPT, it reinforces the already established pre-training / fine-tuning killer duo. From the paper: Language Models are Unsupervised Multitask Learners by Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever.

💯 XLNet

Overcoming the unidirectional limit while maintaining an independent masking algorithm based on permutation, XLNet improves upon the state-of-the-art autoregressive model that is TransformerXL. Using a bidirectional context while keeping its autoregressive approach, this model outperforms BERT on 20 tasks while keeping an impressive generative coherence. From the paper: XLNet: Generalized Autoregressive Pretraining for Language Understanding, by Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov and Quoc V. Le.

☠️ GPT

Released by OpenAI, this architecture has shown that large gains on several NLP tasks can be achieved by generative pre-training a language model on unlabeled text before fine-tuning it on a downstream task. From the paper: Improving Language Understanding by Generative Pre-Training, by Alec Radford, Karthik Naraimhan, Tim Salimans and Ilya Sutskever.

Do you want to contribute or suggest a new model checkpoint? Open an issue onpytorch-transformers🔥.

“It is to write what calculators are to calculus.”

Brave Browser
Read More

What do you think?

Leave a Reply

Your email address will not be published.

GIPHY App Key not set. Please check settings

This Vegan ETF Is the Most Woke Millennial Thing Ever –, Crypto Coins News

This Vegan ETF Is the Most Woke Millennial Thing Ever –, Crypto Coins News

Postgres Makes Transactions Atomic, Hacker News

Postgres Makes Transactions Atomic, Hacker News