Skip to the content.

CtrlGen is a NeurIPS 2021 workshop taking place virtually on Monday, December 13, 2021. It aims to explore disentanglement, controllability, and manipulation for the generative vision and language modalities. We feature an exciting lineup of speakers, a live QA and panel session, interactive activities, and networking opportunities.


Call for Papers and Demonstrations

Please see our call for papers and call for demonstrations pages for details.

Important Dates

Note that the above deadlines are all 11:59pm AOE.

Workshop Description

Over the past few years, there has been an increased interest in the areas of language and image generation within the ML, NLP, and Vision communities. As generated texts by models like GPT-3 start to sound more fluid and natural, and generated images and videos by GAN models appear more realistic, researchers began focusing on qualitative properties of the generated content such as the ability to control its style and structure, or incorporate information from external sources and other texts or images into the output. Such aims are extremely important to make language and image generation useful for human-machine interaction and other real-world applications including machine co-creativity, entertainment, ethical purposes, enhanced training for self-driving vehicles, and improving the ability of conversational agents and personal assistants to effectively interact.

Achieving these ambitious but important goals introduces challenges not only from NLP and Vision perspectives, but ones that closely pertain to Machine Learning as a whole, which has concurrently witnessed a growing body of work in relevant domains such as interpretability, disentanglement, robustness, and representation learning. We believe progress towards the realization of human-like language and image generation may benefit greatly from insights in these and other ML areas.

In this workshop, we propose to bring together researchers from the NLP, Vision, and ML communities to discuss the current challenges and explore potential directions for controllable generation and improve its quality, correctness, and diversity. As excitement about language and image generation has significantly increased recently thanks to the advent and improvement of large language models, transformer networks, and GANs, we feel this is the opportune time to hold a new workshop about this subject. We hope CtrlGen will foster discussion and interaction across NLP, Vision, and ML communities, including areas that range from structured prediction to disentanglement, and sprout fruitful cross-domain relations that may open the door for enhanced controllability in language and image generation.

Schedule (Times in PST)

8:00 - 8:10

8:10 - 9:00

9:00 - 9:50

9:50 - 10:10

10:10 - 11:30

11:30 - 12:30

12:30 - 13:30

13:30 - 13:50

13:50 - 14:40

14:40 - 15:00

15:00 - 16:00

16:00 - 16:20

16:20 - 16:45

16:45 - 17:00

17:00 onward

Opening Remarks

First & Second Invited Talks (+QA)

Third & Fourth Invited Talks (+QA)

Virtual Coffee/Networking Break

Panel and QA Session

Virtual Poster Session #1

Lunch Break

Demonstrations / Interactive Activity

Fifth & Sixth Invited Talks (+QA)

Virtual Coffee/Networking Break

Virtual Poster Session #2

Demonstrations / Interactive Activity

Seventh Invited Talk (+QA)

Best Paper Awards and Closing Remarks

GatherTown Open (for continued socializing)

Invited Speakers and Panelists

Yejin Choi
University of Washington

Yulia Tsvetkov
Carnegie Mellon University

Jason Weston
Facebook AI

Irina Higgins

He He
New York University

Or Patashnik
Tel-Aviv University

Alex Tamkin
Stanford University

Additional Panelists

Angela Fan
LORIA and Facebook AI


Tatsunori Hashimoto
Stanford University

Joel Tetreault
Dataminr, Inc.

Steven Y. Feng
Carnegie Mellon University

Dongyeop Kang
UC Berkeley

Anusha Balakrishnan
Microsoft Semantic Machines

Varun Gangal
Carnegie Mellon University

Drew Hudson
Stanford University