Publications

Papers     Technical reports     Google Scholar     Patents     Home


2023

Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs

S. Passaro, C. L. Zitnick

ICML, 2023 (arXiv:2302.03655)

Deep Learning Reconstruction Enables Prospectively Accelerated Clinical Knee MRI

P. Johnson, D. Lin, J. Zbontar, C. L. Zitnick, A. Sriram, M. Muckley, J. Babb, M. Kline, G. Ciavarra, E. Alaia, M. Samim, W. Walter, L. Calderon, T. Pock, D. Sodickson, M. Recht, F. Knoll

Radiology, 2023

The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysis

R. Tran, J. Lan, M. Shuaibi, B. Wood, S. Goyal, A. Das, J. Heras-Domingo, A. Kolluru, A. Rizvi, N. Shoghi, A. Sriram, Z. Ulissi, C. L. Zitnick

ACS Catalysis, 2023 (arXiv:2206.08917)

AdsorbML: A Leap in Efficiency for Adsorption Energy Calculations Using Generalizable Machine Learning Potentials

J. Lan, A. Palizhati, M. Shuaibi, B. Wood, B. Wander, A. Das, M. Uyttendaele, C. L. Zitnick, Z. Ulissi

Nature Computational Materials, 2023 (arXiv:2211.16486)

From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction

N. Shoghi, A. Kolluru, J. Kitchin, Z. Ulissi, C. L. Zitnick, B. Wood

arXiv:2310.16802, 2023


2022

Spherical Channels for Modeling Atomic Interactions

C. Lawrence Zitnick, A. Das, A. Kolluru, J. Lan, M. Shuaibi, A. Sriram, Z. Ulissi, B. Wood

NeurIPS, 2022 (arXiv:2206.14331)

Open Challenges in Developing Generalizable Large Scale Machine Learning Models for Catalyst Discovery

A. Kolluru, M. Shuaibi, A. Palizhati, N. Shoghi, A. Das, B. Wood, C. L. Zitnick, J. Kitchin, Z. Ulissi

ACS Catalysis, 2022 (arXiv:2206.02005)

GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets

J. Gasteiger, M. Shuaibi, A. Sriram, S. Günnemann, Z. Ulissi, C L. Zitnick, A. Das

TMLR, 2022 (arXiv:2204.02782)

Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations

A. Sriram, A. Das, B. Wood, S. Goyal, C. L. Zitnick

ICLR, 2022 (arXiv:2203.09697)

Transfer learning using attentions across atomic systems with graph neural networks (TAAG)

A. Kolluru, N. Shoghi, M. Shuaibi, S. Goyal, A. Das, C. L. Zitnick, Z. Ulissi

Journal of Chemical Physics, 2022


2021

Rotation Invariant Graph Neural Networks using Spin Convolutions

M. Shuaibi, A. Kolluru, A. Das, A. Grover, A. Sriram, Z. Ulissi, C. L. Zitnick

arXiv:2106.09575, 2021

ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations

W. Hu, M. Shuaibi, A. Das, S. Goyal, A. Sriram, J. Leskovec, D. Parikh, C. L. Zitnick

arXiv:2103.01436, 2021

Compositional Transformers for Scene Generation

D. Hudson, C. L. Zitnick

NeurIPS, 2021 (arXiv:2111.08960)

Generative Adversarial Transformers

D. Hudson, C. L. Zitnick

ICML, 2021 (arXiv:2103.01209)

Creative Sketch Generation

S. Ge, V. Goswami, C. L. Zitnick, D. Parikh

ICLR, 2021 (arXiv:2011.10039)

The Open Catalyst 2020 (OC20) Dataset and Community Challenges

L. Chanussot, A. Das, S. Goyal, T. Lavril, M. Shuaibi, M. Riviere, K. Tran, J. Heras-Domingo, C. Ho, W. Hu, A. Palizhati, A. Sriram, B. Wood, J. Yoon, D. Parikh, C. L. Zitnick, Z. Ulissi

ACS Catalysis, 2021 (arXiv:2010.09990)

Biological Structure and Function Emerge from Scaling Unsupervised Learning to 250 Million Protein Sequences

A. Rives, S. Goyal, J. Meier, D. Guo, M. Ott, C. L. Zitnick, J. Ma, R. Fergus

PNAS, 2021 (bioRxiv:622803)


2020

An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage

C. L. Zitnick, L. Chanussot, A. Das, S. Goyal, J. Heras-Domingo, C. Ho, W. Hu, T. Lavril, A. Palizhati, M. Riviere, M. Shuaibi, A. Sriram, K. Tran, B. Wood, J. Yoon, D. Parikh, Z. Ulissi

arXiv:2010.09435, 2020

Using Deep Learning to Accelerate Knee MRI at 3T: Results of an Interchangeability Study

M. Recht, J. Zbontar, D. Sodickson, F. Knoll, N. Yakubova, A. Sriram, T. Murrell, A. Defazio, M. Rabbat, L. Rybak, M. Kline, G. Ciavarra, E. Alaia, M. Samim, W. Walter, D. Lin, Y. Lui, M. Muckley, Z. Huang, P. Johnson, R. Stern, C. L. Zitnick

American Journal of Roentgenology (AJR), 2020 (arXiv paper describing the approach)

Exploring Crowd Co-creation Scenarios for Sketches

D. Parikh, C. L. Zitnick

International Conference on Computational Creativity (ICCC), 2020 (arXiv:2005.07328)

End-to-End Variational Networks for Accelerated MRI Reconstruction

A. Sriram, J. Zbontar, T. Murrell, A. Defazio, C. L. Zitnick, N. Yakubova, F. Knoll, P. Johnson

MICCAI, 2020 (arXiv:2004.06688)

Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge

F. Knoll, T. Murrell, A. Sriram, N. Yakubova, J. Zbontar, M. Rabbat, A. Defazio, M. Muckley, D. Sodickson, C. L. Zitnick, M. Recht

Magnetic Resonance in Medicine, 2020 (arXiv:2001.02518)

GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction

A. Sriram, J. Zbontar, T. Murrell, C. L. Zitnick, A. Defazio, D. Sodickson

CVPR, 2020 (arXiv:1910.12325)

fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning

F. Knoll, J. Zbontar, A. Sriram, M. Muckley, M. Bruno, A. Defazio, M. Parente, K. Geras, J. Katsnelson, H. Chandarana, Z. Zhang, M. Drozdzal, A. Romero, M. Rabbat, P. Vincent and J. Pinkerton, D. Wang, N. Yakubova, E. Owens, C. L. Zitnick, M. Recht, D. Sodickson, Y. Lui

Radiology Artificial Intelligence, 2020


2019

Order-Aware Generative Modeling Using the 3D-Craft Dataset

Z. Chen, D. Guo, T. Xiao, S. Xie, X. Chen, H. Yu, J. Gray, K. Srinet, H. Fan, J. Ma, C. R. Qi, S. Tulsiani, A. Szlam, C. L. Zitnick

ICCV, 2019

CraftAssist: A Framework for Dialogue-enabled Interactive Agents

J. Gray, K. Srinet, Y. Jernite, H. Yu, Z. Chen, D. Guo, S. Goyal, C. L. Zitnick, A. Szlam

arXiv:1907.08584, 2019

ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

Y. Tian, J. Ma, Q. Gong, S. Sengupta, Z. Chen, J. Pinkerton, C. L. Zitnick

ICML, 2019 (arXiv:1902.04522)


2018

fastMRI: An Open Dataset and Benchmarks for Accelerated MRI

J. Zbontar, F. Knoll, A. Sriram, M. Muckley, M. Bruno, A. Defazio, M. Parente, K. Geras, J. Katsnelson, H. Chandarana, Z. Zhang, M. Drozdzal, A. Romero, M. Rabbat, P. Vincent and J. Pinkerton, D. Wang, N. Yakubova, E. Owens, C. L. Zitnick, M. Recht, D. Sodickson, Y. Lui

arXiv:1811.08839, 2018


2017

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

Y. Tian, Q. Gong, W. Shang, Y. Wu, C. L. Zitnick

NIPS, 2017. (arXiv)

Learn2Smile: Learning Non-Verbal Interaction Through Observation

W. Feng, A. Kannan, G. Gkioxari, C. L. Zitnick

IROS, 2017.

Inferring and Executing Programs for Visual Reasoning

J. Johnson, B. Hariharan, L. van der Maaten, J. Hoffman, L. Fei-Fei, C. L. Zitnick, R. Girshick

ICCV, 2017. (arXiv)

CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, R. Girshick

CVPR, 2017. (arXiv)


2016

Measuring Machine Intelligence Through Visual Question Answering

C. L. Zitnick, A. Agrawal, S. Antol, M. Mitchell, D. Batra, D. Parikh

AI Magazine, 2016. (arXiv)

Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?

A. Das, H. Agrawal, C. L. Zitnick, D. Parikh, D. Batra

EMNLP, 2016. (arXiv)

Shuffle and Learn: Unsupervised Learning using Temporal Order Verification

I. Misra, C. L. Zitnick, M. Hebert

ECCV, 2016. (arXiv)

Visual Storytelling

T. H. Huang, F. Ferraro, N. Mostafazadeh, I. Misra, A. Agrawal, J. Devlin, R. Girshick, X. He, P. Kohli, D. Batra, C. L. Zitnick, D. Parikh, L. Vanderwende, M. Galley, M. Mitchell

NAACL, 2016. (arXiv)

Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels

I. Misra, C. L. Zitnick, M. Mitchell, R. Girshick

CVPR, 2016. (arXiv)

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks

S. Bell, C. L. Zitnick, K. Bala, R. Girshick

CVPR, 2016. (arXiv)

We Are Humor Beings: Understanding and Predicting Visual Humor

A. Chandrasekaran, A. Kalyan, S. Antol, M. Bansal, D. Batra, C. L. Zitnick, D. Parikh

CVPR, 2016. (arXiv)

Reducing Overfitting in Deep Networks by Decorrelating Representations

M. Cogswell, F. Ahmed, R. Girshick, C. L. Zitnick, D. Batra

ICLR, 2016. (arXiv)

Adopting Abstract Images for Semantic Scene Understanding

C. L. Zitnick, R. Vedantam and D. Parikh

PAMI, 2016. (webpage)


2015

VQA: Visual Question Answering

S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, D. Parikh

ICCV, 2015. (webpage, arXiv)

Learning Common Sense Through Visual Abstraction

R. Vedantam, X. Lin, T. Batra, C. L. Zitnick, D. Parikh

ICCV, 2015. (webpage)

VISALOGY: Answering Visual Analogy Questions

F. Sadeghi, C. L. Zitnick, A. Farhadi

NIPS, 2015. (arXiv)

Mind's Eye: A Recurrent Visual Representation for Image Caption Generation

X. Chen and C.L. Zitnick

CVPR, 2015. (arXiv)

From Captions to Visual Concepts and Back

H. Fang, S. Gupta, F. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. Platt, C.L. Zitnick, and G. Zweig

CVPR, 2015. (arXiv, code)

CIDEr: Consensus-based Image Description Evaluation

R. Vedantam, C. L. Zitnick, and D. Parikh

CVPR, 2015. (arXiv, webpage)

Fast Edge Detection Using Structured Forests

P. Dollar and C. L. Zitnick

PAMI, 2015. (code)

Exploring Nearest Neighbor Approaches for Image Captioning

J. Devlin, S. Gupta, R. Girshick, M. Mitchell, C. L. Zitnick

arXiv:1505.04467 , 2015.

Microsoft COCO Captions: Data Collection and Evaluation Server

X. Chen, H. Fang, T.Y. Lin, R. Vedantam, S. Gupta, P. Dollar, C. L. Zitnick

arXiv:1504.00325 , 2015.(MS COCO caption challenge)


2014

Microsoft COCO: Common Objects in Context

T.Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick

ECCV, 2014. (webpage, arXiv)

Edge Boxes: Locating Object Proposals from Edges

C. L. Zitnick and Piotr Dollar

ECCV, 2014. (code)

Zero-Shot Learning via Visual Abstraction

S. Antol, C. L. Zitnick, and D. Parikh

ECCV, 2014. (webpage)

Predicting Object Dynamics in Scenes

D. Fouhey and C. L. Zitnick

CVPR, 2014. (webpage, supp. mat.)

Detecting Objects using Deformation Dictionaries

B. Hariharan, C. L. Zitnick, and P. Dollar

CVPR, 2014.


2013

Structured Forests for Fast Edge Detection

P. Dollar and C. L. Zitnick

ICCV, 2013. (code, supp. mat.)

Learning the Visual Interpretation of Sentences

C. L. Zitnick, D. Parikh, and L. Vanderwende

ICCV, 2013.  (webpage, dataset, supp. mat.)

Handwriting Beautification Using Token Means

C. L. Zitnick

SIGGRAPH, 2013. (video, YouTube, talk slides, handwriting data)

Bringing Semantics Into Focus Using Visual Abstraction

C. L. Zitnick and D. Parikh

CVPR, 2013. (webpage, dataset, talk video, talk slides)

Sketch-Tokens: A Learned Mid-level Representation for Contour and Object Detection

J. Lim, C. L. Zitnick, and P. Dollar

CVPR, 2013. (code)

Exploring Weak Stabilization for Motion Feature Extraction

D. Park, C. L. Zitnick, D. Ramanan, and P. Dollar

CVPR, 2013.

Which Edges Matter?

A. Bansal, A. Kowdle, D. Parikh, A. C. Gallagher, and C. L. Zitnick

Workshop on 3D Representation and Recognition (3dRR), ICCV, 2013.


2012

Exploring the Spatial Hierarchy of Mixture Models for Human Pose Estimation

Y. Tian, C. L. Zitnick, and S. Narasimhan

ECCV, 2012. (webpage)

The Role of Image Understanding in Contour Detection

C. L. Zitnick and D. Parikh

CVPR, 2012. (Additional thoughts)

Seeing through the Blur

H. Mobahi, C. L. Zitnick, and Y. Ma

CVPR, 2012. (webpage)

Exploring Tiny Images: The Roles of Appearance and Contextual Information for Machine and Human Object Recognition

D. Parikh, C. L. Zitnick, and T. Chen

PAMI, 34(10):1978-1991, 2012.

Image Restoration by Matching Gradient Distributions

T. Cho, C. L. Zitnick, N. Joshi, S. B. Kang, R. Szeliski, and W. Freeman

PAMI, 34(4):683-694, 2012.

A Memory Efficient Discriminative approach for Location aided Recognition

V. Hedau, S. Sinha, C. L. Zitnick, and R. Szeliski

Workshop on Visual Analysis and Geo-Localization of Large-Scale Imagery, ECCV, 2012.


2011

Edge Foci Interest Points

C. L. Zitnick, and K. Ramnath

ICCV, 2011. (webpage, executable)

ShadowDraw: Real-Time User Guidance for Freehand Drawing

Y. J. Lee, C. L. Zitnick, and M. Cohen

SIGGRAPH, 2011. (video, YouTube, webpage)

Finding the Weakest Link in Person Detectors

D. Parikh and C. L. Zitnick

CVPR, 2011. (webpage)

A Viewer-Centric Editor for Stereoscopic Cinema

S. J. Koppal, C. L. Zitnick, M.F. Cohen, S.B. Kang, B. Ressler, and A. Colburn

IEEE Computer Graphics and Applications, 2011. (webpage)


2010

C. L. Zitnick

ECCV, 2010. (webpage, executable)

Binary Coherent Edge Descriptors

A. Gupta, N. Joshi, C. L. Zitnick, M. Cohen, B. Curless

ECCV, 2010.  (webpage)

Single Image Deblurring using Motion Density Functions

N. Joshi, S.B. Kang, C.L. Zitnick, and R. Szeliski

SIGGRAPH, 2010.  (webpage)

Image deblurring with inertial measurement sensors

D. Parikh, and C. L. Zitnick

CVPR, 2010.

The Role of Features, Algorithms and Data in Visual Recognition

T. Cho, N Joshi, C. L. Zitnick S. B. Kang, B. Freeman, and R. Szeliski

CVPR, 2010. (webpage)

A Content-Aware Image Prior

P. Bhat, C. L. Zitnick, M. Cohen, B. Curless

ACM Transactions on Graphics, 2010. (webpage)

GradientShop: A Gradient-Domain Optimization Framework for Image and Video Filtering


2009

D. Parikh, C. L. Zitnick, and T. Chen

CVPR, 2009.

Unsupervised Learning of Hierarchical Spatial Structures in Images 

N. Joshi, C. L. Zitnick, R. Szeliski, and D. Kriegman

CVPR, 2009.

Image Deblurring and Denoising using Color Priors

F. Schroff, C. L. Zitnick, and S. Baker

BMVC, 2009.

Clustering Videos by Location


2008

D. Parikh, C. L. Zitnick, and T. Chen

ECCV, 2008.

Determining Patch Saliency Using Low-Level Context

P. Bhat, B. Curless, M. Cohen and C. L. Zitnick

ECCV, 2008.

Fourier Analysis of the 2D Screened Poisson Equation for Gradient Domain Problems

Y. Taguchi, B. Wilburn and C. L. Zitnick

CVPR, 2008.

 Stereo Reconstruction with Mixed Pixels Using Adaptive Over-Segmentation

D. Parikh, C. L. Zitnick, and T. Chen

CVPR, 2008.

From Appearance to Context-Based Recognition: Dense Labeling in Small Images

C. Liu, R. Szeliski, S.B. Kang, C.L. Zitnick, and W.T. Freeman

PAMI 30(2):299-314, 2008.

Automatic estimation and removal of noise from a single image

G. Schindler, C. L. Zitnick, M. Brown

IEEE Workshop on Internet Vision (CVPR), 2008.

Internet Video Category Recognition


2007

P. Bhat, C. L. Zitnick, N. Snavely, A. Agarwala, M. Agarwala, B. Curless, M. Cohen, S. B. Kang

Eurographics Symposium on Rendering (EGSR), 2007. (webpage)

Using Photographs to Enhance Videos of a Static Scene

C. L. Zitnick and S. B. Kang

IJCV 75(1):49-65, 2007.

Stereo for image-based rendering using image over-segmentation


2006

J. Sivic, C. L. Zitnick and R. Szeliski

BMVC, 2006.

Finding people in repeated shots of the same scene

V. Vaish, R. Szeliski, C. L. Zitnick, S.B. Kang, and M. Levoy

CVPR, 2006.

Reconstructing occluded surfaces using synthetic apertures: Shape from focus vs.

shape from stereo

N. Jojic, J. Winn, C. L. Zitnick

CVPR, 2006.

Escaping local minima through hierarchical model selection: Automatic object discovery, segmentation, and tracking in video

N. Snavely, C. L. Zitnick, S. B. Kang, and M. Cohen

Int’l Symp. on Non-Photorealistic Animation and Rendering (NPAR), 2006.  (video)

Stylizing 2.5-D video

E. Bennett, M. Uyttendaele, C. L. Zitnick, R. Szeliski, and S. B. Kang

ECCV, 2006.

Video and Image Bayesian Demosaicing With A Two Color Image Prior


2005 and earlier

C. L. Zitnick, N. Jojic, S. B. Kang

ICCV, 2005. (webpage)

Consistent segmentation for optical flow estimation

C. L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski

SIGGRAPH 2004.  (video, data, webpage)

High-quality video view interpolation using a layered representation.

C. L. Zitnick, T. Kanade

Conference on Uncertainty in Artificial Intelligence (UAI), 2004.

Maximum Entropy for Collaborative Filtering

C. L. Zitnick, Thesis 2003.

 

Computing Conditional Probabilities in Large Domains by Maximizing Renyi's Quadratic Entropy

J. Gemmell, C. L. Zitnick, T. Kang, K. Toyama and Steven Seitz 

IEEE MultiMedia, pp. 26-35, 2000.

Gaze-awareness for Videoconferencing: A Software Approach

C. L. Zitnick and T. Kanade

PAMI 22(7), 2000. (webpage)

A Cooperative Algorithm for Stereo Matching and Occlusion Detection

S. B. Kang, J. Webb, C. L. Zitnick, and T. Kanade

ICCV, 1995.

An Active multibaseline stereo system with real-time image acquisition


Videos

J. Berry, C. L. Zitnick

Siggraph Animation Theater, 2004

Massive Arabesque


Technical reports

A Framework for Encoding Object-level Image Priors
J. Yuen, C. L. Zitnick, C. Liu, A. Torralba
Tech. Report MSR-TR-2011-99, Microsoft Research, 2011

 Color Source Separation for Enhanced Pixel Manipulations
C. L. Zitnick, D. Parikh
Tech. Report MSR-TR-2011-98, Microsoft Research, 2011

Detecting Objects using Unsupervised Parts-based Attributes
S. Divvala, C. L. Zitnick, A. Kapoor, and S. Baker
Tech. Report CMU-RI-TR-11-10, Robotics Institute, Carnegie Mellon University, 2010

Local Bi-gram Model for Object Recognition
Xiangyang Lan, C. L. Zitnick, Richard Szeliski
Tech. Report MSR-TR-2007-54, Microsoft Research, 2007

Object instance recognition using triplets of feature symbols
C. L. Zitnick, Jie Sun, Richard Szeliski, Simon Winder
Tech. Report MSR-TR-2007-53, Microsoft Research, 2007 (webpage)

Manipulation of Video Eye Gaze and Head Orientation for Video Teleconferencing
C. L. Zitnick; Jim Gemmell; Kentaro Toyama
tech
. report MSR-TR-99-46, Microsoft Research, 1999

A Cooperative Algorithm for Stereo Matching and Occlusion Detection
C. L. Zitnick and T. Kanade
tech. report CMU-RI-TR-99-35, Robotics Institute, Carnegie Mellon University, October, 1999.

A Volumetric Iterative Approach to Stereo Matching and Occlusion Detection
C. L. Zitnick and T. Kanade
tech. report CMU-RI-TR-98-30, Robotics Institute, Carnegie Mellon University, December, 1998.

Multi-baseline Stereo Using Surface Extraction
C. L. Zitnick and J.A. Webb
tech. report CMU-CS-96-196, Computer Science Department, Carnegie Mellon University, 1996.

An Active Multibaseline Stereo System with Real-Time Image Acquisition
S. Kang, J. Webb, C. L. Zitnick, and T. Kanade
tech. report CMU-CS-94-167, Computer Science Department, Carnegie Mellon University,


Unpublished

Content-free Image Retrieval
C. L. Zitnick and T. Kanade, 2003