How do growing bacterial colonies get their shapes? While colony morphogenesis is well studied in two dimensions, many bacteria grow as large colonies in three-dimensional (3D) environments, such as gels and tissues in the body or subsurface soils and sediments. Here, we describe the morphodynamics of large colonies of bacteria growing in three dimensions. Using experiments in transparent 3D granular hydrogel matrices, we show that dense colonies of four different species of bacteria generically become morphologically unstable and roughen as they consume nutrients and grow beyond a critical size-eventually adopting a characteristic branched, broccoli-like morphology independent of variations in the cell type and environmental conditions. This behavior reflects a key difference between two-dimensional (2D) and 3D colonies; while a 2D colony may access the nutrients needed for growth from the third dimension, a 3D colony inevitably becomes nutrient limited in its interior, driving a transition to unstable growth at its surface. We elucidate the onset of the instability using linear stability analysis and numerical simulations of a continuum model that treats the colony as an "active fluid" whose dynamics are driven by nutrient-dependent cellular growth. We find that when all dimensions of the colony substantially exceed the nutrient penetration length, nutrient-limited growth drives a 3D morphological instability that recapitulates essential features of the experimental observations. Our work thus provides a framework to predict and control the organization of growing colonies-as well as other forms of growing active matter, such as tumors and engineered living materials-in 3D environments.