Although basketball is a dualistic sport, with all players competing on both offense and defense, almost all of the sports conventional metrics are designed to summarize offensive play. As a result, player valuations are largely based on offensive performances and to a much lesser degree on defensive ones. Steals, blocks, and defensive rebounds provide only a limited summary of defensive effectiveness, yet they persist because they summarize salient events that are easy to observe. Due to the inefficacy of traditional defensive statistics, the state of the art in defensive analytics remains qualitative, based on expert intuition and analysis that can be prone to human biases and imprecision. Fortunately, emerging optical player tracking systems have the potential to enable a richer quantitative characterization of basketball performance, particularly defensive performance. Unfortunately, due to computational and methodological complexities, that potential remains unmet. This paper attempts to fill this void, combining spatial and spatio-temporal processes, matrix factorization techniques, and hierarchical regression models with player tracking data to advance the state of defensive analytics in the NBA. Our approach detects, characterizes, and quantifies multiple aspects of defensive play in basketball, supporting some common understandings of defensive effectiveness, challenging others, and opening up many new insights into the defensive elements of basketball.