Current methods of characterizing quantum devices focus on diagnosing individual components. While important, this falls short of the primary goal, namely, determining when the total noise in a quantum circuit is sufficiently small, so that the output of a quantum computer is reliable. In this work, we first obtain an optimal bound on the total fidelity of a circuit in terms of the component fidelities, which can be efficiently estimated by randomized benchmarking. This optimal bound grows quadratically in the length of the circuit. We then obtain an improved bound using additional information about the noise, namely, the unitarity, which can also be efficiently estimated. This improved bound smoothly interpolates between the worst-case quadratic and best-case linear scalings. We then show that our analysis substantially improves the bounds on the infidelities of individual gates obtained using interleaved randomized benchmarking.