We examine a two-level system coupled to a quantum oscillator, typically representing experiments in cavity and circuit quantum electrodynamics. We show how such a system can be treated analytically in the ultrastrong coupling limit, where the ratio g/Omega between coupling strength and oscillator frequency approaches unity and goes beyond. In this regime the Jaynes-Cummings model is known to fail because counter-rotating terms have to be taken into account. By using Van Vleck perturbation theory to higher orders in the qubit tunneling matrix element Delta we are able to enlarge the regime of applicability of existing analytical treatments, including, in particular, also the finite-bias case. We present a detailed discussion on the energy spectrum of the system and on the dynamics of the qubit for an oscillator at low temperature. We consider the coupling strength g to all orders, and the validity of our approach is even enhanced in the ultrastrong coupling regime. Looking at the Fourier spectrum of the population difference, we find that many frequencies contribute to the dynamics. They are gathered into groups whose spacing depends on the qubit-oscillator detuning. Furthermore, the dynamics is not governed anymore by a vacuum Rabi splitting which scales linearly with g, but by a nontrivial dressing of the tunneling matrix element, which can be used to suppress specific frequencies through a variation of the coupling.