The van-der-Waals stacking technique enables the fabrication of heterostructures, where two conducting layers are atomically close. In this case, the finite layer thickness matters for the interlayer electrostatic coupling. Here we investigate the electrostatic coupling of two graphene layers, twisted by 22 degrees such that the layers are decoupled by the huge momentum mismatch between the K and K' points of the two layers. We observe a splitting of the zero-density lines of the two layers with increasing interlayer energy difference. This splitting is given by the ratio of single-layer quantum capacitance over interlayer capacitance C and is therefore suited to extract C. We explain the large observed value of C by considering the finite dielectric thickness d of each graphene layer and determine d=2.6 Angstrom. In a second experiment we map out the entire density range with a Fabry-Pérot resonator. We can precisely measure the Fermi-wavelength in each layer, showing that the layers are decoupled. We find that the Fermi wavelength exceeds 600nm at the lowest densities and can differ by an order of magnitude between the upper and lower layer. These findings are reproduced using tight-binding calculations.