When two dimensional crystals are atomically close, their finite thickness becomes relevant. Using transport measurements, we investigate the electrostatics of two graphene layers, twisted by θ = 22° such that the layers are decoupled by the huge momentum mismatch between the K and K′ points of the two layers. We observe a splitting of the zero-density lines of the two layers with increasing interlayer energy difference. This splitting is given by the ratio of single-layer quantum capacitance over interlayer capacitance Cm and is therefore suited to extract Cm. We explain the large observed value of Cm by considering the finite dielectric thickness dg of each graphene layer and determine dg ≈ 2.6 Å. In a second experiment, we map out the entire density range with a Fabry-Pérot resonator. We can precisely measure the Fermi wavelength λ in each layer, showing that the layers are decoupled. Our findings are reproduced using tight-binding calculations.