A mathematical framework is developed for the analysis of causal fermion systems in the infinite-dimensional setting. It is shown that the regular spacetime point operators form a Banach manifold endowed with a canonical Frechet-smooth Riemannian metric. The so-called expedient differential calculus is introduced with the purpose of treating derivatives of functions on Banach spaces which are differentiable only in certain directions. A chain rule is proven for Holder continuous functions which are differentiable on expedient subspaces. These results are made applicable to causal fermion systems by proving that the causal Lagrangian is Holder continuous. Moreover, Holder continuity is analyzed for the integrated causal Lagrangian.