Causal fermion systems and Riemannian fermion systems are proposed as a framework for describing non-smooth geometries. In particular, this framework provides a setting for spinors on singular spaces. The underlying topological structures are introduced and analyzed. The connection to the spin condition in differential topology is worked out. The constructions are illustrated by many simple examples such as the Euclidean plane, the two-dimensional Minkowski space, a conical singularity, a lattice system as well as the curvature singularity of the Schwarzschild space-time. As further examples, it is shown how complex and Kahler structures can be encoded in Riemannian fermion systems.