In highly open systems like the Internet, attributebased access control (ABAC) has proven its appropriateness. This is reflected in the utilization of ABAC in authentication and authorization infrastructures (AAIs). However, specification and maintenance of ABAC policies has turned out to be complex and error-prone even in federations of limited size, especially if heterogeneous attribute schemes are involved. Here, the arising Semantic Web can contribute to a solution. This paper describes an architecture for embedding the access control process into a semantic context employing external knowledge in form of ontologies. We base our proposal on extensions of established open standards. Using the approach presented, policy management at the different sites of a federation is simplified by a semantic attribute management facility.