Software is becoming increasingly embedded into our lives, and the potential cost of a failure is greater than ever. At the same time, as systems become more complex, our ability to predict and control their behavior seems to be lagging behind. Testing large, complex software remains a challenge, and achieving a formal proof of correctness is too costly to be applied to an entire system. Are there ways to design systems to be safe and secure from early on, and reduce the cost of validation?
In this talk, I will argue that building a secure system requires a particular mindset that is fundamentally different from how we typically construct complex software. I will then present two approaches to aid developers in designing a system with a security mindset. First, I will describe a technique called multi-representational security analysis, which can be used to anticipate and address security attacks that exploit details across multiple abstraction layers of a system. Second, I will present a framework for reasoning about the overall safety of a system when some of its components may be compromised, and demonstrate an application to an analysis of an industrial water treatment plant. Finally, I share some future directions for designing safe and secure systems that operate in an increasingly open, evolving environment.