Expert Perspectives on Developing Secure and Trustworthy AI Frameworks

By: Dr. Charles Vardeman, Dr. Christ Sweet and Dr. Paul Brenner

In line with President Biden’s recent decision executive order Emphasizing safe and trustworthy ai, we share our Trustworthy ai (TAI) lessons learned two years after the course of our research projects. This research initiative, visualized in the figure below, focuses on operationalizing ai that meets rigorous ethical and performance standards. It aligns with a growing industry trend toward transparency and accountability in ai systems, particularly in sensitive areas such as national security. This article reflects on the shift from traditional software engineering to ai approaches where trust is paramount.

Transition from “Software 1.0 to 2.0 and 3.0″ notions need a trusted infrastructure that not only conceptualizes but also practically enforces trust in ai. Even a simple set of example ML components, like the one shown in the figure below, demonstrates the important complexity that must be understood to address trust concerns at every level. Our TAI Frameworks subproject addresses this need by providing an integration point for software and best practices from TAI research products. Frameworks like these reduce barriers to TAI implementation. By automating configuration, developers and decision makers can channel their efforts toward innovation and strategy, rather than dealing with upfront complexities. This ensures that trust is not an afterthought but a prerequisite, and that each phase, from data management to model implementation, is inherently aligned with ethical and operational standards. The result is a simplified path to deploying ai systems that are not only technologically advanced but also ethically sound and strategically reliable for high-risk environments. The TAI Frameworks project examines and leverages existing software tools and best practices that have their own sustainable open source communities and can be leveraged directly within existing operating environments.

GitOps has become an integral part of ai engineering, especially in the TAI framework. It represents an evolution in the way software development and operational workflows are managed, offering a declarative approach to application and infrastructure lifecycle management. This methodology is essential to ensure continuous quality and incorporate ethical responsibility into ai systems. The TAI Frameworks project leverages GitOps as a critical component to automate and streamline the development process, from code to deployment. This approach ensures that software engineering best practices are automatically followed, enabling an immutable audit trail, a version-controlled environment, and seamless rollback capabilities. Simplifies complex deployment processes. Additionally, GitOps facilitates the integration of ethical considerations by providing a structure where ethical controls can be automated as part of the CI/CD process. Adopting CI/CD in ai development is not just about maintaining code quality; it’s about ensuring that ai systems are reliable, secure, and work as expected. TAI promotes automated testing protocols that address the unique challenges of ai, particularly as we enter the era of generative ai and prompt-based systems. Testing is no longer limited to static code analysis and unit testing. It extends to the dynamic validation of ai behaviors, covering the results of generative models and the effectiveness of indications. Automated test suites must now be able to evaluate not only the accuracy of responses, but also their relevance and security.

In the pursuit of TAI, a data-centric approach is critical as it prioritizes data quality and clarity over the complexities of algorithms, thereby establishing trust and interpretability from the ground up. Within this framework, a range of tools are available to maintain data integrity and traceability. dvc (data versioning) is particularly favored for its congruence with the GitOps framework, enhancing Git to encompass data management and experiments (see alternatives ai/blog/best-data-version-control-tools” rel=”noopener” target=”_blank”>here). It facilitates accurate version control for datasets and models, just as Git does for code, which is essential for effective CI/CD practices. This ensures that the data engines that drive ai systems are constantly fed with accurate and auditable data, a prerequisite for trustworthy ai. We take advantage ai/” rel=”noopener” target=”_blank”>nbdev which complements dvc by turning Jupyter Notebooks into a medium for literate programming and exploratory programming, streamlining the transition from exploratory analysis to well-documented code. The nature of software development is evolving towards this style of “programming” and is only accelerating with the evolution of ai “co-pilots” who assist in documenting and building ai applications. Software Bill of Materials (SBoM) and BoM of ai, championed by open standards such as SPDX, are an integral part of this ecosystem. They serve as detailed records that complement dvc and nbdev, summarizing the provenance, composition, and compliance of ai models. SBoMs provide a complete list of components, ensuring that every element of the ai system is considered and verified. ai BoMs expand this concept to include data sources and transformation processes, offering a level of transparency to the models and data in an ai application. Together, they form a complete picture of an ai system’s lineage, promoting trust and facilitating understanding between stakeholders.

Ethical and data-centric approaches are fundamental to TAI, ensuring that ai systems are effective and trustworthy. Our TAI frameworks project leverages tools like dvc for data versioning and nbdev for literate programming, reflecting a shift in software engineering that adapts to the nuances of ai. These tools are emblematic of a larger trend toward integrating data quality, transparency, and ethical considerations from the beginning of the ai development process. In both the civil and defense sectors, the principles of TAI remain constant: a system is only as reliable as the data on which it is based and the ethical framework to which it adheres. As the complexity of ai increases, so does the need for robust frameworks that can handle this complexity transparently and ethically. The future of ai, particularly in mission-critical applications, will depend on adopting these ethical and data-centric approaches, solidifying trust in ai systems across domains.

About the authors

Charles Vardeman, Christ Sweet and Paul Brenner are research scientists at the University of Notre Dame Computing Research Center. They have decades of experience developing scientific software and algorithms with a focus on applied research for technology transfer to product operations. They have numerous technical papers, patents, and funded research activities in the fields of data science and cyberinfrastructure. Weekly TAI nuggets can be found aligned with student research projects here.

Dr. Charles Vardeman is a research scientist at the University of Notre Dame Computing Research Center.