In March, the secretary of protection directed the Division of Protection (DoD) to undertake the Software program Acquisition Pathway (SWP) to speed up the event and deployment of capabilities to the warfighter. The directive to default to the SWP arrives at a time when DoD missions more and more depend on software program and the associated applied sciences of cybersecurity and synthetic intelligence (AI)—all of that are focus areas on the SEI. These technical areas develop in significance as nationwide safety and protection organizations require enhanced capabilities to guard a broader vary of targets in opposition to extra refined and adept threats.
On this submit, I’ll spotlight the methods by which our analysis and growth assist DoD’s use of contemporary software program practices at every section of the software program growth and operation lifecycle.
The SEI and the Software program Acquisition Pathway
The SEI’s depth of expertise with data-driven strategies, strategies, and approaches; software program engineering; and acquisition science catalyzed our work on the SWP.
Part 255 of the FY2020 Nationwide Protection Authorization Act (NDAA) known as on the DoD to orient its software program actions in analysis, growth, testing, and acquisition towards trendy software program engineering practices described in two research that SEI additionally contributed to: the Protection Innovation Board’s 2019 Software program Acquisition and Practices (SWAP) and the Protection Science Board Job Drive’s 2018 Design and Acquisition of Software program for Protection Programs.
As a part of our SWP work, the SEI labored hand-in-hand with stakeholders throughout the DoD and the protection industrial base within the iterative growth, testing, and updating of the coverage and supporting instruments and assets. As known as for within the coverage doc DoDI 5000.87, DoD applications are to emphasise risk-based cybersecurity all through the lifecycle, counting on trendy software program practices together with DevSecOps. Cybersecurity and DevSecOps kind a basis on which these applications can incorporate new AI applied sciences extra securely.
For the reason that SEI’s inception as a federally funded analysis and growth heart (FFRDC), we now have labored with academia, authorities, and trade to conduct analysis and assist DoD applications apply these important instruments, strategies, practices, and insurance policies. The SEI prioritizes growth of strategies to assist steady, resilient, and well timed deployment of software program functionality for the warfighter whereas making certain that software program system efficiency and safety will not be compromised in mission-critical settings.
SEI Impression on Modernizing DoD Software program Growth
Over the past decade, the DoD has been incorporating ideas and practices that promote steady, iterative deployment of software program functionality. One such initiative was the event of the 250-plus member Agile Collaboration Group. Members share classes realized to assist DoD practitioners extra readily profit from utilizing Agile strategies of their larger-scale techniques. When the group started in 2012, it targeted on overcoming adoption challenges round Agile. In 2017, it expanded into DevSecOps analysis, growth, and discipline engagement.
In 2024 the SEI performed a research analyzing the state of DevSecOps within the DoD, the outcomes of which had been not too long ago launched by the DoD Chief Data Officer (CIO). The research discovered that whereas sure applications have had success adopting DevSecOps practices, the DoD nonetheless must implement these successes at scale. The research holds partially that
- Investing in DoD software program factories is vital to securing our future functionality.
- DevSecOps is a key technique to speed up supply time.
- Success rests on reimagining a mission-ready DevSecOps workforce.
- Sturdy management dedicated to creatively driving options is crucial to overcoming limitations.
The DevSecOps research additionally acknowledges that it’s essential to align these practices with the mission.
To a big diploma, the efficient use of contemporary software program practices rests on recognizing the strategic worth of information that’s now out there to the DoD in more and more huge quantities. To present DoD analysts larger visibility into DevSecOps pipeline information, the SEI not too long ago launched Polar, an answer to the restrictions of conventional batch information processing. Polar provides visibility into the present state of a corporation’s DevSecOps infrastructure, permitting for everything of the information to be engaged for knowledgeable choice making. The Polar framework, which might be downloaded from the SEI’s GitHub web site, helps DevSecOps organizations monitor and acquire insights into safety facets and deal with the challenges posed by constructing advanced software program techniques in extremely regulated environments.
DoD program leaders should more and more handle all the software program growth functionality. This accountability typically means dealing successfully with the technical debt that may accumulate in ageing techniques in addition to that which may accumulate in fast, iterative growth. The SEI has been a pioneer in growing and making use of technical debt administration practices in advanced techniques for nationwide safety and protection. Starting in 2010, the SEI challenged the software program engineering analysis neighborhood to seek out methods to handle technical debt and convened annual workshops on the subject. These workshops produced case research, empirical outcomes from making use of strategies, and comparisons of instruments shared by the SEI and the software program neighborhood in a whole bunch of publications within the Affiliation for Computing Equipment (ACM) and IEEE digital libraries. In 2018, the SEI’s neighborhood efforts resulted within the first worldwide convention on the topic, TechDebt. The eighth TechDebt convention was held in April 2025.
The SEI additionally led by way of ground-breaking analysis on the subject. An early paper on the subject, In Search of a Metric for Managing Architectural Debt, authored by SEI and College of British Columbia researchers, obtained the Most Influential Paper Award for its lasting impression on software program structure analysis and observe on the 2022 IEEE Worldwide Convention on Software program Structure. The SEI’s experience in technical debt R&D is the rationale that the DoD commissioned a staff of our software program growth consultants to put in writing a report addressing the NDAA 2022 Part 835 mandate, delivered to Congress in December 2023.
The SEI has used its function to allow the DoD’s efficient use of contemporary software program engineering practices in different methods, as effectively, together with instruments to
SEI Impression on Modernizing DoD Software program in Operation
A core tenet of the SEI’s mission is to allow the DoD to quickly deploy resilient software program capabilities. To do that, they want the fitting tooling to make use of trendy software program practices and the means to guarantee system efficiency.
To facilitate DevSecOps use with large-scale techniques, the SEI created the Platform-Unbiased Mannequin (PIM)—out there on the SEI’s GitHub web site—to explain a DevSecOps pipeline on the highest stage: necessities, the product growth lifecycle course of, and the organizational roles wanted to supply software program. Since its launch, a cross-disciplinary SEI staff has enhanced the PIM by incorporating risk situations: assault sort, actors, results, and pipeline belongings for cover. The ensuing upgraded instruments can be utilized to create safer processes and pipelines or spot safety weaknesses in current ones. When the pipeline is safer, so too is the software program it produces.
To enhance software program threat evaluation, SEI researchers and power builders not too long ago launched an open-source software that streamlines and automates high quality assurance testing and evaluation, Silent Sentinel. This software gives a repeatable, constant course of to offer system stakeholders a sensible evaluation of how an utility will have an effect on their deployment surroundings.
Steady supply of software program functionality additionally signifies that techniques utilizing these capabilities want steady assurance of security, safety, and different qualities. In ongoing work, the SEI is searching for to cut back the effort and time required to re-assure giant techniques. This notion of system assurance extends past safety to embody a number of architecturally important considerations, together with efficiency, modifiability, security, and reliability.
SEI Impression on AI System Assurance
The world, and positively the DoD’s expertise surroundings, is turning into more and more AI-augmented. When AI techniques for nationwide safety fail in growth or operation, they trigger critical, real-world penalties. Sadly, there are few accepted greatest practices for testing AI techniques as a result of challenges of correctly defining necessities and evaluating standards.
In 2023, the Workplace of the Underneath Secretary of Protection for Analysis and Engineering (OUSD(R&E)) and the SEI launched a middle aimed toward establishing strategies for assuring trustworthiness in AI techniques with emphasis on interplay between people and autonomous techniques. The Heart for Calibrated Belief Measurement and Analysis (CaTE) goals to assist the DoD be sure that AI techniques are protected, dependable, and reliable earlier than being fielded to operators in crucial conditions.
Additional, as detailed in a latest weblog submit and podcast, a gaggle of SEI software program and AI consultants not too long ago launched Machine Studying Take a look at and Analysis (MLTE), a brand new course of and power collectively developed by the SEI and the Military AI Integration Heart (AI2C) to create safer, extra dependable ML techniques. MLTE addresses three issues widespread within the ML mannequin growth course of which can be limitations to efficient take a look at and analysis processes.
- Communication limitations between product growth staff members. Crew members are sometimes siloed throughout organizations, resulting in issues in gathering ML mannequin necessities cognizant of the system context and speaking ML mannequin analysis outcomes.
- Documentation issues for ML mannequin necessities. Eliciting and documenting ML mannequin necessities is commonly a problem for organizations, and documentation for ML system necessities is commonly lacking or of low high quality.
- Requirement analysis. Even when necessities are correctly outlined and documented, there isn’t a ML-specific methodology to assist their implementation, testing, and analysis.
Broadening our perspective to AI threat administration, we explored the best way to conceptualize trendy AI threat administration frameworks (RMFs) analogous to these for cyber threat. This work illustrates the broad scope of challenges that AI Engineering practices should deal with, together with software program engineering and cybersecurity issues. A latest SEI weblog submit famous this:
We should take into account, in different phrases, the conduct of a system or an related workflow below each anticipated and surprising inputs, the place these inputs could also be significantly problematic for the system. It’s difficult, nonetheless, even to border the query of the best way to specify behaviors for anticipated inputs that aren’t precisely matched within the coaching set. A human observer could have an intuitive notion of similarity of recent inputs with coaching inputs, however there isn’t a assurance that this aligns with the precise that includes—the salient parameter values—inner to a skilled neural community.
SEI analysis groups additionally noticed the necessity for an AI safety response staff analogous to pc safety response. An knowledgeable and motivated attacker could intentionally manipulate operational inputs, coaching information, and different facets of the system growth course of to create circumstances that impair right operation of an AI system. To deal with this want, the SEI launched the first-of-its sort AI Safety Incident Response Crew (AISIRT).
To guarantee that future AI techniques might be sturdy, safe, scalable, and able to serving warfighter wants, the SEI has been main the initiative to advance the self-discipline of AI Engineering. This emergent self-discipline will allow practitioners to focus R&D efforts in AI on growing instruments, techniques, and processes for nationwide safety contexts.
SEI Impression on Software program Acquisition Safety
Just about all services {that a} DoD program acquires are supported by or combine with info expertise that features third-party parts or providers. Practices crucial to monitoring and managing these dangers might be scattered, leading to inconsistencies, gaps, and sluggish response to disruptions. To deal with these points, SEI researchers created the Acquisition Safety Framework (ASF), which gives the DoD with a roadmap for constructing safety and resilience right into a system fairly than bolting them on after deployment. The ASF promotes higher communication and data sharing throughout all program and provider groups to coordinate their administration of engineering and provide chain dangers. On this approach, the ASF helps applications match threats in a dynamic surroundings with the fast evolution of wanted software program capabilities.
As well as, in early 2020 the SEI partnered with Johns Hopkins College Utilized Physics Laboratory (APL), a college affiliated analysis heart, to launch the preliminary model of the cybersecurity maturity mannequin on the coronary heart of the Cybersecurity Maturity Mannequin Certification (CMMC) program. CMMC gives the DoD Workplace of the Underneath Secretary of Protection for Acquisition and Sustainment (OUSD(A&S)) with a strong software to enhance risk-informed choices and contractor safety within the protection industrial base provide chain.
SEI Impression on the Way forward for Software program Engineering Analysis
An vital a part of the SEI mission is to anticipate each challenges and alternatives in its three mission areas: software program engineering, cybersecurity, and AI. To develop an agenda for the subsequent decade of software program engineering analysis, the SEI introduced collectively an advisory board of visionaries and senior thought leaders to develop an agenda for the subsequent decade of software program engineering analysis. This effort led to the 2021 publication, Architecting the Way forward for Software program Engineering: A Nationwide Agenda for Software program Engineering Analysis and Growth. The research is a catalyst for analysis and growth at Carnegie Mellon College and the SEI in areas akin to AI-augmented software program growth, the assurance of constantly evolving software program techniques, and engineering AI-enabled software program techniques.
That SEI-led research is influencing the DoD software program ecosystem and scary follow-on actions. As an illustration, in 2023 the SEI and the Networking and Data Know-how Analysis and Growth (NITRD) program organized and hosted the U.S. Management in Software program Engineering and AI Engineering workshop; see the workshop’s govt abstract. As well as, we labored with the Nationwide Protection Industrial Affiliation’s Rising Applied sciences Institute (NDIA ETI) to supply suggestions for the DoD because it shapes its software program modernization actions and analysis portfolio. Additional, the SEI is partnering with the software program engineering and AI communities to implement the suggestions of the analysis agenda. The SEI partnered with the Federal Aviation Administration and Vanderbilt College to convene two workshops in 2024 to handle the reassurance of constantly evolving software program techniques, one of many research’s areas of focus. SEI researchers together with leaders from the software program engineering neighborhood might be main a workshop to handle how AI may remodel end-to-end software program growth workflows within the 2025 Worldwide Workshop on Envisioning the AI-Augmented Software program Growth Lifecycle, collocated with the ACM Worldwide Convention on the Foundations on Software program Engineering.
The SEI: Persistent Give attention to Software program Modernization
By means of the SEI Weblog and our podcast collection, we spotlight the work of our researchers to assist the DoD make software program a strategic benefit by way of integrating our area experience in AI, cybersecurity, and software program. To assist the DoD ship resilient software program functionality on the pace of relevance, the SEI researches advanced engineering issues; creates, prototypes, and refines revolutionary applied sciences; and transitions maturing options into observe to advertise DoD mission success.
