Modernizing Federal Data Infrastructure for AI Readiness

By Dia Adams, TCG Data Program Lead

Artificial intelligence is swiftly changing how federal agencies deliver on their missions, improve services, and increase operational efficiency, but as agencies set their sights on ambitious AI goals, a significant obstacle still remains.  True readiness for AI depends on modern, robust data infrastructure. Without this foundation, even the most advanced models and algorithms will struggle to deliver meaningful results, limiting the impact of AI initiatives.

The Need for AI-Ready Infrastructure

Federal implementation of artificial intelligence continues to grow, with significant resources dedicated to both AI-specific projects and the data infrastructure that supports them. For instance, in 2023, the federal government disclosed 710 AI use cases, a figure that surged to 1,757 in 2024, underscoring the accelerating pace of experimentation within the public sector.1  As an illustration of investment growth, federal AI spending reached approximately $3.3 billion in FY22. The FY25 budget request for total IT R&D and AI funding (unclassified) reflects this continuing trend, with total AI funding (core and cross-cutting) expected to be $3.316 billion.2 In addition, key agencies such as the National Science Foundation (NSF) and the National Institutes of Health (NIH) are slated for significant increases in their AI-related budgets. For instance, the NSF’s AI-focused research budget is set to rise from $400 million to $494 million.3

Given AI’s reliance on data, the future of government operations will depend on the ability to work with data at scale. However, many agencies still face a gap between their AI ambitions and the reality of their current data environments. This gap is not just a technical issue, it’s also a strategic must have. Agencies that focus on upgrading their data systems now will be best positioned to benefit from AI in the future.

Challenges in Federal Data Modernization

Modernizing data infrastructure within federal agencies is a complex task. Agencies commonly encounter four main challenges:

  1. Fragmented Data Systems

Several federal agencies still rely on siloed, legacy storage solutions.4

A GAO report highlighted this issue by identifying 10 critical legacy systems across federal agencies, some of which are over 50 years old—such as Internal Revenue Service (IRS) applications exceeding 60 years in age and still reliant on programming languages like COBOL. The annual maintenance cost for these 10 systems alone amounts to $337 million.5

This fragmentation makes it difficult to access and use data efficiently, slows down projects, and increases the risk of security breaches. Moving data between disconnected systems can create bottlenecks and expose sensitive information.

  1. Lack of Strategic Alignment

When agencies do not have a clear, organization-wide strategy for AI, modernization efforts can become scattered and reactive. Without clear priorities, progress slows and resources may be wasted.

  1. Security and Compliance Risks

As federal requirements change, agencies must ensure that their data infrastructure meets strict standards for security, privacy, and auditability. Older systems often lack the controls needed to support AI in critical environments.

  1. Barriers to Cross-Agency Data Sharing

Federal agencies often struggle with sharing data across organizational boundaries.4 Challenges include inconsistent data management practices, varying access policies, and mistrust among agencies regarding data privacy and security. Agencies may lack awareness of what data is available elsewhere or how it is classified, leading to missed opportunities for collaboration and improved service delivery. Additionally, the process of establishing data sharing agreements is often slow and complex, further hindering timely and efficient data exchange. Overcoming these barriers requires investment in interoperable systems, standardized data formats, and strong data governance frameworks.

Federal Mandates and the New AI Roadmap 

The federal government has recognized these challenges and is pushing agencies to accelerate AI adoption through innovation and governance.6
Agencies are now required to appoint leaders responsible for AI strategy, develop comprehensive plans, and implement risk management practices for high-impact AI use cases.7

As of October 2024, progress in appointing these key leaders was evident, with 80 out of 266 federal agencies (30%) having publicly disclosed their CAIOs. Among Chief Financial Officers (CFO) Act agencies, the adoption rate was significantly higher, at 94%.8 Investments in high-performance computing, cloud platforms, and agile development pipelines are also  now seen as essential steps.

Building Blocks of an AI-Ready Data Infrastructure

Agencies aiming to prepare for AI can take several important steps to create a secure, scalable, and innovative data foundation:

  1. Centralize and Standardize Data Assets

Bringing data together in unified, well-managed environments reduces fragmentation and improves accessibility.9  Adopting common standards and investing in metadata management ensures that data is high-quality and ready for advanced analytics.

  1. Prioritize Network Scalability and Performance

AI workloads require networks that offer high bandwidth and low latency.10   Upgrading to faster connectivity, integrating edge computing, and adopting modern networking technologies support real-time applications and large-scale data processing.

  1. Embed Robust Security and Compliance

AI systems often process sensitive or mission-critical data. Agencies must implement advanced security protocols, including encryption, intrusion detection, and continuous auditing, to protect information and maintain public trust.

  1. Foster Interoperability and Open Competition

To avoid being locked into a single vendor and to maintain flexibility, agencies should choose solutions that work well with others and support open data architectures. This approach allows for easy integration of new tools and encourages innovation from a wide range of technology providers.

  1. Invest in Workforce and Governance

Modern infrastructure is only as effective as the people and processes behind it. Agencies should focus on upskilling current staff, recruiting experts in AI and data, and establishing clear governance frameworks to oversee data quality, ethics, and responsible use of AI.11 

The Role of Collaboration and Shared Innovation

Modernizing federal data infrastructure is not something agencies can do in isolation. Many are now working together with contracting teams to implement best practices and share resources. This collaborative approach not only speeds up progress but also brings in fresh ideas and solutions from both established federal staff and innovative contracting companies.

Looking Forward

The adoption of AI in government is no longer a far off goal, but a current reality. Agencies that prioritize data infrastructure modernization today will lay the groundwork for scalable, secure, and innovative AI solutions that deliver real value for citizens and the U.S. as a whole. Federal leaders who act with strategic purpose, aligning their strategies, investing in modern platforms, and encouraging a data-driven culture of continuous improvement, will stand out in the next era of government.

Notes

  1. Wirtschafter, V. (2025, January 16). For AI to make government work better, reduce risk and increase transparency. Brookings. https://www.brookings.edu/articles/for-ai-to-make-government-work-better-reduce-risk-and-increase-transparency/ [↩]
  2. Holihan, M (2025, January 7). Federal AI and IT Research and Development spending analysis. Federal Budget IQ. https://federalbudgetiq.com/insights/federal-ai-and-it-research-and-development-spending-analysis/ [↩]
  3. Federal Budget IQ. (2025, January 7). Federal AI and IT research and development spending analysis. Retrieved June 5, 2025, from https://federalbudgetiq.com/insights/federal-ai-and-it-research-and-development-spending-analysis/ [↩]
  4. The White House. (2025, April 15). Presidential memorandum on updating permitting technology for the 21st century. https://www.whitehouse.gov/presidential-actions/2025/04/updating-permitting-technology-for-the-21st-century/ [↩] [↩]
  5. Agencies Need to Continue Addressing Critical Legacy Systems, accessed June 5, 2025, https://www.mlogica.com/resources/blogs/agencies-need-to-continue-addressing-critical-legacy-systems [↩]
  6. The White House. (2023, October 30). Executive Order 14110: Safe, secure, and trustworthy development and use of artificial intelligence. https://bidenwhitehouse.archives.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/ [↩]
  7. The White House. (2025, January 25). Executive Order 14117: Removing barriers to American leadership in artificial intelligence. https://www.whitehouse.gov/presidential-actions/2025/01/removing-barriers-to-american-leadership-in-artificial-intelligence/ [↩]
  8. Stanford Institute for Human-Centered Artificial Intelligence. (2025, January). Assessing the implementation of federal AI leadership and compliance mandates. https://hai.stanford.edu/policy/assessing-the-implementation-of-federal-ai-leadership-and-compliance-mandates [↩]
  9.  Khatri, V., & Brown, C. V. (2010). Designing data governance. Communications of the ACM, 53(1), 148–152. https://doi.org/10.1145/1629175.1629210 [↩]
  10. Kawka, T. (2025, April 16). How AI changes your network infrastructure requirements. Equinix Blog. https://blog.equinix.com/blog/2025/04/16/how-ai-changes-your-network-infrastructure-requirements/ [↩]
  11. MeriTalk. (2024, June 27). Future-proofing federal IT: A blueprint for AI workforce readiness. https://www.meritalk.com/study/blueprint-for-ai-workforce-readiness/ [↩]