The Australian Taxation Office is preparing to pilot an enterprise-grade AI coding assistant for its roughly 800 core developers, a move that could reshape how government software is produced — from legacy COBOL modernization to automated test generation — while raising familiar questions about security, governance and vendor lock‑in.
The ATO’s request for tender describes a Software-as-a-Service solution intended to sit inside developers’ workflows and free engineers from repetitive tasks so they can concentrate on “higher value work” such as test-case planning, application security controls and maintaining existing systems. The tender explicitly requires deep integration with Microsoft development tools — Visual Studio 2019 and 2022, and Visual Studio Code — plus connections to Azure DevOps and Git repositories. It also asks for capabilities to translate legacy languages like COBOL into modern languages, and sets a strict privacy requirement that any code processed by the tool must not be stored or used to train the underlying model.
This procurement sits within a broader ATO strategy to embed AI across the agency. A recent independent audit and the ATO’s own disclosures show the agency already operates dozens of AI models and is experimenting with large multimodal models for tasks such as document understanding and auditing taxpayer-submitted material. The ATO has been building governance structures but remains in the process of formalising enterprise-wide AI controls, monitoring and reporting.
However, success will depend on execution: conservative pilots, verifiable privacy safeguards, realistic expectations about legacy translation, and strengthened governance and monitoring. When deployed with these guardrails, an AI assistant can be a force multiplier — but without them, the technology risks introducing new operational and compliance liabilities. The ATO’s challenge — and opportunity — is to turn vendor capability into durable, auditable, and secure developer productivity gains that the public service can responsibly own and maintain. (anao.gov.au, community.ibm.com)
Source: iTnews ATO considers AI coding assistance for 800 core developers
Background
The ATO’s request for tender describes a Software-as-a-Service solution intended to sit inside developers’ workflows and free engineers from repetitive tasks so they can concentrate on “higher value work” such as test-case planning, application security controls and maintaining existing systems. The tender explicitly requires deep integration with Microsoft development tools — Visual Studio 2019 and 2022, and Visual Studio Code — plus connections to Azure DevOps and Git repositories. It also asks for capabilities to translate legacy languages like COBOL into modern languages, and sets a strict privacy requirement that any code processed by the tool must not be stored or used to train the underlying model. This procurement sits within a broader ATO strategy to embed AI across the agency. A recent independent audit and the ATO’s own disclosures show the agency already operates dozens of AI models and is experimenting with large multimodal models for tasks such as document understanding and auditing taxpayer-submitted material. The ATO has been building governance structures but remains in the process of formalising enterprise-wide AI controls, monitoring and reporting.
Why this matters: scale, legacy, and compliance
The ATO’s environment is a textbook example of complexity: hundreds of developers, multi-technology stacks, multiple code repositories and legacy mainframe assets that still include COBOL. Deploying an AI assistant here is not the same as installing a plugin in a single team’s IDE; it is an enterprise integration project with implications across code quality, security, procurement and staff capability.- Scale — 800 core developers means any productivity gains or mistakes will be amplified across high‑impact production systems.
- Legacy code — COBOL and other legacy languages require specialist handling. Tools that claim to “translate” COBOL to Java or C# often deliver scaffolding that still needs human validation and systems‑level understanding. Independent comparisons show specialised mainframe tools tend to outperform generic coding assistants for full fidelity COBOL migrations.
- Compliance and privacy — government data (and code that potentially contains PII or operational secrets) cannot be processed by third-party models in ways that create regulatory or legal exposure. The ATO’s tender requires that code processed by the assistant is not used to train the model — a non‑negotiable point in public‑sector procurement. (cyberdaily.au, anao.gov.au)
What the ATO is asking for — functional checklist
The tender specifies a practical set of features the ATO expects from a candidate solution:- Real‑time code suggestions and completions in Visual Studio 2019/2022 and VS Code.
- Bug detection and suggested fixes, including the ability to propose and apply refactorings across multi‑tech stacks.
- Automated test case and script generation, to accelerate unit and integration testing.
- Integration with Azure DevOps pipelines and Git repositories for pull request reviews, CI feedback, and traceability.
- Capability to assist with legacy code translation (COBOL → modern languages), while preserving business logic and integration patterns. (cyberdaily.au, community.ibm.com)
- Data handling and privacy controls: processed code must not be stored or used to train the vendor’s models; logging and telemetry must comply with government requirements.
Technical reality check: integration and limitations
IDE integration and workspace context
Modern coding assistants can integrate with Visual Studio and VS Code to provide inline suggestions, workspace‑wide context and chat-like query interfaces. Vendor documentation and platform APIs support such extensions, but enterprise integrations carry extra requirements:- Secure authentication and conditional access (SSO, managed identities).
- Local indexing or on‑prem proxies for repository context to avoid uploading proprietary code to third‑party clouds.
- Tight lifecycle controls so that any AI‑suggested change still flows through existing code review and CI pipelines. (code.visualstudio.com, visualstudio.microsoft.com)
Legacy translation is not “lift and shift”
Automatic COBOL translations have improved, but vendor comparisons show outcomes vary greatly. Generic LLM‑based assistants can generate syntactically valid modern code but often miss system integration details — database access patterns, CICS/TPX interactions, nuanced error handling and stateful mainframe services. Purpose‑built mainframe conversion tools and human subject matter experts remain necessary to validate business logic and produce production‑ready artifacts. The ATO’s tender requirement for translation capability is realistic, but it should be scoped as assisted modernization rather than a fully automated migration.Security and supply chain constraints
The ATO’s insistence that code processed by the assistant not be retained for model training echoes a wider industry trend: public bodies and regulated enterprises demand “no training” or “in‑private model hosting” options. Vendors respond with isolated model instances, on‑premise or within approved cloud tenancy, and contractual commitments not to use customer data for model improvements. However, guarantees depend on verifiable technical measures (no persistent storage, ephemeral compute, audited logging) and contract law — both of which must be scrutinised in procurement. (cyberdaily.au, anao.gov.au)Organisational impacts: productivity, skills and governance
Productivity — promise and caveats
Private-sector pilots and academic studies repeatedly show AI assistants can lift developer productivity — with numbers varying by context, team maturity and measurement methodology. Reported gains range from modest improvements to high single‑digit or even double‑digit percentages in certain controlled experiments, but such figures rarely transfer 1:1 into complex, regulated environments. The ATO should assume measurable improvements are possible, particularly for routine tasks (boilerplate, tests, refactors), but should also expect an initial learning and stabilization period.Skills and role evolution
Rather than replacing developers, AI assistants will shift their work: more architecture, security reviews, test strategy and stakeholder engagement; less boilerplate typing. This raises two priority workforce actions:- Reskilling and upskilling — training on safe AI usage, prompt engineering, and validating AI outputs.
- Role redesign — clarifying responsibilities for final code ownership, security sign‑off and testing obligations.
Governance — lifecycle controls and auditing
The ATO’s audit record shows it is building AI governance but still maturing its monitoring and deployment oversight. Integrating an AI coding assistant accentuates the need for:- Formal approval gates for new AI tools.
- Clear ownership for AI‑produced artefacts.
- Continuous monitoring to detect performance drift, hallucination rates, and security alerts.
- Audit trails linking AI suggestions to PRs, reviewers and test outcomes.
Vendor selection risks and procurement pitfalls
When buying an AI coding assistant at this scale, a few procurement pitfalls are common:- Vendor lock-in: Choosing a deeply integrated platform (e.g., a vendor‑hosted Copilot tied to a single cloud) can make future migration costly.
- Opaque SLAs: Uptime and response-time SLAs may not cover model degradation or “hallucination” performance.
- IP and copyright exposure: Depending on the vendor model, generated code may carry licensing implications if the model was trained on public repositories.
- False positives/negatives in security scanning: AI can propose fixes that appear correct but introduce subtle vulnerabilities.
Practical recommendations for the ATO (and similar agencies)
The ATO’s stated objectives are sensible: accelerate routine work, free senior developers for strategic activities, and modernise legacy assets. To balance value and risk, the following sequence reduces exposure and builds evidence:- Start with a tight pilot: a subset of teams, non‑production repositories, and synthetic sensitive data. Measure impact on cycle time, defect rates, and developer satisfaction.
- Require private model hosting or on‑prem proxies and contractual no‑training clauses, then validate these with audits and penetration testing.
- Enforce CI/CD guardrails: AI suggestions can create PRs but must not be auto‑merged; integrate static analysis and security scanners as mandatory gates.
- Treat legacy translation as an assisted workflow: combine AI scaffolding with SME validation and automated equivalence testing for migrated components.
- Build human-in‑the‑loop standards for code ownership, review responsibilities, and incident escalation for AI‑introduced defects.
- Establish continuous monitoring: track hallucination rates, suggestion acceptance, and security flags; feed findings into procurement and governance cycles. (code.visualstudio.com, anao.gov.au)
What success looks like — measurable indicators
A realistic performance dashboard for an AI coding assistant should include:- Average time to close a ticket or feature before and after adoption.
- PR review cycle time and the proportion of AI‑suggested lines accepted.
- Regression and security test pass rates on AI‑generated code.
- Developer satisfaction and self-reported productivity.
- Number and severity of production incidents tied to AI‑influenced changes.
Broader policy implications: AI in government software delivery
The ATO’s procurement mirrors what other governments are considering: a move from manual, labour‑intensive development to AI‑assisted software engineering. This creates policy questions:- How should public agencies standardise vendor obligations around data residency, training bans and audit rights?
- What regulatory frameworks are required for AI‑produced code in safety‑critical or legally consequential systems?
- How will procurement rules evolve to judge not just price and function, but model governance and verifiability?
Strengths and weaknesses of the ATO’s stated approach
Strengths
- Clear scope: The tender specifies exact IDEs (Visual Studio 2019/2022, VS Code), DevOps platforms (Azure DevOps, Git), and realistic use cases (tests, refactors, COBOL translation), making supplier responses comparable.
- Privacy-first requirement: Mandating that processed code won’t be used to train models addresses a major organisational risk early in procurement.
- Alignment with existing AI activity: The ATO already uses and audits AI broadly, so adding a developer assistant fits an incremental adoption path.
Weaknesses and risks
- Overreliance on vendor claims for COBOL translation: Unless the tender enforces demonstration projects with representative legacy code, the agency risks paying for superficial translations that require heavy remediation.
- Governance gaps: ANAO findings show the ATO is still maturing monitoring and deployment controls; integrating a high‑impact tool without tightened governance could create blind spots.
- Vendor lock‑in and cost dynamics: Deep integrations with Microsoft or a single cloud vendor have benefits, but the ATO should weigh them against long‑term cost and portability risks.
The vendor landscape — who can meet these needs?
The large cloud vendors (Microsoft, AWS, IBM) and specialist firms offer competing approaches:- Microsoft’s Copilot/IntelliCode ecosystem ties closely to Visual Studio and GitHub, with enterprise options for private telemetry and compliance controls. Integration is strong, but deep embedding increases dependency on Microsoft’s stack. (visualstudio.microsoft.com, code.visualstudio.com)
- AWS CodeWhisperer supports Visual Studio integration and emphasises security controls and code scanning, with different hosting models available.
- IBM’s watsonx Code Assistant for Z and similar specialised products are tailored for mainframe modernization and have demonstrated more faithful COBOL→Java conversions in comparative testing.
- Boutique vendors and open‑source assisted offerings exist for code review, automated PR feedback and test generation, and may offer flexible deployment models that better meet strict public‑sector privacy requirements.
Conclusion
The ATO’s procurement to supply an AI coding assistant for its 800 core developers is a forward‑looking step that aligns with a global trend: government organisations leveraging generative AI to reduce rote work and accelerate delivery. The tender’s focus on Visual Studio/VS Code, Azure DevOps, COBOL translation and strict no‑training rules shows an appreciation of both developer workflows and governance constraints. (cyberdaily.au, australiantenders.com.au)However, success will depend on execution: conservative pilots, verifiable privacy safeguards, realistic expectations about legacy translation, and strengthened governance and monitoring. When deployed with these guardrails, an AI assistant can be a force multiplier — but without them, the technology risks introducing new operational and compliance liabilities. The ATO’s challenge — and opportunity — is to turn vendor capability into durable, auditable, and secure developer productivity gains that the public service can responsibly own and maintain. (anao.gov.au, community.ibm.com)
Source: iTnews ATO considers AI coding assistance for 800 core developers
