Published research
Autonomous AI and legacy modernization: claims, bottlenecks, and economics
Summary
Autonomous coding agents can materially improve bounded, repeatable modernization, but public evidence does not establish autonomous end-to-end replacement of complex enterprise systems. Cognition’s Mercedes-Benz announcement reports analysis of more than 200,000 COBOL lines and an estimated eight-month timeline reduced to eight days; it does not disclose the production-delivery milestones needed to verify that scope. (cognition.com) AWS now addresses requirements extraction and functional-test preparation as well as conversion, while retaining human and operational responsibilities. (aws.amazon.com) The report’s illustrative economics show that dramatic conversion savings can become much smaller total-project savings. Real ROI depends on independent validation, acceptance timing, target operating costs, and actual retirement of avoidable legacy spending—not lines converted.
Autonomous AI and legacy modernization: claims, bottlenecks, and economics
Review period: August 10, 2018–October 13, 2026 Scope: Verification of AWS and Cognition claims and end-to-end modernization economics across COBOL, Java, SAP, .NET, and other enterprise platforms. Source policy: Primary customer/vendor publications and documentation, original research, and reputable institutional analysis; vendor estimates and Revoqra-derived scenarios are labeled separately.
Executive summary
Autonomous coding agents can materially improve modernization economics, but the evidence does not establish autonomous, end-to-end replacement of complex enterprise systems. Their strongest applications are bounded changes with clear requirements, repeatable transformations, executable feedback, and independently verifiable outputs. Cognition’s own evaluation identifies this pattern—and acknowledges difficulty with ambiguous projects. (cognition.com)
The eight-month-to-eight-day Mercedes-Benz claim is genuine as a published vendor claim, but its delivery scope remains unverified. Cognition reports analysis of more than 200,000 lines of COBOL during a pilot; the announcement does not establish that the entire application was converted, integrated, migrated, accepted, and deployed into production in eight days. Mercedes-Benz’s customer statement confirms the partnership and broader development-cycle acceleration, not those specific delivery milestones. (cognition.com)
AWS’s offering extends beyond conversion into requirements extraction, functional-test preparation, and deployment tooling. That matters economically, because validation—not typing replacement code—can dominate modernization effort. Nevertheless, AWS’s documented workflows retain human review and operational responsibilities. (aws.amazon.com)
The decisive financial question is whether agents reduce the cost of achieving accepted production behavior and retiring avoidable legacy expenses. In the illustrative model below, a 75% reduction in conversion labor produces a 25% reduction in the total project budget—not a 75% reduction. That saving can make a bounded migration attractive, while a partial mainframe exit can still have negative five-year returns.
1. What the headline claims actually establish
1.1 Mercedes-Benz: eight months to eight days
Cognition’s April 27, 2026 announcement describes a four-week pilot in which Devin analyzed more than 200,000 lines of COBOL and reduced an estimated eight-month modernization timeline to eight days. Its September 15 AWS collaboration announcement repeats the result. Neither publication provides a work-package breakdown or production acceptance record. (cognition.com)
The distinction is important: the baseline is a forecast, not an observed eight-month execution of the same project by a control team. The published material therefore supports a reported acceleration, not a controlled measurement of end-to-end productivity. (cognition.com)
| Delivery milestone | What the reviewed announcements establish |
|---|---|
| Source analysis | More than 200,000 COBOL lines analyzed, according to Cognition. |
| Replacement code | Target language, converted volume, and delivered application boundaries are not disclosed. |
| Testing | Test inventory, coverage, independent holdout results, and defect counts are not disclosed. |
| Integration | Completion of external interfaces, schedulers, middleware, and operational integrations is not established. |
| Data migration | Migrated records, reconciliation results, and synchronization procedures are not disclosed. |
| Production deployment | No application-specific go-live date, cutover acceptance, or stabilization results are provided. |
| Legacy retirement | No corresponding mainframe capacity or contract retirement is established. |
Evidence boundary: these are disclosure gaps in the reviewed publications, not proof that the work was not performed. The appropriate conclusion is insufficient public evidence to verify the eight days as a complete production modernization. (cognition.com)
Mercedes-Benz’s own April 28 statement reports development-cycle acceleration of up to 25%, with professional responsibility and decisions remaining with development teams. This is a broader, differently defined metric; it neither independently validates nor contradicts the exceptional COBOL pilot result. (group.mercedes-benz.com)
1.2 A separate automaker: 25,000 lines, AWS Lambda, 73% lower migration cost
This must not be conflated with Mercedes-Benz. Cognition’s AWS announcement explicitly identifies another global automaker for the 25,000-line COBOL-to-Lambda project. (cognition.com)
The detailed account describes a batch customs workflow: supplied input/output examples, a Python replacement, iterative comparisons, and refined migration playbooks. Cognition says completion took months, with an estimated 73% reduction in migration costs. It does not publish the accounting ledger or production acceptance package. (cognition.com)
This is more informative than the eight-day announcement because it identifies the workload, target, feedback mechanism, and iteration process. But the 73% figure is a migration-cost estimate—not an established reduction in the automaker’s ongoing infrastructure bill.
1.3 Cognition’s strongest boundary statement
Cognition’s April 8, 2026 COBOL article explicitly says autonomous migration of transactional systems remains beyond what agents can reliably do. It contrasts those systems with batch workloads whose known outputs restore an executable feedback loop. (cognition.com)
The same article reports a narrower production-oriented success at Itaú: experts identified approximately 20 tax-ID field variations and encoded constraints into playbooks; Devin then helped refactor hundreds of programs, finishing three months ahead of a deadline with zero reported production errors. That is evidence for expert-directed, constrained change—not autonomous replacement of the banking platform. The observation window and error denominator are not published. (cognition.com)
1.4 AWS: distinguish stage metrics from completed-system metrics
| AWS claim | What it measures—and what remains uncertain |
|---|---|
| Mainframe decomposition in minutes rather than months; VMware network conversion up to 80× faster | Specific analysis/configuration activities, not complete application migration or production acceptance. (aws.amazon.com) |
| .NET modernization up to 4× faster; up to 40% licensing savings | Vendor-reported transformation and licensing outcomes. Licensing savings arise from the target platform; agent acceleration affects how quickly and cheaply it is reached. (aws.amazon.com) |
| Transformation and testing of 250,000-line mainframe applications in six weeks | AWS’s May 15, 2026 statement includes testing, making it broader than a conversion-only claim. It does not disclose the project sample size, test completeness, integration scope, or production-retirement results. (aws.amazon.com) |
| Seven billion lines processed and an estimated two million hours saved | AWS’s June 29, 2026 aggregate. Lines processed are not synonymous with lines successfully replaced in production; hours saved are estimated, not necessarily payroll reductions. (aws.amazon.com) |
| Amazon’s Java upgrades: more than 4,500 developer-years and $260 million annual savings | An internal, production-application upgrade campaign. AWS explains that developer effort was estimated from dependencies migrated, while annual savings were estimated from hosts removed following Java 17 performance improvements. (aws.amazon.com) |
The Java result is important evidence of scalable automation in a comparatively standardized environment. It should not be extrapolated directly to COBOL platform replacement, ERP transformation, or undocumented transactional applications.
AWS’s partner statements also present more modest expectations: HCLTech describes potential engineering-effort improvement of 10–15% and approximately 15% cycle-time reduction; Kyndryl projects that a 12-month project could take about eight months. These are partner observations/projections, not independently audited project distributions; sample sizes are undisclosed. (aws.amazon.com)
Revoqra synthesis: the published evidence supports substantial acceleration in selected activities and workloads. It does not support a universal multiplier for total enterprise modernization cost or elapsed time.
2. Conversion, modernization, and production acceptance are different deliverables
A useful completion ladder is:
- Understood: assets, dependencies, data meanings, and business rules mapped.
- Converted: replacement code generated and built.
- Behaviorally validated: representative and adversarial cases pass independent checks.
- Integrated: surrounding applications, data stores, schedulers, identity, and operational systems work together.
- Migrated: production data is transformed, reconciled, synchronized, and recoverable.
- Operationally accepted: load, resilience, security, support, and recovery requirements are demonstrated.
- Deployed and stabilized: real production traffic is served successfully.
- Retired: redundant legacy infrastructure, software, and contracts actually stop costing money.
This ladder is a Revoqra evaluation framework, informed by AWS’s separate conversion, testing, data-migration, and deployment workflows and SAP’s distinction between custom-code adaptation and system conversion. (aws.amazon.com)
AWS’s CardDemo instructions illustrate the separation particularly clearly: COBOL-to-Java transformation is followed by Terraform infrastructure deployment, a separate VSAM-to-RDS data migration, and application setup. It is a sample demonstration, not evidence that those activities were completed in the Mercedes-Benz pilot. (docs.aws.amazon.com)
Likewise, generating infrastructure-as-code is useful automation, but is not itself evidence of enterprise production acceptance, successful disaster recovery, or safe rollback after the new system has accepted writes.
3. Comparison across enterprise platforms and traditional approaches
3.1 AI is competing with existing automation—not only manual rewriting
Traditional options include retaining and improving the existing platform, rehosting or replatforming, deterministic conversion, incremental replacement, full rewriting, and replacement with packaged software. AI can assist several of these options; it does not make platform exit the default economically optimal choice. AWS itself distinguishes multiple modernization patterns, while Gartner advises workload-specific placement and modernization in place where appropriate. (aws.amazon.com)
Automatic COBOL-to-Java conversion also predates contemporary agents. A 2018 experience report documents a production migration using automatic translation, VSAM-to-Oracle conversion, and Linux deployment. In that single project, testing consumed approximately 70–80% of project time and contributed to a one-year delay. The authors also reported production problems and continuing knowledge-transfer and maintainability difficulties. This is an informative case, not an industry-wide percentage. (arxiv.org)
3.2 Platform comparison
| Platform/workload | Traditional baseline | Where agents can improve economics | What remains outside simple code conversion |
|---|---|---|---|
| COBOL batch applications | Translators, replatforming, manual reconstruction, batch-output comparisons | Rule discovery, first-pass documentation, replacement implementation, test harnesses, mismatch repair, repetitive edits | File semantics, scheduler dependencies, historical data, exception behavior, reconciliation, and production batch windows. Cognition’s customs case fits this bounded pattern. (cognition.com) |
| COBOL/CICS transactional systems | In-place improvements, API exposure, incremental extraction, specialized conversion and runtimes | Navigation, dependency analysis, documentation, scoped refactoring, and test preparation | Transaction boundaries, state, concurrency, recovery, external effects, and safe coexistence. AWS notes that modular monoliths or larger services may suit consistency requirements better than indiscriminate microservices. (aws.amazon.com) |
| Java | Build tools, dependency management, established refactoring recipes, manual upgrade fixes | Large portfolios of runtime/framework/API upgrades, build repair, and repeatable test-driven changes | Application-server behavior, proprietary libraries, performance, security, and release acceptance. Amazon’s internal upgrade campaign demonstrates production use but not a controlled comparison with every alternative automation tool. (aws.amazon.com) |
| .NET | Porting tools, containerization, compatibility analysis, manual replacement of Windows-specific functionality | Framework/package upgrades, Linux-readiness changes, unit-test execution, and portfolio orchestration | Windows-only dependencies, UI and database changes, identity, integration, and deployment. AWS’s launch materials include examples where further backend deployment was still a next step. (aws.amazon.com) |
| SAP ECC to S/4HANA | Readiness checks, simplification analysis, custom-code tooling, system conversion, process and data work | Explaining and resolving ABAP Test Cockpit findings, deterministic quick fixes, AI remediation, package processing, and transport logging | Business-process changes, add-on compatibility, system conversion, data-transition validation, authorization changes, and end-to-end acceptance. Custom-code migration is one workstream of the ERP program. (help.sap.com) |
| ETL, PL/I, MUMPS, assembly, and other legacy estates | Specialized parsers, compilers, translators, domain experts, platform-specific migration tools | Repetitive transformation and documentation where source context and executable validation are available | Language coverage, proprietary execution environments, obscure interfaces, and trustworthy evaluation. Nubank reports a large ETL migration; AWS added PL/I capabilities in 2026; MITRE’s research identifies evaluation challenges in less-common legacy languages. (building.nubank.com) |
3.3 SAP shows both meaningful progress and a clear boundary
SAP’s Q2 2026 release announcement identifies its S/4HANA custom-code migration agent as generally available. It executes readiness checks, interprets findings, applies deterministic and AI-generated fixes, and records changes in transports. High-confidence fixes are applied automatically; lower-confidence proposals are left for review. (news.sap.com)
That is substantive automation. But the agent’s post-processing requirement is developer review and activation, and SAP recommends checking AI-generated code for functional, semantic, and non-functional correctness. A confidence score and successful syntax check are not business-process acceptance. (help.sap.com)
SAP’s conversion guide separately requires custom-code testing and supports comparison of business data before and after conversion. Its earlier custom-code guide also warns that static ATC checks do not find every issue, including dynamic-code problems. Thus, an ABAP remediation percentage cannot be treated as the same percentage reduction in the whole S/4HANA implementation budget. (help.sap.com)
4. The difficult constraints: meaning, state, and evidence
4.1 Undocumented business rules
Revoqra synthesis: agents can recover descriptions of behavior encoded in source; they cannot establish from source alone which behavior is still commercially, legally, or operationally intended. Manual exceptions, customer agreements, configuration, operational workarounds, and external policies require additional evidence and accountable owners.
AWS’s 2026 requirements workflow adds traceability from extracted rules to source locations and generated requirements. That improves auditability, but traceability is not proof that every relevant rule has been captured or interpreted correctly. MITRE’s experiments similarly found that automatic evaluation measures did not align adequately with subject-matter experts’ assessments, recommending highly supervised use for mission-critical modernization. (aws.amazon.com)
A sound process therefore separates:
- Behavioral reconstruction: what the old system does.
- Business acceptance: what the replacement must do.
- Intentional change: behavior deliberately corrected, removed, or redesigned.
Treating all three as a single translation task risks either losing required exceptions or preserving obsolete defects.
4.2 Dependencies and execution environments
The relevant dependency graph extends beyond source calls to job schedules, files, database operations, shared state, and external interfaces. AWS’s documented testing workflow explicitly includes program, data, job, and scheduler dependencies and retains system-programmer control over mainframe data collection and transfer. (aws.amazon.com)
Claims that COBOL cannot execute in a Linux-based agent environment are too categorical: GnuCOBOL is an available compiler implementation. The harder issue is recreating the application’s actual platform semantics and surroundings. A compiler or mocked middleware environment is not equivalent to the production CICS/IMS/Db2 estate. (gnucobol.sourceforge.io)
4.3 Transaction integrity and operational risk
The following are recommended acceptance obligations, not claims that any particular vendor case failed them:
| Risk | Evidence required before production acceptance |
|---|---|
| Business-rule loss | Rule inventory with source evidence, approved exceptions, and independent domain-owner review. |
| Numeric and record-layout changes | Explicit tests for decimal scaling, rounding, signs, packed fields, padding, encoding, aliases, and boundary values. |
| Transaction integrity | Demonstration that postings, balances, inventory movements, and external effects remain correct under retries, concurrency, crashes, and restart. |
| Data migration | Reconciled record counts, control totals, keys, relationships, historical balances, rejected records, and transformation exceptions—not merely a completed load. |
| Integration failures | Contract and end-to-end tests covering schedulers, identity, downstream consumers, third parties, and error handling. |
| Operational instability | Peak-load, batch-window, failover, recovery, monitoring, support, and capacity tests against agreed service objectives. |
| Unsafe rollback | A rehearsed rollback or forward-recovery plan that accounts for writes already accepted by the new system. |
These obligations follow the distinction between functional, integration, non-functional, and user-acceptance testing in AWS’s guidance, and the resource-aware validation approach in IBM’s research. (aws.amazon.com)
A particularly dangerous architectural shortcut is replacing one atomic business operation with several independently committed services without explicitly validating the new failure behavior. Incremental coexistence reduces the size of changes, but introduces synchronization and recovery questions that must themselves be designed and tested. AWS’s banking guidance recognizes both coexistence patterns and the importance of preserving suitable transaction boundaries. (aws.amazon.com)
4.4 Security has two separate layers
- Generated-system security: authentication, authorization, secrets, dependencies, input handling, data protection, and deployment configuration.
- Agent-execution security: repository and tool access, credentials, network egress, untrusted instructions, audit trails, and authority to change production resources.
Cognition describes single-tenant AWS environments, dedicated VPCs, ephemeral session machines, and audit APIs. These are useful isolation and governance features, but do not prove application correctness or eliminate the need to inspect model endpoints, permissions, retention, and contractual data handling. (cognition.com)
Security research also cautions against autonomous acceptance of generated fixes. In a peer-reviewed study’s September 2025 retest, three models repaired 42–44 of 56 confirmed vulnerabilities, or 75–78.6%. These were selected code issues, not a modernization failure-rate sample. The study found improvements but incomplete detection and repair, supporting independent scanners and human supervision. (link.springer.com)
Recommended controls: least privilege, isolated execution, protected branches, independent security checks, controlled secrets, auditable changes, and separate approval for production actions. Agent-specific research identifies attack chains involving untrusted artifacts and tool execution; established secure-development practices remain applicable regardless of who writes the code. (sciencedirect.com)
5. Validation can be automated—but the oracle cannot be assumed
An agent can generate both replacement code and tests that agree with its own misunderstanding. Consequently, passing generated tests is stronger evidence when expected results come from an independent source: the original executable, approved business specifications, historical reconciliations, or a separately constructed reference implementation.
IBM’s automated-validation research uses symbolic execution, mappings between COBOL and Java, generated assertions, and mocked external resources. Its demonstration evaluated nine programs, selecting a paragraph with substantial resource interaction from each; the programs contained approximately 300–1,500 lines. This demonstrates useful validation machinery, not proof of equivalence for a complete transactional estate. (arxiv.org)
AWS has also moved beyond code generation. Its December 2025 testing capabilities prepare test plans, collection scripts, and execution scripts, initially focusing on batch functional testing. Its guidance nevertheless says programmers review and execute collection scripts, and comprehensive testing includes integration, performance, availability, disaster recovery, and user acceptance. AWS’s statement that testing typically consumes more than half of modernization resources is a first-party estimate with no published sample denominator. (aws.amazon.com)
Revoqra synthesis: reliable automation means errors are observable and acceptance is independently repeatable—not that the generator never makes mistakes. A reusable behavioral harness can be more economically valuable than another increase in raw code-generation speed.
Evidence outside vendor case studies
The available independent productivity evidence is mixed and not directly comparable with migration claims:
- METR’s early-2025 randomized trial involved 16 experienced developers and 246 issues in familiar open-source projects; AI access increased completion time by 19%. This was a specific setting and historical tool snapshot, not a finding about all modernization. (metr.org)
- Its February 2026 follow-up warned that selection effects and concurrent-agent time measurement made the new productivity estimate unreliable. (metr.org)
- Its May 2026 survey of 349 technical workers found median self-reported work-value gains of 1.4–2×, but self-reported value is not the same measurement as experimentally observed completion time. (metr.org)
Gartner’s June 18, 2026 warning predicts that more than 70% of mainframe-exit projects initiated in 2026 will fail to deliver intended benefits because of overestimated AI capabilities. This is a forecast, not an observed failure rate; the public release does not provide a cohort dataset or statistical denominator. It is relevant contradictory institutional judgment, not proof that individual migrations cannot succeed. (gartner.com)
6. Which activities can be reliably automated?
The ratings below are Revoqra synthesis, conditional on adequate inputs, access, and acceptance checks. They describe task execution, not permission to release independently.
| Activity | Automation assessment | Remaining control |
|---|---|---|
| Inventory, parsing, dependency extraction | Strong for observable assets; combine deterministic analysis with AI explanation | Check completeness against runtime, configuration, and operational inventories. |
| Documentation and rule extraction | Valuable first-pass automation | Domain experts verify meanings, omissions, and exceptions. |
| Repetitive runtime/API/package upgrades | Strong candidate after representative calibration | Independent build, regression, security, and performance gates. |
| Bounded batch conversion | Strong conditional candidate with representative input/output evidence | Holdout cases, full data reconciliation, and production scheduling acceptance. |
| Test generation and mismatch repair | Substantial automation potential | Expected behavior must not depend solely on the same agent’s interpretation. |
| Test execution and output comparison | Highly automatable; much of this is deterministic rather than uniquely AI | Coverage, reference data, and tolerances require approval. |
| Infrastructure provisioning and deployment scripts | Highly automatable within approved templates | Security, resilience, change approval, and recovery remain accountable. |
| Cross-system transaction redesign | Assistance rather than reliably unattended acceptance | Architecture and failure semantics need expert ownership. |
| Data cleansing and business mapping | Partial automation | Ambiguous meaning, exceptions, and financial reconciliation remain governed. |
| Production cutover and legacy retirement | Automatable runbook steps, not autonomous business authorization | Operations, business owners, security, and contract owners approve completion. |
The supporting evidence is strongest for well-specified repetitive work, functional validation tooling, and governed deployment preparation—not unconstrained architectural replacement. (cognition.com)
Why 30× conversion is not 30× modernization
For a simplified sequential project, if fraction f of elapsed time is accelerated by factor s, total speedup is:
Total speedup = 1 / [(1 − f) + f/s]
Revoqra-derived illustration: if conversion occupies 20% of the critical path and becomes 30× faster, total speedup is only 1.24×, or approximately 19% less elapsed time, before new review or rework. The 20% share and 30× factor here are assumptions, not measured project statistics.
This is a boundary calculation, not a forecast: real projects have overlapping work and dependencies. Agents can improve additional stages, increasing the affected share. But more parallel coding sessions do not automatically create more domain-review capacity, production-like test environments, external-system cooperation, or acceptable cutover windows.
7. Reconstructing total costs
7.1 What cannot be reconstructed from public claims
The reviewed Cognition announcements do not disclose the Mercedes-Benz project’s headcount, human hours, agent usage, services fees, testing costs, data volumes, or operational spending. The separate 73% case likewise lacks an auditable cost ledger. Actual all-in dollar costs cannot responsibly be inferred from the reported elapsed times. (cognition.com)
The 73% claim implies only that the reported AI migration-cost bucket was estimated at 27% of its counterfactual baseline. Revoqra-derived illustration: if that bucket represented 40% of the whole program, a 73% reduction would save 29.2% of the total before added tools, enablement, review, and rework. Neither the 40% share nor the resulting whole-program saving is reported case data.
An AWS economic example offers a better cost taxonomy, although for a different workload. Its hypothetical VMware migration divides a $7.68 million cost of change into $4.57 million of planning/migration/training and $3.11 million of dual running. The AI scenario models $4.82 million total and a shorter migration. These are modeled figures, not an observed COBOL ledger, and are used here only to illustrate cost categories. (aws.amazon.com)
7.2 An illustrative production-modernization budget
All figures below are Revoqra assumptions and calculations—not Mercedes-Benz estimates, vendor quotations, or empirical benchmarks. The example represents one medium-sized, production-oriented application program. Its cost structure reflects the separate workstreams in modernization guidance, but the dollar allocations are independently stated assumptions. (aws.amazon.com)
Assumptions:
- Traditional duration: 12 months; AI-assisted duration: eight months.
- Same required production functionality and acceptance standard.
- Blended fully loaded human cost: $125/hour, assumed rather than asserted as a market rate.
- Tools include project-period usage and test infrastructure; ongoing production costs are assessed separately.
- Dual running is the incremental cost over business as usual, avoiding double-counting normal legacy operations.
- The contingency allowance is assumed spent for this budget screen; it is not a measured failure probability.
| Workstream | Traditional | AI-assisted |
|---|---|---|
| Discovery and business-rule reconstruction | $350,000 | $200,000 |
| Architecture and transformation design | $200,000 | $180,000 |
| Code conversion and implementation | $600,000 | $150,000 |
| External integration | $250,000 | $210,000 |
| Data migration and reconciliation | $250,000 | $230,000 |
| Functional, performance, security, and user-acceptance testing | $900,000 | $750,000 |
| Cutover, operational readiness, and training | $150,000 | $140,000 |
| Program management and governance | $150,000 | $130,000 |
| Tools, agent usage, and test infrastructure | $100,000 | $180,000 |
| Incremental dual running | $300,000 | $200,000 |
| Rework/contingency allowance | $250,000 | $250,000 |
| Total | $3,500,000 | $2,620,000 |
Revoqra-derived results: total savings are $880,000, or 25.1%. Human workstream spending falls from $2.85 million to $1.99 million—equivalent to 22,800 versus 15,920 hours at the assumed blended rate. Conversion alone saves $450,000; the remaining improvement comes from other workstreams and shorter coexistence, partly offset by higher tooling expenditure.
These are accounting buckets, not universal stage shares. Validation also occurs within conversion, integration, data migration, and domain review; the dedicated testing line should not be directly compared with historical percentages of total time described as testing.
Two sensitivities are revealing:
- Even making the $600,000 conversion bucket 30× cheaper would save approximately $580,000, or 16.6% of the original total, before additional costs.
- If unbudgeted rework and risk costs exceed $880,000, this modeled cost advantage disappears.
7.3 The target platform must also be priced
For modernization paths using its licensed mainframe runtime, AWS publishes an on-demand rate of $0.31 per AWS CPU core-hour. Revoqra-derived illustration: 64 billable cores operating continuously for 8,760 hours would cost approximately $173,800 annually for that runtime charge alone, before infrastructure and other services. Use the vendor’s billable-core definition rather than assuming it equals a particular instance’s vCPU count. (aws.amazon.com)
A complete target operating budget should include databases, high availability, disaster recovery, storage, network transfer, observability, security, support, and continuing engineering. Agent subscriptions and usage are another budget item, not a substitute for the implementation team; the actual order form and usage bills should govern the estimate. (devin.ai)
Recommended financial discipline: count only legacy expenses that can actually be avoided. Removing one workload may release little cash if shared infrastructure, minimum commitments, or other applications still require the platform. Do not equate an allocated share of mainframe cost with a realizable saving.
8. Five-year ROI scenarios
Analysis horizon: five years from project authorization. This is separate from the evidence review period.
Method
The scenarios below are Revoqra-derived economic illustrations, not forecasts or observed customer results:
- All project costs are conservatively placed at authorization, year zero.
- Annual net benefit includes avoidable legacy spending and any explicitly assumed operational benefit, less replacement-platform costs and ongoing support.
- Benefits start only after production acceptance and are prorated in the acceptance year.
- Cash flows are booked at year-end and discounted at an assumed 10%.
- No terminal value, taxes, or financing effects are included.
- Simple payback assumes uniform benefit accrual after acceptance; it is an approximation, not the discounted cash-flow measure.
NPV = −Project cost + Σ[Annual net benefit_t / 1.10^t]
Five-year net ROI = (Five-year undiscounted benefits − Project cost) / Project cost
Results
T = traditional. Dollar amounts are millions; acceptance is in months and payback in years. Every input in this table is an assumption.
| Scenario | Cost: T / AI | Acceptance: T / AI | Annual net benefit | Five-year NPV: T / AI | Five-year net ROI: T / AI | Payback from authorization: T / AI |
|---|---|---|---|---|---|---|
| Repetitive Java/.NET upgrade portfolio | $0.80 / $0.35 | 9 / 4 | $0.18 | −$0.24 / +$0.28 | −4% / 140% | 5.2 / 2.3 |
| Bounded COBOL batch migration | $1.20 / $0.65 | 12 / 6 | $0.25 | −$0.48 / +$0.18 | −17% / 73% | 5.8 / 3.1 |
| Core application, limited legacy-cost release | $3.50 / $2.62 | 12 / 8 | $0.50 | −$2.06 / −$1.03 | −43% / −17% | 8.0 / 5.9 |
| Same core budget, with achievable contract/capacity retirement | $3.50 / $2.62 | 12 / 8 | $1.30 | +$0.25 / +$1.52 | 49% / 115% | 3.7 / 2.7 |
| Broad SAP ERP program with modest delivery savings | $12.00 / $10.80 | 24 / 21 | $3.00 | −$5.83 / −$4.01 | −25% / −10% | 6.0 / 5.4 |
Interpretation
1. AI can change whether a project clears the investment hurdle. In the assumed upgrade and bounded-batch cases, lower delivery cost and earlier acceptance move five-year NPV from negative to positive. This illustrates a fundamental economic improvement without requiring full autonomy.
2. Faster conversion does not rescue every platform-exit case. The limited-retirement core scenario remains negative despite $880,000 of project savings. With the assumed acceptance dates, the AI case needs approximately $823,000 of annual net benefit to break even over five years, versus approximately $1.215 million for the traditional case. These thresholds are calculated from the stated cash-flow assumptions.
3. Actual retirement matters more than the language label. Raising annual net benefit from $500,000 to $1.3 million changes the conclusion dramatically. That row is conditional: the required costs must actually disappear at the assumed date. If retirement lags application go-live, or requires a larger decommissioning program, both timing and cost must be remodeled.
4. Earlier delivery has value only after acceptance. In the central core example, moving acceptance from month 12 to month eight adds four months of assumed benefit: approximately $167,000 undiscounted, or $152,000 present value under the annual convention. Producing code early without advancing production acceptance does not earn that benefit.
5. A long ERP program may need a longer economic horizon. The SAP illustration is negative over five years largely because benefits arrive late. Longer-lived benefits, strategic requirements, or other objectives may justify the program—but a custom-code productivity claim should not be used to invent those benefits.
What must be measured before calling ROI real
- Accepted functionality per dollar, including failed agent attempts and human repair.
- Human hours across engineering, subject-matter expertise, validation, operations, and governance.
- Actual usage, environment, vendor, and services bills.
- Time to production acceptance—not just time to generated code or merged pull requests.
- Defects and incidents through an agreed stabilization period.
- Net target-platform operating costs.
- The dates and amounts of legacy expenses actually retired.
These are recommended measurement requirements. No comparable public dataset of independently audited, all-in project costs and acceptance outcomes was found in the reviewed material; therefore no cross-project average saving or independent significance test is claimed.
9. Is AI removing the main bottleneck—or moving it?
It can do more than accelerate one stage. AWS’s requirements and testing developments address discovery and validation preparation, while Cognition’s playbook-based cases address repeated transformation and repair. These are economically meaningful extensions beyond generating code. (aws.amazon.com)
However, the residual bottleneck is often trusted evidence and accountable decisions:
- Who confirms that recovered rules mean the right thing?
- Where do representative reference outputs and production-like environments come from?
- Who owns cross-system transaction behavior?
- When can the organization safely migrate live state?
- Which costs will actually stop after deployment?
Revoqra synthesis: agents substitute scalable execution for some scarce labor, but do not automatically substitute for the authority, context, and evidence required to approve a business-critical replacement. They may also increase review demand by generating changes faster than the organization can validate them.
The most powerful economic configuration is therefore not an unrestricted agent. It is an agent inside a well-instrumented modernization factory: deterministic analysis, domain-approved playbooks, independent behavioral references, automated tests, controlled deployment, and explicit retirement milestones.
10. Practical decision framework
Prefer these initial candidates
- Repeated runtime, framework, and dependency changes across many similar repositories.
- Batch components with clear inputs, outputs, and limited external effects.
- Documentation and discovery work whose outputs can be checked by existing experts.
- Narrow, mandated refactoring with explicit constraints and production tests.
These candidate patterns match the better-described vendor cases and tooling capabilities; they are not guarantees of success. (cognition.com)
Treat these as higher-risk programs
- Transactional cores with poorly understood state and recovery behavior.
- Simultaneous changes to language, data model, architecture, interfaces, and operating platform.
- ERP transformations driven primarily by process and master-data change.
- Projects whose savings require retirement of infrastructure shared with many untouched systems.
Design the pilot to test economics, not theater
- Compare against the best available traditional approach, including deterministic tools—not a deliberately manual baseline.
- Define identical acceptance criteria before either approach starts.
- Include representative difficult modules, not only easy successes.
- Record all human and tool costs, including setup, review, exceptions, and abandoned attempts.
- Use independent reference behavior and holdout cases.
- Require operational evidence, not just compilation or a demonstration.
- Attach the business case to a retirement plan.
- Contract for accepted outcomes and evidence, rather than lines processed or nominal agent activity.
These are Revoqra recommendations based on the disclosure gaps, validation research, and documented human-in-the-loop workflows reviewed above.
Conclusion
Yes—autonomous AI agents can fundamentally improve the economics of selected legacy-modernization programs. They can lower the fixed cost of understanding systems, make repetitive changes economical at portfolio scale, and reduce the labor and coexistence time needed to achieve production acceptance.
No—the available evidence does not show that they have eliminated the central risks of enterprise modernization. The eight-day COBOL headline lacks the disclosures needed to verify a complete production migration; transactional integrity, data reconciliation, independent validation, organizational acceptance, and retirement economics remain decisive.
The useful investment question is not how quickly an agent can convert the source. It is: How much does it cost to deliver independently validated production behavior, operate it safely, and stop paying for the old capability?
Sources and evidence notes
- Primary claim audit: Cognition’s April 27 Mercedes-Benz announcement, Mercedes-Benz’s April 28 customer statement, and Cognition’s September 15 AWS collaboration announcement. These establish the published claims and their disclosure boundaries, not an independent delivery audit. (cognition.com)
- COBOL workload evidence: Cognition’s April 8, 2026 case discussion, including batch migration, expert-directed refactoring, and its transactional-migration limitation. Customer identities and accounting detail are incomplete for some cases. (cognition.com)
- AWS metrics and delivery capabilities: May 2025 launch, May and June 2026 milestone statements, December 2025 testing/refactoring guidance, and the CardDemo deployment pattern. Marketing metrics are distinguished from documented workflow responsibilities. (aws.amazon.com)
- Amazon Java economics: AWS’s August 1, 2024 original explanation of dependency-based effort estimates and host-based annual savings. (aws.amazon.com)
- Partner expectations: AWS’s partner page, including HCLTech and Kyndryl statements; projected benefits are not treated as audited averages. (aws.amazon.com)
- SAP: Q2 2026 agent-release announcement, agent documentation, AI constraints, and the S/4HANA 2025 conversion guide. These distinguish custom-code remediation from whole-system conversion and acceptance. (news.sap.com)
- Original validation research: IBM-authored automated COBOL-to-Java validation and testing papers. Their experimental scope and use of mocked resources limit conclusions about whole production systems. (arxiv.org)
- Historical production comparison: De Marco, Iancu, and Asinofsky’s 2018 migration experience report; one project, not an industry sample. (arxiv.org)
- Independent/institutional assessment: MITRE modernization research; METR’s experimental and survey evidence; Gartner’s explicitly forward-looking mainframe-exit warning. Their different methods and populations are not pooled into a common productivity estimate. (mitre.org)
- Other platform and execution evidence: Nubank’s first-party ETL account and GnuCOBOL’s official project documentation. (building.nubank.com)
- Security: Original peer-reviewed vulnerability-repair research, code-agent security research, and NIST’s secure-development framework. (link.springer.com)
- Costs and pricing: AWS’s hypothetical migration economic model, published mainframe-runtime pricing, and Devin’s pricing publication. The report’s project budgets and ROI results are separately labeled Revoqra-derived assumptions and calculations. (aws.amazon.com)
Sources
- Devin's 2025 Performance Review: Learnings From 18 Months of Agents At Work | Cognition
- Engineering in the fast lane: Mercedes-Benz partners with Cognition | Cognition
- AWS Transform: From migration to continuous modernization | Migration & Modernization
- Agentic AI in Software Development | Mercedes-Benz Group > Technology > Digitalisation > Artificial Intelligence
- Cognition and AWS team up to help ambitious teams ship more, faster | Cognition
- How Devin Is Modernizing COBOL at Fortune 500 Companies | Cognition
- Transform Enterprise Workloads up to 4x Faster with Agentic AI | Migration & Modernization
- One year. 4.5 billion lines of code. 1.6 million hours saved. Here’s what we learned. | Migration & Modernization
- Amazon Q Developer just reached a $260 million dollar milestone | AWS DevOps & Developer Productivity Blog
- Partners – AWS Transform – AWS
- Accelerate mainframe modernization with AWS Transform: A comprehensive refactor approach | Migration & Modernization
- Modernize and deploy mainframe applications using AWS Transform and Terraform - AWS Prescriptive Guidance
- AWS Transform for mainframe introduces Reimagine capabilities and automated testing functionality | AWS News Blog
- arxiv.org
- Modernizing Core Banking Systems: A Strategic Guide for Financial Leaders | AWS for Industries
- SAP S/4HANA Custom Code Migration Agent | SAP Help Portal
- Enhancing engineering workflows with AI: a real-world experience - Building Nubank
- SAP Business AI: Release Highlights Q2 2026 | SAP News Center
- help.sap.com
- GnuCOBOL - GNU Project
- Accelerating mainframe modernization testing with AWS Transform | Migration & Modernization
- Secure coding with AI – from detection to repair | Empirical Software Engineering | Springer Nature Link
- A focused survey of code agent security: Attacks, defenses, and evaluation - ScienceDirect
- Automated Validation of COBOL to Java Transformation
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METR
- We are Changing our Developer Productivity Experiment Design - METR
- Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity - METR
- Gartner Predicts More Than 70% of Mainframe Exit Projects Will Fail Due to Overestimation of Generative AI’s Capabilities
- Accelerating Cloud Migration with AWS Transform and Generative AI | Migration & Modernization
- Managed Mainframe Migration Service | AWS Mainframe Modernization Pricing
- Plans and Pricing | Devin
- LEGACY IT MODERNIZATION WITH AI