SG Pivot: Banks Move From In-House LLMs to Copilot for Enterprise AI

Société Générale’s abrupt pivot from its bespoke internal assistant, SecureGPT, to Microsoft Copilot crystallizes a fast-moving truth in enterprise AI: building an in‑house large language model and keeping it competitive at scale is now a very different challenge from proving a concept in a lab.

Background​

Launched less than a year ago as an internal, proprietary assistant, SecureGPT was positioned as Société Générale’s answer to data‑sensitive, regulated AI use: trained on the bank’s own data, governed by internal teams, and designed to support both front‑office and back‑office workflows without exposing sensitive information to third parties. The tool reportedly had a dedicated development and maintenance squad and was promoted internally as a driver of operational efficiency, compliance‑aware automation, and staff productivity.
The decision to phase out SecureGPT and standardize on Microsoft Copilot was taken late last year, according to reporting that has circulated across industry outlets. The key drivers cited were escalating maintenance costs, an accelerating technology gap between bespoke models and hyperscaler solutions, and a sustained shortfall in feature parity and freshness when compared to market leaders. The move highlights three connected realities: the speed of model advancement, the total cost of ownership for production LLMs, and the tradeoffs between sovereignty and capability.

---

The technological gap: why in‑house LLMs struggle to keep up​

The cadence of innovation​

Large language models and their associated toolchains evolve rapidly. New model families, optimizations for latency and cost, multimodal and retrieval‑augmented architectures, and continual updates to guardrails and safety systems arrive frequently. Hyperscalers and leading AI model vendors can push major architectural and deployment improvements across massive fleets overnight; internal teams rarely have the same scale of compute, engineering headcount, or telemetry to iterate at the same pace.
This velocity drives two practical problems for internal projects:

• Feature lag: internal models fall behind in capabilities that users increasingly expect—reasoning improvements, contextual memory, multimodal inputs, and integrated plug‑ins or connectors.
• Maintenance burden: keeping pipelines, fine‑tuning datasets, safety filters, and evaluation suites current becomes a full‑time engineering task with recurring costs that often outstrip the initial build budget.

Economics of scale​

Running production LLMs—especially ones tuned for low latency, high throughput, and strong privacy controls—requires significant compute (and therefore cost), experienced MLOps teams, and robust SRE and security investment. Over time, these ongoing costs compound: model retraining, prompt engineering, data curation, monitoring for hallucinations and bias, and legal/compliance reviews are continuous expenditure lines that many in‑house projects underestimate.
For many tier‑one AI vendors, the unit economics work differently. They amortize R&D across millions of users, operate massive GPU fleets, and negotiate energy and infrastructure costs at scale. That concentration of investment translates to rapid feature delivery and continuous cost optimizations that are hard for a single enterprise to match.

Productization and integrations​

Modern enterprise AI buyers want more than an LLM behind a chat window: they demand deep integrations with identity, document stores, CRM and ERP systems, secure connectors, enterprise search, telemetry dashboards, and corporate governance features (audit logs, access controls, data residency options). Hyperscaler Copilots have matured into platforms with built‑in integrations and operational tooling that reduce time‑to‑value—another dimension where in‑house builds face a steep catch‑up curve.

---

Make or buy: the market tilt toward “buy” and why it matters​

The shifting preference​

What once was a balanced debate—do we build specialized models to retain control, or buy to accelerate—has tilted for many organizations toward buying. The reasons are practical: speed of deployment, lower short‑term cost to get a working tool, and access to feature sets maintained by large vendors.
Buying a Copilot offering gives organizations:

• Immediate access to continuously updated models and safety tooling.
• Prebuilt connectors to common enterprise systems.
• Compliance and legal commitments from a vendor with enterprise contract practices.
• A transfer of operational risk (monitoring, uptime, patching) to the supplier.

That said, surveys of enterprise AI strategies still show nuance: some organizations maintain internal teams and proprietary datasets for highly sensitive workloads while outsourcing general productivity copilots—creating hybrid architectures rather than an all‑or‑nothing choice.

The compromise approach: hybrid and layered ownership​

Most realistic enterprise strategies now split the stack:

• Data and governance layers remain tightly controlled—document stores, vector databases, and access rules are kept on‑premise or in dedicated tenant environments.
• Model inference and heavy compute are outsourced or commercialized—enterprises consume hosted models or Copilot services for generic reasoning and productivity tasks.
• Customization is achieved through controlled fine‑tuning, adapters, or retrieval layers that sit between the enterprise data and the external model.

This architecture attempts to capture the best of both worlds: sovereignty over sensitive data, paired with the raw capability and maintenance economics of external models.

---

Sovereignty, regulation and strategic risks​

Data privacy and regulatory exposure​

Banking is among the most tightly regulated industries for good reason. Client confidentiality, anti‑money laundering (AML) controls, and strict data residency requirements complicate any decision to route customer data through third‑party systems. Choosing an external Copilot raises legitimate questions:

• Where is inference performed and where do logs reside?
• What controls prevent leakage of proprietary prompts or outputs back into vendor training sets?
• How are audit trails, retention policies, and e‑discovery implemented to align with banking regulations?

These aren’t theoretical. Regulators are increasingly clear that outsourcing AI functions does not absolve institutions of accountability. Contracts and technical controls must therefore be explicit and verifiable.

Vendor concentration and strategic dependency​

Adopting a major non‑domestic AI provider increases supplier concentration risk. For European banks, this introduces a geopolitical and strategic dimension: dependence on U.S. hyperscalers may leave institutions exposed to policy shifts, export controls, or contractual changes. Vendor lock‑in can also constrain future architecture and bargaining power. Over time, reliance on a small set of providers may erode competitive differentiation and increase strategic vulnerability.

Intellectual property and model behavior​

Using an external Copilot introduces another class of risk: how the model handles and reuses enterprise inputs. Vendors have published various commitments—some offering indemnities around copyright or promising not to use customer data for training without consent—but contractual language and technical enforcement can vary. Companies must verify:

• Model training and caching policies.
• The scope and enforceability of vendor indemnities.
• The mechanisms for deleting or quarantining data.

---

Practical benefits of buying Copilot‑class solutions​

Despite the risks, moving to a Copilot model delivers clear, immediate advantages for many organizations:

• Rapid feature parity: advanced reasoning, multimodal support, and prompt‑tuning features arrive without the need to redesign internal stacks.
• Lower short‑term operational load: fewer embedded hiring needs for large MLOps teams and less capital expended on GPUs and infrastructure.
• Integrated enterprise tooling: single sign‑on (SSO), M365/Teams integration, document connectors, and administrative consoles are often provided out of the box.
• Vendor security programs and compliance certifications: many providers already maintain SOC 2, ISO, or similar attestations that accelerate procurement and risk approval.

For many banks, these advantages translate to faster rollouts of productivity enhancements across thousands of employees—a tempting operational win.

---

Assessing the true costs: TCO beyond the sticker price​

Total cost of ownership for AI includes direct fees, hidden overheads, and opportunity costs. A robust TCO assessment should include:

• Direct subscription or usage fees for model inference and platform features.
• Integration costs to connect Copilot with legacy systems, identity, and document repositories.
• Ongoing vendor management and contract negotiation overhead.
• Residual internal staffing for governance, data pipelines, and prompt engineering.
• Migration and exit planning costs if the institution decides to pivot later.

A counterintuitive finding is that buying can sometimes increase short‑term OPEX but lower long‑term engineering expense—depending on an organization’s scale and sensitivity needs. For institutions with massive, unique datasets or crown‑jewel models in their value chain, build may remain the better long‑term bet. For general productivity assistants and mainstream knowledge work, buy often wins.

---

Governance and technical controls organizations should demand​

To reduce the downsides of using a Copilot provider, institutions should insist on hardened contractual and technical controls:

• Explicit data handling guarantees: no training on customer data without explicit opt‑in, clear retention windows, and hard deletion options.
• On‑prem or private‑cloud inference: options to run models in customer‑owned clouds or dedicated tenant environments.
• Auditability: full access to logs, redaction capabilities, and immutable audit trails that meet regulatory standards.
• Explainability and testing: vendor support for model explainability, deterministic testing suites, and bias mitigation documentation.
• Exit and portability clauses: clearly defined data export, model handover, and escrow mechanisms to avoid lock‑in.

Institutions that negotiate these controls can capture the speed of external models while maintaining stronger sovereignty postures.

---

What layers to own — a practical, ranked approach​

Not every organization can own an entire AI stack. A pragmatic distribution of ownership might look like this:

• Data governance and vector stores — fully owned and controlled.
• Retrieval and RAG pipelines — co‑managed, with local processing of sensitive retrieval.
• Prompt engineering, safety policies, and user‑level access controls — owned by the enterprise with vendor enforcement hooks.
• Model inference and heavy compute — outsourced to hyperscalers or Copilot platforms for generic workloads.
• Domain‑specific or revenue‑critical models — owned or tightly co‑developed with vendors.

This split helps manage risk while leveraging external scale.

---

Operationalizing the switch: steps Société Générale and peers likely face​

• Inventory and classification: map which workloads are safe to route through Copilot and which must remain internal.
• Pilot and validate: run parallel comparisons on accuracy, latency, hallucination rates, and security controls.
• Contract hardening: negotiate explicit data‑use and indemnity clauses, plus SLAs for latency, availability, and support.
• Integration layers: build secure connectors that keep raw sensitive data internal and only expose redacted or tokenized query contexts.
• Governance rollout: update policies, training, and audit processes to reflect new supplier relationships.
• Exit planning: design migration pathways and data export processes in case vendor strategy changes.

These steps are not theoretical—they are operational necessities for large regulated enterprises.

---

Strengths of the Copilot choice​

• Immediate uplift in employee productivity via richer features and continuous updates.
• Reduced complexity in keeping safety filters and model monitoring current.
• Rapid access to vendor security programs, compliance artifacts, and enterprise support.
• Economies of scale for compute and R&D that translate into faster access to breakthrough capabilities.

---

Risks and trade‑offs​

• Strategic dependence on non‑European technology providers with geopolitical implications.
• Potential exposure of sensitive metadata or prompts unless rigorous technical controls are in place.
• Vendor roadmap divergence: customers may find themselves outpaced by vendor product decisions (e.g., feature deprecations).
• Long‑term cost growth if usage scales rapidly without effective FinOps controls.

Any institution considering a similar move must balance immediate business value against these strategic considerations.

---

Broader market signal: what this move means for European AI sovereignty​

Société Générale’s decision—if matched by other European financial institutions—sends a sober market signal. The aspiration for European AI sovereignty remains strategic and desirable, but practical execution is constrained by capital, talent, and compute economics. Sovereignty can still be pursued, but it will likely require:

• A more federated model where critical datasets and governance remain local.
• Joint European investments in shared infrastructure and national/regional model initiatives to achieve scale.
• Clearer regulatory frameworks that incentivize domestic capability building without penalizing pragmatic adoption of external services.

Absent those structural changes, many organizations will default to vendor technologies for core productivity functions while protecting only the most sensitive layers internally.

---

Recommendations for enterprise IT leaders​

• Treat Copilot adoption as a strategic procurement: evaluate governance, exit options, and long‑term cost models as rigorously as you would any core banking vendor.
• Prioritize data minimization and strong redaction/tokenization for any externalized queries.
• Maintain an internal competency center for prompt engineering, safety testing, and model evaluation even if inference is outsourced.
• Design hybrid architectures from day one: don’t make the choice binary; plan to own the layers you can realistically protect.
• Push for industry‑level consortia to build shared European infrastructure where sovereignty objectives require scale.

---

Conclusion​

Société Générale’s pivot illustrates a turning point for enterprise AI strategy: the aspiration to build a fully sovereign internal LLM remains attractive, but the operational and economic realities increasingly favor buying best‑in‑class capabilities for mainstream productivity use cases. The pragmatic path forward is layered and hybrid—retain control where it materially matters, outsource where scale and continuous innovation are decisive.
That tradeoff will define enterprise AI architecture for the next several years: a balancing act between sovereignty, speed, cost, and capability. Organizations that explicitly design for a hybrid future—owning the governance and data layers while leveraging external model economies—will be best positioned to capture immediate benefits without ceding long‑term strategic control.

---

Source: DirectIndustry e-Magazine Société Générale Drops Internal AI for Copilot - DirectIndustry e-Magazine