SQL Server 2025 Can Call an LLM Directly from T-SQL — Here's What That Actually Means

I've been working with SQL Server for over 30 years. I've watched it evolve from a workgroup database to an enterprise platform capable of running petabyte-scale workloads. Most of those releases were incremental. SQL Server 2025 is not.

The feature I want to talk about today isn't the native vector type or DiskANN indexing — I'll cover those separately. It's something more fundamental: the ability to call an external LLM directly from T-SQL, right inside a stored procedure, using a system stored proc called sp_invoke_external_rest_endpoint.

When I first saw this, I did a double-take. The query layer and the AI layer just merged.

Let me walk you through what it is, how it works, a practical example, and — because this is a DBA blog — where you need to be careful.

What Is sp_invoke_external_rest_endpoint?

This is a new system stored procedure introduced in SQL Server 2025 (version 17.x). It was previously available in Azure SQL Database and Azure SQL Managed Instance, but it's now on-premises too. That distinction matters. A lot of SQL Server AI features require Azure. This one doesn't.

What it does is straightforward: it lets the SQL Server engine make an outbound HTTP POST or GET request to any REST endpoint. In the context of AI, that means you can call Azure OpenAI, OpenAI directly, Google Gemini, Anthropic, Ollama running locally, or any other model exposed over a REST interface — all from T-SQL.

The feature is disabled by default. That's the right call. You opt into it explicitly, which gives you a conscious decision point before your database starts making outbound network calls.

Setup: Three Things You Need Before You Write a Single Query

1. Enable the Feature

sqlEXEC sp_configure 'external rest endpoint enabled', 1;

RECONFIGURE WITH OVERRIDE;

This is a server-level setting. Enable it deliberately, document it, and make sure your change management process knows about it.

2. Create a Database Master Key

If your database doesn't already have one, you need it to store credentials securely:

sqlIF NOT EXISTS (

SELECT 1 FROM sys.symmetric_keys

WHERE name = '##MS_DatabaseMasterKey##'

)

BEGIN

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'UseAStrongPasswordHere!2025';

END

3. Store Your API Key as a Scoped Credential

This is the part I want to emphasize to every DBA reading this: do not hardcode your API key in query text. SQL Server's credential model gives you a clean way to handle this.

sqlCREATE DATABASE SCOPED CREDENTIAL [https://api.openai.com]

WITH IDENTITY = 'HTTPEndpointHeaders',

SECRET = '{"Authorization":"Bearer YOUR_OPENAI_API_KEY_HERE"}';

The credential name is the base URL of the endpoint. SQL Server matches it automatically when you pass the @credential parameter. Your API key never appears in query text, execution plans, or query store — it lives in the credential store, protected by the master key.

Your First LLM Call from T-SQL

Here's a minimal working example calling GPT-4o-mini:

sqlDECLARE @response NVARCHAR(MAX);

EXEC sys.sp_invoke_external_rest_endpoint

@url = N'https://api.openai.com/v1/chat/completions',

@method = 'POST',

@headers = '{"Content-Type":"application/json"}',

@payload = N'{

"model": "gpt-4o-mini",

"messages": [

{

"role": "user",

"content": "What is the capital of France? Answer in one word."

}

],

"max_tokens": 50

}',

@credential = [https://api.openai.com],

@response = @response OUTPUT;

-- The response is JSON. Parse it with native T-SQL JSON functions.

SELECT JSON_VALUE(@response, '$.result.choices[0].message.content') AS LLM_Response;

The @response output variable contains the full HTTP response as JSON. The $.result path is where SQL Server puts the parsed response body. From there, standard JSON_VALUE and OPENJSON do the extraction.

A Real-World Example: Automated Error Log Summarization

Let me show you something with actual operational value. SQL Server generates a lot of noise in the error log. Most of it is benign — login failures, backup completions, DBCC output. But buried in there are signals worth acting on.

Here's a stored procedure that reads recent error log entries, sends them to an LLM for triage, and returns a plain-English summary:

sqlCREATE OR ALTER PROCEDURE dbo.usp_SummarizeErrorLog

@HoursBack INT = 1,

@MaxEntries INT = 50,

@ModelEndpoint NVARCHAR(500) = N'https://api.openai.com/v1/chat/completions',

@ModelName NVARCHAR(100) = N'gpt-4o-mini'

AS

BEGIN

SET NOCOUNT ON;

-- Step 1: Pull recent error log entries into a temp table

CREATE TABLE #ErrorLog (

LogDate DATETIME,

ProcessInfo NVARCHAR(100),

LogText NVARCHAR(MAX)

);

INSERT INTO #ErrorLog (LogDate, ProcessInfo, LogText)

EXEC xp_readerrorlog 0, 1; -- 0 = current log, 1 = SQL Server errors

-- Step 2: Filter to the window we care about and build a text block

DECLARE @LogContent NVARCHAR(MAX) = N'';

SELECT TOP (@MaxEntries)

@LogContent = @LogContent +

CONVERT(NVARCHAR(30), LogDate, 120) + N' [' + ProcessInfo + N'] ' + LogText + CHAR(10)

FROM #ErrorLog

WHERE LogDate >= DATEADD(HOUR, -@HoursBack, GETDATE())

ORDER BY LogDate DESC;

IF LEN(@LogContent) = 0

BEGIN

SELECT 'No error log entries found in the specified window.' AS Summary;

RETURN;

END

-- Step 3: Build the LLM payload

DECLARE @Payload NVARCHAR(MAX) = N'{

"model": "' + @ModelName + N'",

"messages": [

{

"role": "system",

"content": "You are a senior SQL Server DBA. Analyze the following SQL Server error log entries. Identify any entries that indicate real problems requiring attention. Ignore routine entries like successful backups, login auditing noise, and informational messages. Return a concise bulleted summary of actionable issues only. If nothing requires attention, say so."

},

{

"role": "user",

"content": "' + REPLACE(REPLACE(@LogContent, '"', '\"'), CHAR(10), '\n') + N'"

}

],

"max_tokens": 500,

"temperature": 0.2

}';

-- Step 4: Call the LLM

DECLARE @Response NVARCHAR(MAX);

EXEC sys.sp_invoke_external_rest_endpoint

@url = @ModelEndpoint,

@method = 'POST',

@headers = '{"Content-Type":"application/json"}',

@payload = @Payload,

@credential = [https://api.openai.com],

@response = @Response OUTPUT;

-- Step 5: Extract and return the summary

SELECT

JSON_VALUE(@Response, '$.result.choices[0].message.content') AS ErrorLogSummary,

GETDATE() AS GeneratedAt,

@HoursBack AS WindowHours;

DROP TABLE #ErrorLog;

END

Run it like this:

sql-- Summarize the last hour of error log activity

EXEC dbo.usp_SummarizeErrorLog @HoursBack = 1;

-- Or go back further for a longer window

EXEC dbo.usp_SummarizeErrorLog @HoursBack = 8, @MaxEntries = 100;

Instead of an analyst wading through hundreds of error log lines, you get a plain-English summary of what actually needs attention. Wire this into a SQL Agent job, send the output to a monitoring table or an email, and you've got a genuinely useful operational tool.

What Models Can You Call?

The proc works with any REST-compatible model endpoint that speaks the OpenAI chat completions format, which covers a lot of ground:

Azure OpenAI (GPT-4o, GPT-4o-mini, o3-mini) via Azure AI Foundry

OpenAI directly (GPT-4o, GPT-4o-mini, o4-mini)

Ollama running locally — fully air-gapped, no data leaves your network

Anthropic Claude via the Messages API

Google Gemini via the REST API

The Ollama path deserves a callout for shops with data sovereignty requirements. Run a local model on your SQL Server host or on a machine in your network, point the endpoint at http://localhost:11434, and your prompts never touch the internet. The performance won't match a cloud-hosted frontier model, but for internal triage and summarization tasks, models like llama3.2 or mistral are more than capable.

The Security and Operational Realities

I'd be doing you a disservice if I stopped at the happy path. Here's what your DBA instincts should be flagging:

Network dependency in the query layer. Every call to sp_invoke_external_rest_endpoint is a synchronous outbound network request. If the endpoint is slow, your query is slow. If it's unreachable, your procedure fails. Design accordingly — don't run this inside a transaction that's holding locks, implement retry logic for transient failures, and set sensible @timeout values.

Data egress. You are sending data from your SQL Server to an external endpoint. If that data includes PII, PHI, financial records, or anything covered by a compliance framework, you need to think carefully before using a cloud-hosted model. Ollama or another self-hosted option solves this completely. If you're using a cloud endpoint, review your data processing agreements.

Cost awareness. LLM API calls aren't free. A SQL Agent job running sp_invoke_external_rest_endpoint on a large result set every 15 minutes can generate real API spend. Monitor token consumption from the start.

The credential model is solid. API keys stored as database-scoped credentials are protected by the database master key, don't appear in query text or execution plans, and can have REFERENCES permissions granted to specific roles. That's a clean security posture. Use it correctly and don't bypass it.

Structured output is your friend. By default, LLMs return free-form text — hard to parse reliably in T-SQL. OpenAI's structured output feature, passed via the response_format parameter in the JSON payload, forces the model to return strict JSON matching a schema you define. For anything you intend to insert into a table or pass downstream, use structured output. It eliminates the brittle string manipulation problem.

The Ecosystem Is Already Moving

One signal I watch for is adoption in trusted tooling. Brent Ozar's First Responder Kit already wired this up. sp_BlitzCache and sp_BlitzIndex now support an @AI = 1 parameter that calls an LLM endpoint directly and returns tuning recommendations inline in the result set. When the most widely used DBA toolset in the community builds on a feature in its first major release cycle, that's a signal the feature is real and here to stay.

Where I See This Going

The obvious near-term applications are operational: log summarization, alert enrichment, anomaly explanation. Things where you already have structured data in SQL Server and want an intelligent narrative layer on top of it.

The more interesting medium-term play is pairing sp_invoke_external_rest_endpoint with the native vector type and DiskANN indexing to build full RAG pipelines entirely inside SQL Server. Your documents, your embeddings, your vector index, and your LLM call — all in one place, no separate orchestration layer, no Pinecone account, no Python middleware. For the shops I work with, that single-platform story is compelling.

The question I'd encourage every DBA to ask right now isn't "should we use this?" — it's "what problem do we already have that this solves?" Start there, build something small, and let the use cases emerge from actual operational pain rather than from the hype cycle.

Thirty years in, I still occasionally see something in SQL Server that makes me stop and rebuild my mental model of what the platform is. This is one of those things.