On August 26, 2025, OpenAI gave developers twelve months of notice: the Assistants API would shut down on August 26, 2026, and every request to it would stop working. For a lot of companies that notice became one more ticket on a crowded roadmap. For 211 San Diego, it put a date on the tool its operators use, every day, to connect people with food, housing, and healthcare.
This is the story of how Oktana helped 211 San Diego move that AI-powered referral search to the OpenAI Responses API before the cutoff, and how the team used the migration to fix a quieter problem along the way: search results that were coming back incomplete.
Why search quality matters so much at a 211 line
211 San Diego is a local nonprofit and the region’s trusted source for information and connections to community, health, and disaster resources. Help is available 24 hours a day, 365 days a year, in more than 200 languages. The organization maintains a database of more than 6,000 services and resources that are updated in real time, and its Community Connectors rely on that information to help San Diegans find what they need.
Those Community Connectors work inside Salesforce, where an AI-powered referral search helps them match each caller with nearby services. In most business software, a search that misses a few results is an annoyance. At a 211 line, the resource that does not show up might be the closest food pantry, the shelter with an open bed, or the clinic that speaks the caller’s language. Completeness is the whole job.
Two problems arriving at the same time
The first problem had a calendar date attached. The referral search had been built on the OpenAI Assistants API. Once OpenAI reached feature parity in the Responses API, it deprecated Assistants and set the shutdown for August 26, 2026. Staying put was never an option, because on that day the search would simply go dark.
The second problem was harder to see. Operators were getting incomplete resource results, and the root cause sat in the data. The datasets behind the search were unstructured, which made it difficult for the model to reliably retrieve every relevant service for a given query. A straight port to the new API would have carried that weakness over intact.
So the goal became twofold: keep the search running on a supported foundation, and make its answers more complete, all without asking operators to change how they work.
What Oktana changed, and what stayed exactly the same
The guiding principle was simple. Operators should notice better results and nothing else. That meant preserving the existing Salesforce search experience, the ranking logic, and the response structure, while rebuilding what sat underneath. The work ran across five fronts.
1. Migrating to the OpenAI Responses API
Moving off Assistants is more involved than swapping an endpoint. OpenAI’s own migration guidance maps the old building blocks to new ones, with assistants becoming prompts, threads becoming conversations, and runs becoming responses. Oktana migrated the integration to the Responses API and kept the output in the shape the Salesforce experience already expected, so nothing downstream had to be rewritten.
2. Optimizing the Apex integration logic
The connection between Salesforce and OpenAI lives in Apex. The team reworked that integration logic for the new API, which also gave 211 San Diego a cleaner and more maintainable codebase for whatever comes next.
3. Restructuring the search data for retrieval
This is where the completeness problem got solved. Oktana restructured the XML search datasets so that resources are described in a consistent, well-defined way the model can retrieve accurately. When the information is organized for retrieval, the model has far less room to overlook a relevant service.
4. Proving it with benchmark-driven A/B testing
“It looks better” is a weak standard for a system people rely on in a crisis. The team implemented A/B testing frameworks that compare search output against benchmark datasets, validating whether the search returns the exhaustive set of resources it should. Every improvement could be measured before it reached operators.
5. Documenting for long-term governance
Finally, Oktana documented the AI evaluation approach and the API framework. That documentation gives 211 San Diego a governance base for future model updates and API changes, so the next migration starts from a known baseline.
The results
- Critical AI search functionality migrated before the OpenAI Assistants API shutdown.
- Improved search result completeness and resource retrieval accuracy.
- A benchmark-driven A/B testing framework for validating AI responses.
- XML-based search datasets optimized for structured retrieval.
- Long-term governance through AI evaluation and API framework documentation.
Just as important is what did not happen. Operators kept the same search experience they already knew, and the people calling 211 San Diego kept getting help without interruption. You can explore more projects like this one in our customer success stories.
Four questions to ask about your own AI in production
The specifics here belong to 211 San Diego, but the pattern shows up in almost every organization that put generative AI into production over the last two years. If your team runs AI features on Salesforce or anywhere else, these four questions are a good place to start.
- Which of your AI features depend on an API with a published end date? Model and endpoint deprecations are now routine. An inventory of what calls what, and on which version, turns a surprise into a plan.
- Do you know what a complete answer looks like? Without a benchmark dataset, nobody can say whether a change made results better or worse. Building one early is usually the most useful step in the whole project.
- Is your data shaped for retrieval? Many accuracy problems blamed on the model are really data problems. Structuring the source content can do more for answer quality than switching models.
- Who owns the documentation when the next change arrives? AI systems keep evolving after go-live. Written evaluation methods and integration notes are what let a team move quickly the next time.
Why this matters for Salesforce teams
More and more AI work runs through Salesforce, whether that means calling external models from Apex, grounding answers in CRM data, or rolling out Agentforce. The discipline is the same in each case: protect the experience people depend on, get the data into shape, and measure before you claim success.
Oktana is a Salesforce Summit Consulting Partner with deep AI engineering experience across model providers such as OpenAI and Anthropic. Our teams build, migrate, and evaluate AI capabilities inside the platforms our clients already use, from Agentforce implementation to custom LLM integrations like the one 211 San Diego relies on.
Running AI on an API that is about to change?
If you have AI features built on endpoints or models with an end date, or search and agent results you are not fully confident in, we would be glad to help you map the path forward. Talk to the Oktana team.
Frequently asked questions
What happened to the OpenAI Assistants API?
OpenAI notified developers of the Assistants API deprecation on August 26, 2025 and shut it down on August 26, 2026. OpenAI recommends the Responses API for all new and migrated integrations.
How is the Responses API different from the Assistants API?
The Responses API is OpenAI’s replacement for Assistants and reached feature parity before the deprecation. In OpenAI’s migration mapping, assistants become prompts, threads become conversations, and runs become responses.
Can OpenAI models be integrated with Salesforce?
Yes. In the 211 San Diego project, Apex integration logic connects the Salesforce search experience to the OpenAI Responses API, and results are returned in the same structure operators already use.
How do you measure whether an AI search returns complete results?
Oktana uses benchmark datasets that define the expected set of results for a query, then runs A/B tests that compare search output against that benchmark. This shows whether a change improves completeness before it reaches users.
Why does data structure affect AI search accuracy?
A model can only retrieve what it can reliably find. Restructuring unstructured content into consistent, well-defined datasets, such as the XML datasets used for 211 San Diego, makes relevant records easier to retrieve and reduces missed results.