Last year, AI showed up in almost every product conversation.

It’s the shiny new toy. AI copilots and that now-ubiquitous sparkle icon ✨ are popping up across most of the apps I use.

Some integrations genuinely save me time. Many don’t. It’s usually obvious which ones were built to demo well. They look impressive in a pitch, but once you actually use them, the novelty wears off. Often times, I’ve ended up replacing their built-in feature with my own LLM that understands my workflows, writing preferences, and long-term context.

For every AI product that delivers real value, there are just as many that miss the mark.

The failure is not the technology itself. It’s formalizing AI behavior before understanding how people actually work, make decisions, and build trust. In many cases, AI is asked to act before it had earned the right to.

As we worked through our strategy and product roadmap in 2025, we were tempted many, many times to build AI directly into the product. Each time, we paused on the same question… were we ready to formalize it yet?

Hardcoding AI directly into clinician and coach workflows would have forced us to define prompts, UX patterns, and system behavior before we understood how people would actually use the tool. That felt premature.

Instead of jumping into a direct integration, we took the lean path: learn first, then automate.

Creating a unified view on patient medical history, RPM data, and clinician notes

So instead of automating prompts and workflows, we automated the data view.

For each patient, we created a secure data file that brought together a full view of their health: recent doctor visits, lab results, medications, RPM readings, coaching notes, session transcripts, and longitudinal goals.

We gave our clinicians access to a HIPAA-compliant LLM where they could upload the patient’s file and ask questions while preparing for (or during) conversations.

They could ask it anything:
When was the last time we talked about nutrition?
What motivational barriers came up recently?
Did they ever get that mammogram we discussed?
What was their granddaughter’s name?

Coaches and providers were able to create their own prompts and use the tool in ways that actually supported their day-to-day work, giving them flexibility to prepare more thoughtfully and deliver more personal care.

They also used it to generate patient health progress reports that combined RPM trends with qualitative coaching insights, which we shared with the patient’s primary care team.

Using clinician prompts as product research

This approach avoided prematurely hard-coding AI into the product, which respected their existing workflows, and gave them the freedom to explore prompts and use cases we may not have uncovered otherwise. Those prompts became a form of research.

This gave us early insight into usage patterns: which questions kept coming up, where people needed synthesis rather than retrieval, and what context was missing when answers fell short.

Instead of guessing what to build or defining workflows based on assumptions, we were were able to learn directly from real behavior. Over time, this gave us a much stronger foundation for deciding what should eventually be productized.

Why we avoided deep AI integration into our care management platform

In an AI-heavy world, the fastest way to lose flexibility is to overbuild before you understand where the value actually is. By keeping the setup lightweight, we avoided locking ourselves in too early.

Clinicians did not have to wait on engineering cycles to experiment. Product could learn from real usage and move toward deeper integration deliberately, not by default.

Just as importantly, this kept us flexible as models continued to change. Instead of building around a specific model or prompt, we built around the data. That reduced long-term risk in a space where things change quickly.

We put one foot in and one foot out: enough structure to be useful and safe, without over-committing before we had evidence.

Where the impact showed up

Patients benefited from this approach even if they never saw the AI directly. Coaches showed up more prepared. Conversations felt more continuous. Important details were less likely to be missed.

Over time, this creates a responsible path toward deeper AI-powered experiences… grounded in real usage rather than assumptions.

I am not anti-AI. I am anti committing too early.

AI will keep changing. Models will come and go. What lasts, and what we need to keep getting right, is a deep understanding of real workflows, real context, and real user needs. That is what earns the right to automate.

Practical Implementation Notes for Product Teams

The most important decision we made was about the data itself. The patient file was designed to be compact and human-readable. If a coach could skim it and quickly find medications, recent encounters, labs, device readings, notes, and goals, a model usually could too.

Principles we followed

• Make data the foundation. When outputs looked off, we resisted the urge to write a smarter prompt. We improved the data instead. Better context consistently produced better answers across many different questions.

• Prefer plain text over heavy structure. Most of our patient data lives in FHIR, exported as JSON. For patients with very large medical records, that structure was token-heavy and often required explaining the schema inside the prompt (which we were trying to avoid). We transformed and flattened the data into plain text or light markdown, so the model used fewer tokens and produced more accurate, reliable responses.

• Design for many prompts, not one perfect prompt. We did not aim for a single “right” question. Coaches asked different questions at different moments. As the data improved, the same prompts produced better results, which was the signal we cared about.

• Keep experimentation visible. Prompt usage was not hidden. We reviewed patterns, talked openly about what worked, and adjusted the data accordingly. Learning was shared and not siloed.