Design agents visually, compose them with tools and subagents, trace every execution step, then validate with automated evaluation and human expert review — a complete platform for building and proving your AI agents.
Real-World Scenario
PrecisionCare Biomedicals is a pharmaceutical company with a comprehensive knowledge base of 300+ documents spanning SOPs, regulatory filings, clinical trial reports, and training materials.
They need to build intelligent AI agents that help employees find answers, check regulatory compliance, and navigate complex procedures. But they also need to compose agents with enterprise tools, trace every decision, and systematically validate that their agents work correctly.
Here's how they use Sharely Agents to design, compose, trace, evaluate, and continuously improve their AI agents.
Enterprise Knowledge Base
300+
Documents
6
Categories
GPT-4o
AI Model
12 SMEs
Reviewers
Visually design agents, compose them with tools and subagents, and trace every execution step across PrecisionCare's regulatory workflows.
Index and classify real user conversations. Visualize question types, answer quality, and trends across PrecisionCare's deployed agents.
Build evaluation datasets from real conversations. Auto-generate expected answers and run automated evaluations with AI-judged metrics.
Invite PrecisionCare's scientists and regulatory specialists to review AI answers with a mobile-first review interface.
PrecisionCare doesn't just connect single search nodes to their agent. They compose tools as workflows — each tool is a mini-flow with conditional logic, specialized retrieval steps, and result merging. The agent's model decides which workflow-tool to invoke based on the question and its current state.
PrecisionCare's Regulatory Compliance Agent has two workflow-tools registered: a Regulatory Search tool (SOPs, FDA filings) and a Clinical Trials Search tool (protocols, phase reports). The model decides which tool to call based on the user's question — and can call both iteratively to build a complete answer.
What would you like to do with your flow?
Validate Flow
Check for missing connections, configuration issues, and best practices
Modify Flow
Get help adding nodes, changing prompts, adjusting retrieval settings, or rewiring connections
Test Flow
Run your agent with a question and see live execution on the canvas
Explain Flow
Understand what each node does and how data flows through your agent
Each tool connected to the agent is itself a workflow. Here’s the inside of PrecisionCare’s Regulatory Search Tool — it classifies the query by document category (SOP, Regulatory Filing, or Clinical Protocol), routes to specialized search nodes with different RBAC roles and top-K settings, then merges and ranks the results before returning them to the agent.
Used by: Regulatory Compliance Agent · Invoked when the model needs regulatory or SOP information
The agent doesn't know or care about the internal complexity of its tools. It sees a tool with a name and description (“Search SOPs, regulatory filings, and FDA guidance documents”) and decides when to call it based on the user's question and the conversation state. When the agent invokes the tool, the workflow executes — classifying the query, routing to the right search branch, and returning merged results. The agent then uses those results to continue reasoning.
When the agent handles a question, PrecisionCare can inspect the full execution trace — the agent’s reasoning, its decision to invoke a workflow-tool, the workflow expanding into its internal steps (classification, branching, search), and the agent continuing with the results. Every step across both levels is visible.
Run #1,847 · Regulatory Compliance Agent v2.1 · 2.8s total
3
LLM Calls
1
Tool Invocations
4
Workflow Steps
4,216
Total Tokens
2.8s
Total Time
What are the storage requirements for reconstituted mAb samples per SOP-042?
This question is about SOP storage requirements. I should search the regulatory knowledge base for SOP-042. → Decided to call: Regulatory Search Tool
Invoked with query: "storage requirements reconstituted mAb samples SOP-042"
Classified as "SOP" (confidence: 0.96) → routing to SOP Search branch
Role: Quality Assurance, Top-K: 10 → 10 chunks retrieved, top score 0.94 (SOP-042 Section 4.3)
De-duplicated 10 results → 8 unique chunks, ranked by relevance. Top: SOP-042 §4.3 (0.94), SOP-042 §4.1 (0.89)
Returned 8 ranked results with source citations to the agent
Found SOP-042 §4.3 with storage requirements. I have enough information to answer the question directly. → Generating final response.
Per SOP-042 §4.3: reconstituted monoclonal antibody samples must be stored at 2-8°C (refrigerated) and used within 24 hours of reconstitution. Samples must not be frozen after reconstitution. Storage must be in amber glass vials to protect from light degradation.
PrecisionCare's AI agent handles hundreds of questions weekly from lab technicians, regulatory staff, and quality engineers. Sharely indexes every conversation and classifies each Q&A by type and quality.
Sharely automatically pulls conversations from PrecisionCare's deployed AI agents. Each Q&A pair is classified by question type (FAQ, Technical, How-To, etc.) and answer quality (Answered, Partial, Failed, etc.).
Classifying 2,841 conversations from PrecisionCare agents...
2,841
Total Q&As
2,756
Classified
9
Question Types
5
Quality Levels
PrecisionCare's quality team sees conversation volume over time, question type distribution, and a heatmap crossing question types with answer quality — immediately revealing that Technical and Troubleshooting questions have the highest failure rates.
Click any cell in the heatmap to see the actual Q&A pairs. PrecisionCare notices 74 Technical questions that Failed — they create an evaluation dataset directly from this segment to systematically fix the issues.
74 items
Questions will be imported without expected answers. Generate expected answers using AI from the dataset page.
| Date | Space | Question | Answer | Type | Quality |
|---|---|---|---|---|---|
| Mar 10 | Lab Operations | What is the required temperature range for storing reconstituted mAb samples per SOP-042? | I'm sorry, but I couldn't find specific temperature requirements for reconstituted mAb samples... | Technical | Failed |
| Mar 9 | Quality Assurance | What are the acceptance criteria for endotoxin testing on batch release? | I don't have information about the specific acceptance criteria for endotoxin testing... | Technical | Failed |
| Mar 8 | Regulatory Affairs | What is the shelf life for lyophilized drug product per the stability protocol? | I'm unable to find the shelf life specifications for lyophilized drug product... | Technical | Failed |
| Mar 7 | Clinical Research | How do we handle protocol deviations during Phase II trials per SOP-118? | I'm sorry, but I don't have the information on protocol deviation handling procedures... | Technical | Failed |
PrecisionCare creates evaluation datasets from the failed conversations they identified. They auto-generate expected answers using their knowledge base + a strong AI model, then run automated evaluations to measure improvement.
The 74 failed Technical questions are imported without expected answers. PrecisionCare uses RAG retrieval + Claude Sonnet 4.5 to auto-generate gold-standard expected answers from their knowledge base.
74 questions · Imported from Conversation Insights
Use RAG retrieval + a strong model to generate ideal expected answers for 74 questions.
What are the acceptance criteria for endotoxin testing on batch release?
What is the required temperature range for storing reconstituted mAb samples per SOP-042?
What are the acceptance criteria for endotoxin testing on batch release?
What is the shelf life for lyophilized drug product per the stability protocol?
How do we handle protocol deviations during Phase II trials per SOP-118?
PrecisionCare runs the 74 Technical questions against their RAG agent. Each question is answered by the agent, then judged by an AI evaluator on relevance, groundedness, correctness, and similarity to the expected answer.
Dataset: Technical - Failed (Mar 2026) · 74 questions · PrecisionCare RAG Agent
0.72
Relevance
0.81
Groundedness
68%
Correctness
0.65
Similarity
1.2s
Avg Latency
0.74
Source Recall
| # | Question | Relevance | Grounded | Correct | Similarity | Latency |
|---|---|---|---|---|---|---|
| 1 | Required temperature range for reconstituted mAb samples per SOP-042? | 0.85 | 0.92 | 0.78 | 1.1s | |
| 2 | Acceptance criteria for endotoxin testing on batch release? | 0.45 | 0.60 | 0.32 | 1.4s | |
| 3 | Shelf life for lyophilized drug product per stability protocol? | 0.72 | 0.88 | 0.71 | 0.9s | |
| 4 | Protocol deviation handling during Phase II trials per SOP-118? | 0.90 | 0.95 | 0.85 | 1.0s |
Automated metrics reveal that 32% of PrecisionCare's Technical questions are still failing. Before deploying fixes company-wide, they invite their scientists, regulatory specialists, and quality engineers to validate the AI's answers with human judgment.
The quality team creates a human evaluation campaign, configuring how many evaluations each knowledge chunk should receive and which AI model to test.
Human Evaluation Campaign
Validate AI answers across regulatory and clinical documents
Total Reviewers
12
Active Reviewers
8
Flagged Chunks
47
Answer Correctness
84%
200
High
94
Medium
160
Low
The admin invites SMEs — scientists, regulatory specialists, and quality engineers — to review the AI-generated content. Each reviewer gets a unique, tokenized link.
Limit how many items this reviewer can evaluate
This workspace has RBAC enabled. Select which roles this reviewer should have access to.
Dr. Sarah Chen
s.chen@precisionbio.com
James Rodriguez
j.rodriguez@precisionbio.com
Dr. Aisha Patel
a.patel@precisionbio.com
Michael Torres
m.torres@precisionbio.com
Lisa Wang
l.wang@precisionbio.com
Each SME receives a branded email with a unique, tokenized link. No login required — they click and start reviewing immediately.
noreply@sharely.ai
To: s.chen@precisionbio.com
You're invited to evaluate AI content for PrecisionCare Biomedicals
Hi Dr. Sarah Chen,
The Quality Team has invited you to participate in the PrecisionCare Biomedicals - Q1 Audit evaluation campaign for PrecisionCare Biomedicals.
Your feedback will help improve the quality of AI-generated questions and answers based on the knowledge base.
Or copy and paste this link into your browser:
https://agents.sharely.ai/review/eyJhbGciOi...kT3dFz
This link is unique to you. Please do not share it with others.
Powered by Sharely.ai
Subject matter experts review AI-generated questions from the knowledge base, rate their quality, then evaluate the AI's answer for correctness. The mobile-first interface makes it easy to review from anywhere.
Welcome Screen
Thank you for helping us improve our AI knowledge base.
Campaign
PrecisionCare Biomedicals - Q1 Audit
Organization: PrecisionCare Biomedicals
Rate the Question
Question
What is the required temperature range for storing reconstituted monoclonal antibody samples according to SOP-042?
Rate this question
Generating AI answer...
Rate the Answer
Question
What is the required temperature range for storing reconstituted monoclonal antibody samples according to SOP-042?
Rate this question
AI Answer
According to SOP-042 Section 4.3, reconstituted monoclonal antibody samples must be stored at 2-8°C (refrigerated) and used within 24 hours of reconstitution. Samples must not be frozen after reconstitution.
Rate this answer
See at a glance which of your 300 knowledge resources have been evaluated. The heat map reveals coverage gaps, quality distribution, and flagged content across your entire knowledge base.
300
Total Resources
37% have questions
1356
Total Questions
454 evaluated
200
High Quality
44% of evaluated
47
Flagged
Needs attention
Filter and explore evaluation results at the individual resource level. See which documents have quality issues, review flagged content, and export results for regulatory audits.
| Resource | Questions | Evaluated | Quality | Flagged | Status |
|---|---|---|---|---|---|
SOP-007 Cold Chain Management | 7 | 2/7 | 3 | In Progress | |
SOP-009 Environmental Monitoring | 17 | 8/17 | - | In Progress | |
SOP-011 Change Control | 13 | 9/13 | - | In Progress | |
SOP-063 Bioassay Validation v2 | 4 | 2/4 | - | In Progress | |
SOP-069 Environmental Monitoring v2 | 17 | 13/17 | - | In Progress | |
SOP-070 Deviation Handling v2 | 20 | 13/20 | - | In Progress |
By connecting conversation analytics, automated evaluation, and human expert review, PrecisionCare built a complete feedback loop that continuously improves their AI agent.
PrecisionCare built a Regulatory Compliance Agent that orchestrates 3 subagents and 6 tools, handling complex compliance workflows without custom code.
Conversation insights revealed that Technical and Troubleshooting questions had the highest failure rates. Targeted datasets let them systematically fix these gaps.
Full execution tracing plus human validation from 12 SMEs provided the evidence needed for compliance audits and company-wide deployment.
PrecisionCare designs agents, traces executions, evaluates with automation and humans, then iterates — a complete lifecycle for AI improvement.
Design agents visually, compose with enterprise tools, trace every decision, then validate with automated evaluation and human experts. Get started in minutes.
Get Started Free