1. Choose a task with a clear owner
Start with a repeatable task that has an identifiable process owner, a manageable set of inputs, and a result someone can review. Ticket summarization and routing can be a candidate when a support team spends time sorting similar requests.
Avoid beginning with a broad goal such as “automate support.” Name the exact step: for example, prepare a summary and suggested category for a new request.
2. Write down today’s baseline
Before building, sample the current workflow. Record how long the task takes, how often work is corrected or rerouted, and which exceptions require a person. Agree who will collect the observations and how many examples are enough for a useful comparison.
Keep the baseline and pilot measures consistent. A faster draft is not useful if people spend the saved time correcting it.
3. Define what the AI may do
List the data the workflow can read, the outputs it may prepare, and the actions it may request. Start with suggestions and drafts. Require an authorized person to approve changes to customer records, external messages, or other important systems.
Write down stop conditions too: missing information, low confidence, an unfamiliar request, or a policy exception should lead to a clear human handoff.
4. Verify the real access path
Check the model provider, connector permissions, network route, data retention terms, and credentials in the customer’s intended environment. A connector that passes mocked tests is not the same as a live connection verified against the customer’s account.
Use synthetic or approved test data until the customer has reviewed the full data path. Confirm how secrets are stored, who can approve actions, and how the workflow can be stopped.
5. Test failures as deliberately as successes
Try incomplete inputs, duplicate requests, unavailable services, unexpected formats, and rejected approvals. Confirm that the workflow fails visibly, avoids unintended writes, and provides enough context for a person to continue the task.
Keep a set of representative examples and expected outcomes. Re-run them after changes to the prompt, flow, model, or connector.
6. Agree on the pilot decision
At the end of the pilot, compare the same measures recorded at baseline. Review output quality, correction rate, approval rate, exception handling, and operational effort. Decide whether to stop, adjust, or expand based on the evidence and the customer’s risk requirements.