Choose a unit of work you can inspect

An AI project becomes easier to evaluate when it begins with a sentence such as: ‘When a service request arrives, prepare a summary and identify the missing information before a coordinator responds.’ That sentence identifies a trigger, an output, and a person who can judge the result. ‘Add AI to customer service’ leaves all three unresolved.

Follow a real task from arrival to completion. Record where someone searches for information, copies it between systems, interprets messy text, checks a result, or waits for another person. Ask the people doing the work which step creates the most friction. A polished assistant will accomplish little if the delay is actually an approval nobody owns.

Possible starting points include classifying incoming requests, extracting fields from documents, preparing an internal briefing, or finding an applicable procedure. These are candidate workflows, not claims about results we have delivered for a particular client.

Measure completed work, including the checking

Establish a baseline before building. Count how often the task occurs, the hands-on time it consumes, and the corrections or escalations it creates. Separate active work from elapsed waiting time. Faster drafting may save effort without shortening the customer's wait if the same approval still takes two days.

Consider a hypothetical team handling 240 requests each month. Saving four minutes on the initial draft creates 16 hours of gross capacity. If checking and correcting each draft takes two additional minutes, the net capacity is eight hours. At an assumed internal labor cost of $40 per hour, that represents $320 of monthly capacity before software, support, and implementation costs. Those assumptions are illustrative; they are not a forecast or a customer result.

Capacity is not automatically cash savings. Decide how the recovered time would be used: accepting more work, reducing overtime, improving response quality, or freeing a specialist for harder cases. Budget separately for integration, data cleanup, staff training, monitoring, and ongoing ownership. An inexpensive model call can sit inside an expensive operating process.

Inspect the information the system would need

Collect a representative set of inputs and the corresponding accepted outputs. Include ordinary cases, incomplete requests, contradictory documents, and examples that required an experienced employee. Determine which information changes frequently and who maintains it. A folder of obsolete procedures will undermine an assistant regardless of the model you choose.

Check whether the intended users are allowed to access each source and whether the application can enforce those boundaries. Also identify retention requirements, deletion paths, and data that should stay out of the pilot. NIST's AI Risk Management Framework calls for understanding the intended use, context, and human oversight. For a small team, a short written ownership and data map is a practical starting point.

Compare candidates before choosing the most impressive demo

Use the same questions for every idea. The table below is a decision aid we recommend, rather than a universal scoring standard. Prefer a candidate whose weak points can be tested cheaply. A task with enormous theoretical value may be a poor first project if nobody can recognize a correct answer.

QuestionA promising answerA reason to investigate first
Does it happen often?A recurring queue with visible demandA rare task with little reusable work
Can someone judge the output?A reviewer and clear acceptance criteriaSuccess depends on an undefined feeling
Is the information available?Maintained sources with clear accessConflicting records and unclear ownership
What happens after an error?An editable draft or reversible actionAn immediate commitment that is hard to undo
Will anyone use it?An owner willing to test in the workflowA separate tool nobody plans to open

Match the mechanism to the job

A fixed calculation belongs in ordinary code. A straightforward routing rule may belong in automation. Finding a specific record may call for better search. Generative AI becomes interesting when the work involves interpreting varied language, synthesizing evidence, or preparing a draft that benefits from context. Anthropic's guidance on building agents recommends starting with the simplest approach that meets the need and adding complexity when evidence warrants it.

LAYIQ illustrates how different mechanisms can coexist. PostgreSQL holds durable product records, OpenSearch retrieves opportunities, and LightGBM supports preference-based recommendations. Its Fit Analysis workflow separately evaluates candidate and role evidence. These verified capabilities show a division of responsibilities, not a requirement that every business adopt that entire stack.

Write the decision the pilot must support

Define the smallest trial that answers your uncertainty. Run representative work through the current process and the proposed process. Include review time, correction rate, useful completions, and operational cost. Agree in advance what would justify continuing, changing direction, or stopping. Keep the first release's authority narrow enough that a failed trial remains manageable.

For the hypothetical service-request assistant, a coordinator could review every draft before sending. The pilot should show whether summaries preserve essential facts, missing-information prompts are relevant, and total handling effort improves. An attractive demo with invented requests cannot answer those questions.

Bring a potential development partner a sample input, an accepted output, current volume, the systems involved, and the decision you want to make. That is enough to start a useful discovery conversation. The next deliverable should be a testable plan with explicit assumptions and ownership, followed by evidence from the actual workflow.

Sources & further reading

Primary documentation supporting the technical guidance in this article.