Building Solutions Using Agentic AI Tools

 

 

We are living in a world today where AI agents help us with most aspects of our lives. Like most amazing technologies, they quickly move from being novelty to utility. When creating software, I have identified a few patterns that are particularly helpful. This is an attempt to memorialize those findings, recognizing that my understanding of this topic will likely evolve in the future. 

I generally use the Databricks extension in Visual Studio Code. Yes, I know there are other tools out there, but for me, it works. I do leverage multiple agents, not just Copilot. It does a reasonable job, especially with recent releases. 

In this blog, I will examine the importance of traditional software development methodologies, such as requirements, planning, and the use of unit tests.

Requirements: Documentation for Agents

First, you must utilize requirements documentation. Like traditional software development, you need to know what you must build and stop once you reach it. Reach the requirements and no more. The requirements should be in Markdown and structured in a way that is clear to humans as well as our AI friends.   

You should include elements, such as a problem statement, a value proposition, a differentiation of the project from other solutions, and both functional and non-functional requirements. State the requirements as concisely as possible, leveraging your AI tools to assist in drafting the document. Tell the agent the purpose of the document.  

Visualizing Code for Agents 

If you include images, make sure that you create them in a textual format, such as Mermaid or SVG. This enables the agent to understand the diagram. It may not be as pretty as PNG, but it is more easily parsed. 

This is a pattern that I have been following lately. Rather than asking an LLM to create the finished product, I ask it to generate the script that will ultimately produce the final product. I can then fix that script, adapt it to my needs, and (most importantly) those scripts convey information that is often useful to LLMs. 

You should include any metrics and success criteria in this document. The metric, along with a goal, will help the system during the design process. We must always track the business value our products are generating. Clearly stating that value during the initial stages is essential. 

AI Agents can be a bit messy. Ensure you instruct them to place the documentation in a folder named 'docs.' While you will likely want a README Markdown file in each directory (summarizing the contents thereof), it is a good idea to keep the other documents in one location. Instruct the agents to place them in that location and to review those documents as they perform their tasks, such as planning and execution. 

Plans: Decomposition and Chain of Thought 

Now that we have a high-level requirement document, you should have your agent help you develop a plan to achieve the desired outcome. Asking the agent to create a plan to reach a minimal viable product (MVP) is a great way to start off. Remember to give your agent guidance on how you want your solution to be built. If you prefer an object-oriented approach, then state that in the agent’s instructions. 

Ensure you instruct the agent to break the goals into smaller pieces. By performing this decomposition and having the agent describe its actions, you may achieve better outcomes. Having an agent explain what it is going to do is a known LLM approach. 

This is like the design and project planning that we used to do when projects took many months to complete. Asking your agent to decompose the problem and explain the phases is a great way to start. Remember, it is a partnership between you and the agent. You must review and direct its actions. 

Testing: TDD and Unit Tests 

As important as creating code is ensuring the code is doing what you want it to do. Ideally, you would follow a test-driven development (TDD) methodology. However, making sure that unit tests are created immediately after implementation is completed is a reasonable alternative. Ensure you instruct your agent to create unit tests. Put that in the agent’s instructions. Even with those instructions in place, you may have to remind the agent to create them after adding new or changing existing functionalities. 

Remember, unit tests serve a few purposes. One is to ensure that the requirements are met by the implementation. Another is to test edge conditions or flows that are infrequently encountered.  

You should also read the code that the agents write. Do not accept it as is. If they are misunderstanding the goal, then they might write the code incorrectly and then test it with that same misunderstanding.  

One alternative is to have a different agent create unit tests and a different one to produce the actual implementation. This is akin to not letting a developer test their own code, at least not where that is the only testing of that code that occurs. 

The flow between agents and humans can be complex. As you can see in this sequence diagram, the flow with the human drafting the requirements document. What is not shown is that we often use AI to help create that document. It is essential that you read the document carefully. Sometimes, generated text looks good when you skim over it, only to find the details are problematic. The back and forth with the agents is critical. That said, I am sure we have all had conversations that made the agent forget.

Summary

Given these pillars, you can refine the project's goals, create a plan, and follow it to produce code, then validate that the code aligns with the requirements at each phase. Ideally, break the work into a few manageable chunks.

  In this blog , I have discussed how software engineering skills and processes are essential. While velocity is considerably higher, we must still be concerned with design, planning, testing, and finally, user acceptance testing. Without proper processes, you will struggle to meet quality standards.