Sometimes your “experiment” fails, then you pivot your work slightly and this other experiment was a much better success.
Precisely for that reason, before When designing our final solution, we need to start simple and cover our risks.
- Define a “budget” or deadline. Let's see what we can do in x weeks and then decide how or if to continue. Typically 2-4 weeks will be enough to understand basic PoC. If it looks promising, continue investing resources to improve it.
- Experiment-Whether you choose a bottom-up or top-down approach to your experimentation phase, your goal is to maximize the rate of succession of results. At the end of the first iteration of experimentation, you should have some proof of concept (that interested parties can play with) and a baseline that you've achieved.
- Retrospective— At the end of our research phase, we will be able to understand the feasibility, limitations and cost of creating such an application. This helps us decide whether to produce it and how to design the final product and its UX.
- Productization — Develop a production-ready version of your project and integrate it with the rest of your solution by following standard SWE best practices and implementing a data and feedback collection mechanism.
To implement the experiment-oriented process well, we must make an informed decision about how to approach and build these experiments:
Getting started with Lean: the bottom-up approach
While many early adopters quickly jump on” The last” multichain agentic systems with full Langchain or something similar, I found “The bottom-up approach” often produces better results.
Start lean, very thinhugging the “one message to rule them all” philosophy. Although this strategy may seem unconventional and will probably produce poor results at first, it establishes a base for your system.
From there, continually iterate and refine your cues, employing cue engineering techniques to optimize results. As you identify weaknesses in your lean solution, break up the process by adding branches to address those deficiencies.
As I design each “leaf” of my LLM workflow graph, or native LLM architecture, I follow The Magic Triangle³ to determine where and when to cut branches, split them or thicken roots (using quick engineering techniques) and squeeze more lemon.
For example, to implement “native SQL queries” with the bottom-up approach, we will start by naively sending the schemas to the LLM and ask it to generate a query.
This usually does not contradict the “top-down approach”, but rather serves as one more step before it. This allows us to show quick profits and attract more investments in projects.
The Initial Big Picture: The Top-Down Strategy
“We know that the LLM workflow is not easy, and to achieve our goal, we will probably end up with some native LLM workflow or architecture.”
The Top-Down approach recognizes this and starts by designing the native architecture of the LLM from day one and implementing its different steps/chains from the beginning.
This way, you can test your workflow architecture as a whole and squeeze the whole lemon instead of refining each sheet separately.
For example, to implement “native SQL queries” with the top-down approach, we will start designing the architecture before we even start coding and then move on to the full implementation:
Finding the right balance
When you start experimenting with LLM, you'll probably start at one of the extremes (top-down too complicated or one-shot super simple). In reality, there is no such winner.
Ideally, you will define a good SoP¹ and model it for an expert before coding and experimenting with the model. Modeling is actually very difficult; Sometimes, you may not have access to such an expert.
I found it difficult to get a good architecture/SoP¹ on the first shot, so it's worth experimenting a little before jumping into the big guns. However, that does not mean that all must be too skinny. If you already have a prior understanding that something HAS TO break into smaller pieces, do it.
In any case, you should leverage the Magic Triangle³ paradigm and model the manual process correctly as you design your solution.
