It’s stuck with me for a couple of reasons. One, for a long time, agency has seemed obvious to me. Agency looks like determination. It looks like action. It looks as clear as the color blue. Two, I had considered agency to be binary. You have it or you don’t. Or maybe more specifically, you’re using your agency or you’re not. But looking at the agency triangle, I realized agency is more of a spectrum — the more of each tenet of agency that is present, the more agentic one can be. The trick then becomes how can we activate in these three ways? And for leaders, how can we activate it in others?
As a leader, one of the most leveraged things we can do is spot agency and nurture it. Our job is to help other leaders achieve growth towards autonomy at their level of impact and then push them to grow to the next level of impact. People are agentic up to a point and then they need support. This support can be a foundation to enable the growth. Or it can be amplification to lift and share the impact with others.
“I will figure it out.” We all need to believe we’re capable of something we have not done before. When we see the pieces that comprise the whole of a skillset, we should be calling it out, authentically. And authentically is the key. To give people false confidence is a sin in and of itself.
“I can fix it if it breaks.” Is this a Type 1 or Type 2 decision? Reminding your team members of the many, many decisions we make that are reversible is a key perspective and challenge you can bring to your discussions. If your team member breaks something, give them space to fix it.
“I am willing to be misunderstood.” Give your team members space to share their mental models. Give them the space to refine them. And when your team member takes a novel idea to the organization, support them in the rooms and forums they are not in.
“I will figure it out.” “I can fix it if it breaks.” “I am willing to be misunderstood.” All facets of the same — agency is agency is agency. Identifying these facets and then nurturing the agency in your org is one of the most leveraged ways to grow your impact over a long horizon. Besides, there’s nothing quite like seeing your team members take on new big things and be successful in the process.
]]>There were three places I considered investing in evals:
In comparing the three, I found a lot of resonance with the testing pyramid. Sub-agent tests look like unit tests, synthesis tests are almost like integration tests, and end-to-end, well, they’re like end-to-end tests. While the pyramid analogy was useful for wrapping my head around the options, the right choice was derived from first principles.
Sub-agent evals are easy to set up. But you end up mostly mocking the wrong things and testing third-party output. End-to-end evals fight the wide disparity of outputs that emerge from non-deterministic systems. The noise from the output would be too much given my multi-level agentic system. That process of elimination led me to the synthesis level.
The synthesis layer is unique in that it’s the layer that I fully own. This means every failure point is something I can steer toward the intended output. Sub-agent evals could actually fail on the third-party output. End-to-end failures add too much noise. Synthesis failure is either in the parsing of the third-party output, the synthesis of it or the output itself. All of this is within my system boundaries.
The trick that makes this work: input canned sub-agent output into the real synthesizer as fixtures. If you have set up JSON contracts between your sub-agent and agent, you are not mocking the LLM, you’re mocking the data the LLM sees. This provides a deterministic input boundary around a non-deterministic one focusing the signal on the synthesis behavior. Now you can start providing fixtures to mock potential failure modes.
The fixture patterns present themselves as you write down failure modes. How might your sub-agents fail? How might the synthesis fail? And what category of failure mode might they hit? I found myself generating a lot of fixtures to test for groundedness across sub-agents and the synthesis agent. I found a smaller subset of failures in privacy which might pass groundedness checks but provide sub-agent data I would not want in my output. Lastly, I put together a happy path test of what the system should look like given everything is wired up well.
Given more time, people, and scope, I can imagine spending more time on tuning holistic evals. But given these evals are for my personal projects, focusing on evaling the layer I own provides a ton of value in tightening my feedback loop.
]]>I hear it in the hallways. I read it online. The feeling of being left behind by all the latest is nearly tactile.
But in the mess of new technologies and anxious feelings is the good news. You are only a few months behind. This is all only a couple of years old. The technology is uniquely nascent. And whatever was state-of-the-art a year ago has since been wiped away by something newer. The latest and greatest is building on itself. And the ground you feel slipping under your feet is actually slipping under everyone’s feet.
You’re only a few months behind. And when was the last time in your career you were only a few months behind?
]]>The other day, I was working on a personal project. I found myself fully uncertain of what I wanted to build next. I mean, I knew what I wanted, only, when I went to write it out, my mental model was loose and rickety. As sentence piled onto sentence, I found myself asking: what is my ask? Do I know what I want to be built? Or am I asking for help to figure out what to build? I was quickly able to cut to the core and pivot to that ask.
Magically, when you focus on the ask, it becomes clear that everything else is context. By starting with my ask, I can draw the line between the context my LLM already has and what remains. And my immediate goal becomes to close that gap. “Ok, so here are some important things to keep in mind” is a great signal for my own mind on what else needs to be shared. The beauty of separating task from context: by writing the context you strengthen your own mental model, sharpen your ask, and get to the outcome faster.
When I last explored this a few weeks ago, I had my trio of markdown files: AGENTS, PRD, and BACKLOG. Those have exploded recently, but luckily, in a fairly organized manner. My AGENTS describes the how, the PRD describes the high-level what, and my newly-branded BACKLOG becomes the delta between the product today and the design in the PRD. On top of that, the commands and skills I’ve defined provide further context into how my agents and I should work with a lot of meta guidance on how to write to the PRD and BACKLOG as well as updating AGENTS, agents, skills, and commands. Again, it’s the clear separation of context and task.
So what have you written lately for your collaborators? Are you sharing context or scoping work? The distinction should drive the writing.
]]>About a year ago, I spent a lot of time thinking about how to use LLMs. What are they good at? Where do I need that? It was a pretty daunting exercise to examine everything you’re doing and try to find the right job to be done. And while I did find a few use cases here and there, the models kept getting better. And I kept wondering what else I could be using them for.
It was only recently that I realized I’d fallen into a habit of creating loops for myself. In these loops, I’d use LLMs, learn something about them, then return to the beginning: use LLMs, learn something more. Each discovery pulled me back for another rep. Each time, I would push the boundary on what I did the last time. And after a number of reps, the broader system started to become more clear. This clarity then presented more opportunities to pull in and try LLMs in new workflows. The system became self-propelling.
I fell into two principles I adhere to:
Create loops. If you’re a software engineer, solve a problem you see often. Push it to production where you can gather feedback. Then iterate, iterate, iterate. Start simple. Repeat. The old lessons prevail. Happy looping!
]]>For all the slick features offered to get context to the LLM tool, the crux of it is: how do we canonize our mental model for the agents to act as precisely as we desire. And we canonize in markdown files. These markdown files — best practices, PRDs, todos — are all the lifeblood for agents. What’s the shape these files should take so I can integrate seamlessly with the LLM tools? And how could I decentralize and democratize the todos to be accessible from more devices?
And I landed on: what if it’s all text? What if it’s all markdown? What would it be like to fully bend to the form factor for which LLMs optimize?
So I settled on three files: AGENTS.md, PRD.md, and a TODO.md. Agents to describe the how. A PRD to describe the vision. These two are more stable. But Todo describes what’s happening next. It’s the most malleable and receives the most reads and writes. It needs to follow me.
While the markdown files were portable between LLM tools, they were stuck on my machine. This wasn’t so bad for the first two files, but the Todo was my workhorse. Then I had my a-ha moment. I use Obsidian as my notes app. It syncs markdown files across devices and to the cloud. So I took all of my side projects’ Todo files, moved them to Obsidian, and created symlinks from their original directory. Now I can update my coding agents’ next todos from my phone. And when I’m at my machine, they’re ready to be picked up. The context follows me, not the other way around.
When more of your work is text, the LLMs and the device syncing flow seamlessly. The dream of the always-on agent doesn’t have to be driven by tools or models. It’s about context. It’s always been about context. And that context is just text. We just need to move it where it needs to be.
]]>But information isn’t your only medium. You also deal in culture. Because you are a prism.
One of the strange parts of leadership is that you both lead the team and are on it. When someone asks you, “How is the team doing?”, they’re generally asking about your reports. But imagine two outsiders talking and one asks about your team. At that point, the team includes you. You’re Schrödinger’s manager — sometimes you’re on the team, and sometimes you’re not.
Most of the time this is inconsequential. But two failure modes occur in two distinct scenarios: you’re highly positive while your team is highly negative, or you’re highly negative while your team is highly positive. Let’s look at each.
First, imagine an org-wide process change: for every pull request created, the author must record a short video describing the change. “That can’t be so bad,” you think. “Besides, this will be good public-speaking practice.” But your team is seething. “This will slow us down!” “Who’s going to watch these anyway?” Your optimism is too far removed from your team members’ reality. If you present your rose-colored view, your teammates will not only reject it but also begin to lose trust in you.
Second, the inverse: you’re low; the team is high. From your vantage point, you see rocks they haven’t spotted yet. Your team members, now energized, are able to think beyond themselves. They’re observant. And they notice your distress. When they’re flying high and you’re in it, they attune quickly. Your distress becomes theirs, and your gloom about those rocks becomes a self-fulfilling prophecy.
So how do you manage these disparities? Remember: you are a prism. Reflect the energy that’s necessary, and absorb the energy that’s necessary, for your team to be successful. You also reflect your team’s energy back to the rest of the organization.
When you’re out of sync with your team, the goal is to triangulate where you are, where they are, and where you need the team to go. Then walk them there. Genuinely. Your team and your peers will spot inauthenticity quickly. Get there yourself first. Then you can show the way.
You are a prism. Reflect the light for your team.
]]>In all these cases, I share the same advice: Start with your feet, then look to the horizon.
As a new EM, you’re faced with a series of immediate challenges. A team is looking to you for direction, asking questions like:
In these moments, I encourage EMs to start with their feet. Mentally picture yourself focusing in on the ground at your feet. Everything else washes away. What are the things the team needs today?
Tactically, I recommend EMs start with the backlog. When your team comes in on Monday, do they know what they should be working on? If the answer is anything less than a confident yes, start here. Your goal is to ensure there’s a well-defined, actionable sprint’s worth of work ready to go. Creating a backlog is the first step in establishing clarity and momentum.
Once you have the a sprint of backlog set, picture yourself slowly lifting your head just a little. You’ve handled the immediate, but what about the next sprint? If it takes you a sprint to prepare a sprint’s worth of work, your next step is creating the space for you to think further ahead. Begin by asking: what can I delegate from my sprint-to-sprint activities to that will free up my time for longer-term thinking?
Start small. Hand off certain responsibilities — backlog grooming or sprint planning — to a tech lead or senior engineer. Then look bigger. Can you delegate building a domain-focused chunk of the backlog to a team member? At tha point, you’ree starting to create some self-sustaining momentum.
As the team puts together sprints without your direct interaction, you can start looking further out. It’s time to think in quarters or even longer.
Start by asking:
The key here is that fidelity of the planning doesn’t need to be high — after all, you’re looking at the horizon, not examining every step. This longer-horizoned thinking is essential, but it’s a gradual process, one that only becomes possible once the immediate sprint work is stabilized.
By following this stepwise approach—starting with the immediate needs and gradually lifting your focus—you’ll build both confidence and clarity for yourself and your team. In times of transition, keeping this framework in mind ensures you’re able to balance short-term tactical execution with long-term strategic vision.
Remember: start with your feet, then slowly look to the horizon.
]]>Push learning is when you’re fed material to learn. Imagine picking up a textbook, starting at chapter one, and steadily making your way through the book. Or maybe you’re watching a lecture series on YouTube, moving through each video in sequence. Here, you’re absorbing what’s being given to you, which is great for gaining broad knowledge and understanding theoretical concepts.
Pull learning, on the other hand, is more targeted. Picture a problem set in front of you—pull learning involves diving into resources as needed to solve the issue at hand. At work, this might mean watching a specific lecture to tackle a particular problem. Pull learning is all about learning by doing, which tends to make the material stick.
When you’re onboarding, blending both styles can be incredibly effective. The ratio depends on your siituation. One common consideration is how quickly start contributing. You may choose more push learning to optimize for a longer time horizon. If quick wins are crucial, more pull learning might be in order.
Another important consideration is the risk of not being fully ramped up. If you overindex on pull learning, you will only encounter things that are immediately relevant, which could leave you unprepared for rare but critical challenges. In other words, if a critical piece of your infrastructure rarely goes down, you won’t know much about it until it’s too late. That’s where push learning comes in — it helps you cover the unknowns.
By default, I encourage people to begin by applying pull learning and introducing some level of push learning to start. A great question to check your investment levels: “If I push learn at this rate, how long will it take to learn the material? And “Is that rate acceptable given the criticality of the key pieces of onboarding?” This reality check usually nudges people a bit more toward push learning
Keep in mind, push learning can and should be adjusted. If there are certain parts of the material that are more critical, you can and should get to them first as a part of your push learning. This form of push learning when the push learning is prioritized without being just-in time can be referred to as spread learning, and its inverse is referred to as scatter learning.
Bring this mental model to your next onboarding conversation. Happy learning!
]]>Progressing from being responsible for one thing to many starts at the large project level. If you lead a project, break down the work into parallel pieces, and have teammates contributing, congrats — you have a portfolio of work. This is your first real experience managing a portfolio for the outcomes you desire. Initially, your project might be tightly scoped, but as the scope grows, so will the unknowns. You’ll begin to manage risk in how you approach the project and allocate work. This is your first real taste of managing a portfolio of work.
If you are a tech lead or engineering manager, you are responsible for the team’s outcomes. You’re now managing a portfolio of work across a team. At its most basic, you can manage each project separately to success. But as you gain experience, you can begin to manage the whole as a portfolio. Here are some important questions to ask: Which of the team’s projects do I expect to deliver the majority of expected impact? If a project was at risk, which project would I deprioritize to increase the chances of success? What does the curve of success for each project look like: is it binary or is it linear? Having these mental models can allow you to manage your work for greater impact.
As you grow comfort in managing your portfolio, the next step is to build in an intentional amount of risk. If your team has the capacity, you can begin to organize your portfolio to take on a moonshot: book 60% to “sure thing” work, 30% to “somewhat risky but linear impact” work, and push the remaining 10% toward that home run shot.
Once you manage your quarterly and half year planning well enough, you can intentionally manage your portfolio with time as an added dimension. Set your team up to deliver for a couple quarters in a row, and take the big home run swing once you’ve banked some gains. Or, if you took a swing and it didn’t come through, plan for more “sure thing” work the following quarter. Your portfolio of work exists in a given quarter or half year as your work-in-progress. It exists in hindsight as your team’s reputation. Build up your team’s social capital and spend some of it.
Finally, there’s breadth. As you work across teams, you begin to manage portfolios of portfolios. To do this with intentionality, you need to work with your team counterpart. Encouraging people and teams to take on or off risk becomes a part of your calculus. Two failure modes to consider. In one, teams are continually risk-averse and you need to push one toward risk. In the other, all of your teams are too bold, and you risk little delivery across them. As you manage your teams, manage them to produce the portfolio of work you intend across their breadth.
How’s your portfolio of work performing? What can you do this week to build some more intentionality into it?
]]>