AI Agent Project Structure: Folders, Files, and Responsibilities

Do not memorize every filename at once. Treat the tree like a city map: learn the neighborhoods first, then the streets.

Your path in this article: (1) scan the map, (2) set root config, (3) walk one request through src/, (4) open the agent/ brain, (5) add tools / models / prompts, (6) expose HTTP, (7) prove it with tests and run main.py. After each step, check the takeaway — if it does not stick, re-read that section before moving on.

Start at the repository root with files every teammate and CI job expects. README.md should hold install steps, env vars, and one copy-paste command that works on a fresh clone.

requirements.txt (or pyproject.toml) pins the LLM SDK, HTTP client, and test runner. .env holds API keys — load it at runtime and never commit it. Pair it with .gitignore (.env, logs/, caches). docker-compose.yml is optional: use it when you need Redis, Postgres, or a local vector DB with one command.

Put application code under src/. Six packages cover almost every agent:

api/ receives HTTP → agent/ plans and loops → models/ calls the LLM → tools/ run actions → prompts/ supply instructions → utils/ loads config and logs.

Read the next sentence aloud: request comes in, agent thinks, model decides, tool acts, state updates, response goes out. That is the learning loop — every folder exists to support one of those moments.

This is where product behavior lives. agent.py is the loop: load context, ask the LLM what to do next, stop when done. executor.py turns tool names into real function calls and catches errors. state.py tracks the current session. memory.py recalls longer-lived facts.

Keep tools out of agent.py. The loop should only see tool names and schemas registered from tools/. That separation is what makes testing possible — mock the LLM, assert the executor called the right tool.

Once the brain is clear, add three plugin layers.

tools/ — one file per capability (search.py, calculator.py, weather.py). Each exports name, description, input schema, handler, and a documented success/error result. models/ wraps the vendor SDK so changing providers does not rewrite the loop. prompts/ stores system and task templates as versioned code—review them in PRs like other logic.

Add one tool, wire it through the registry, and trace one successful call plus one timeout before adding the next. The goal is not a universal folder standard; it is stable dependency direction and contracts that tests can replace.

api/ is how other systems talk to your agent. routes.py defines POST /chat (and health). schemas.py validates the body before you spend tokens. Keep handlers thin: validate → agent.run() → JSON.

utils/ is shared wiring. config.py reads env. logger.py emits structured logs. When the CLI and the API both call the same agent/ package, you know the learning stuck — the brain is reusable.

Structure without proof drifts. tests/ mirrors src/: mock the LLM in test_agent.py, fixture tools in test_tools.py, and hit routes with TestClient in test_api.py. The starter below is intentionally small and runnable; it proves the plugin boundary before adding an LLM SDK.

data/ holds examples and a knowledge_base/ for RAG. logs/ stays out of git. A later main.py may wire CLI or HTTP entry points, but the first practice succeeds when the tool contract runs under pytest with no network.