Code Generation / AI Coding

Having a model write code and having it write a piece of copy are both superficially “generating text,” but there is a fundamental difference: code is not text that merely needs to be comfortable for people to read; it must be able to run, and right and wrong are clear. A mediocre piece of copy is only slightly less effective; a piece of code missing one symbol or using an interface incorrectly will crash or produce wrong results, which in the vast majority of cases is a fact that can be verified and judged. This difference determines the fundamental nature of the task of AI coding: it is not free creation, but a generation task that translates an “intent” into “code that can run.”

Specifically, the input to AI coding includes three things: requirements expressed in natural language (or implied in comments or function signatures), existing code in the repository, and acceptance criteria. The output is a candidate patch, an explanation of the new code, or a completely new program. What the model has to do is, after understanding these inputs, produce a piece of code that the machine can execute and that matches the intent.

AI coding covers a whole spectrum from small to large. The smallest end is completion: you are typing a line, and the model guesses the rest. One level up is generating a function from a comment: you write a sentence like “add up all the positive numbers in the list,” and the model generates the corresponding implementation. Further up is implementing a whole feature: turning a requirement into working code across functions and modules. The largest end is refactoring across multiple files: reorganizing the code structure while keeping behavior unchanged. These tasks differ in scale, but the core is exactly the same—they all translate intent into code that can run, and there is no essential difference in the mechanism.

Here is a boundary that must be remembered: code that can run only means the syntax is correct and that the execution paths that have been tested hold. Passing the run does not prove that all requirements are met, that there are no security vulnerabilities, or that performance is up to standard. A program may run without any errors but handle edge cases incorrectly, or exhaust resources on extreme inputs. So “being able to run” is a necessary condition for AI coding, far from a sufficient condition; it is only the first link in the verification chain, not the endpoint.

Both are generation; why has "writing code" become one of the most successful directions for large-model deployment? The answer lies in the fact that code simultaneously possesses three properties that are extremely model-friendly.

First, code itself is text. During Pre-training, the model has seen an astonishing amount of open-source code in massive public data, and has read repositories in various languages and styles. Writing code is not a new skill imposed on the model, but something it originally does—continuing a function or completing a loop is exactly the behavior pattern it repeatedly practiced during Pre-training.

Second, code can be executed and verified. This is the most critical point. Whether code is correct can be known by running it: compilation errors and failing tests are clear, objective feedback signals that do not rely on subjective human judgment. This contrasts sharply with prose—whether a piece of prose is 'good' has no objective standard; the model cannot get measurable feedback, and the direction of improvement is vague. But once code fails to run, the error becomes a concrete, locatable counterexample, and the model can correct itself accordingly. Verifiability compresses 'vague errors' into 'concrete counterexamples'; this is the fundamental reason code generation can form an efficient closed loop.

Third, code patterns are repetitive and structurally strict. Syntax has clear rules, and patterns are highly recurrent—CRUD operations, loop traversal, error handling, resource cleanup appear in similar forms across countless projects. Models excel at 'continuing by pattern', and code happens to be one of the texts with the highest pattern density.

But 'verifiable' has an easily overlooked boundary: verification only covers specifications that have already been expressed. Compilers, tests, and static analysis tools can provide objective feedback, but they can only check whether you have implemented 'the requirements that have been written down'. If the tests lack boundary conditions, or if real requirements were not translated into checkable acceptance criteria at all, then an incorrect piece of code can absolutely achieve 'all tests green'—all tests pass, but the requirements were misunderstood from the start. To form a truly reliable closed loop, you cannot rely only on 'running it'; you must first spend effort translating requirements into checkable acceptance conditions, and then use them together with unit tests, integration tests, security scanning, and human review. In other words, code is suited to large models because the training corpus is rich, syntactic structures are repetitive, and compilation and testing can produce external feedback; the inputs are requirements and code patterns, and the output is a candidate implementation that can be verified by tools. The true value of verification is to force errors into concrete counterexamples; what it cannot do is complete the 'specification' for you.

AI coding is not done in one step; it evolves level by level in the direction of “managing more and more broadly.” To understand this spectrum, the key is to see how broad a decision surface each level hands over to the model.

The earliest level is completion. As you type, the model completes the next line or next block in real time; early Copilot was in this form. It only handles “what follows the line you are currently writing,” with the input being a small piece of context before the cursor and the output being a small piece of continuation code, without needing to understand the entire project or make any decisions on its own.

The next level up is conversational. You can use natural language to have it generate a piece of code, explain a piece of code, or modify an existing piece of code. At this point the model begins to understand requirement statements, but the scope of work is usually still limited to the snippet you point it to.

The next level up is repository-level. The model must understand the entire project and make changes across files—find all relevant call sites, update the interface and its users in sync. This requires retrieval and search capabilities, not just continuation.

The broadest level is the autonomous coding Agent. You give it a task; it reads code, modifies multiple files, runs tests, reads error messages, and modifies again, repeating this loop until the task is complete. The essence of this step is applying the whole “Agent” mechanism to code: planning, calling tools (reading files, running commands), observing results, and adjusting.

The causal chain of this spectrum is clear. Completion only requires “the model can write”; conversational requires “the model understands and writes a piece correctly”; repository-level requires “the model can search and locate”; and autonomous coding requires the model to enter a closed loop like an Agent—plan, call tools, observe results, and modify again. The further along, the broader the scope of work the model is responsible for, and the system correspondingly adds capabilities for search, planning, execution, and recovery.

Therefore, the input at the capability stage is no longer merely a code snippet, but the task scope, repository context, available tools, and granted autonomy; the output correspondingly expands from completion and conversational modification to repository-level patches, and then to complete Agent trajectories. Two boundaries must be made clear. First, a higher stage only means that the model is responsible for a wider scope of work, not that the model itself has become smarter—a model that is mediocre at completion will not suddenly become good at planning just because it is wrapped as an Agent. Second, broader responsibility should not automatically translate into greater permissions; granting a repository-level tool the permissions to delete databases, publish, or make payments is an overreach not supported by the capability stage.

Just being able to "write" code is not enough. To make AI coding truly reliable, the most important step remains: actually run the code. Have the model execute right after writing, and feed back the runtime results and test results; it can then discover its own mistakes and revise accordingly. The loop of "write → run → see errors → fix → rerun" is precisely the key partner that distinguishes code generation from other text generation.

A caching feature example makes the whole process clear. The first step is to turn a vague requirement into checkable acceptance criteria: the requirement says "make queries faster", so rewrite it into three checkable conditions—on a second call with the same parameters, do not hit the backend; results must not be shared between different users; existing tests keep passing. This step determines whether all later verification is valid.

The second patch, the model uses query as a global cache key. Unit tests passed, but a security review found that the cache key lacks user_id—meaning different users will hit the same cache slot and see each other's results.

Then introduce a failing test test_cache_isolated_by_user: it makes user B get user A's results, rewriting an illusion of "logical correctness" into the clear conclusion "cache key incomplete". The key here—the reason the loop is stronger than "just regenerating"—is that the second patch is not because the model randomly chose a different wording, but because the failing test provided a concrete counterexample: the same query under two different user_ids must produce two cache slots. Executable feedback shrinks the vague "might have a bug" into a locatable, re-verifiable constraint.

Next is the minimal fix: change the cache key to (user_id, query), and cache only successful responses. The target test and the full test suite pass, and the diff shows no tests were modified—this guarantees the model passes by fixing the implementation, not by tampering with tests.

Finally, delivery: report the changes, test commands, and remaining risks, but do not auto-merge. Keep in mind that a fully green test suite only supports behavior already covered by tests; it does not replace code review, nor does it grant authority to auto-merge or release.

Figure 1 The loop made possible by verifiability: write then run, correct errors using real error messages and test results, then revise and rerun. It is this objective feedback that enables AI coding to self-correct—something tasks like copywriting cannot offer. The inputs of the entire running example are the caching requirement, the first patch, the user isolation test, and project regression; the output is the corrected minimal patch and the runtime evidence. The system first turns the requirement into three acceptance conditions, then uses the failing test to locate the issue of the cache key lacking user_id, and after the fix verifies from narrow to wide.

Why does the same model perform strongly when writing a small function, yet frequently make mistakes once it is placed in your large project? The most direct reason: it hasn't seen the full picture of the project. In a real codebase, a change often needs to take into account interfaces elsewhere, naming conventions, dependency relationships, and these are scattered across other files. If you don't feed it the relevant context—related files, function signatures, project conventions—it can only guess, and so it writes code that 'looks correct on its own but breaks when put into the project'.

So the core of repository-level AI coding is Context Engineering: feeding the right context into a limited window correctly. The input includes the symptoms of the current failure, relevant interfaces, callers, tests, project conventions, and dependency versions; the output is the minimal package of relevant code needed to complete the modification. The system expands materials step by step, starting from reproducing the problem and symbol relationships—first look at where the error occurs, then what it calls, who calls it, and what conventions exist—rather than indiscriminately stuffing the entire repository into the window. Indiscriminate stuffing only fills up the context window and dilutes the truly useful information.

There is a boundary to keep clear here: sufficient context only means that the key constraints have been seen, not that the model will definitely get it right. The model may still misunderstand the meaning of the constraints, or miss paths that were not loaded. Context Engineering solves the problem of 'not seeing'; it cannot eliminate the problem of 'seeing but misunderstanding'.

Can you blindly merge AI-written code? No. The risks it brings are of several specific kinds.

Hallucinated APIs: the model may seriously call a function or library that does not exist at all—the name looks plausible, but it is actually nonexistent. This is the typical form of hallucination in code; it only surfaces when compilation or execution reaches that point.

Looks right but actually has bugs: being able to run does not mean the logic is correct. Edge cases, concurrency, and numerical precision hide pitfalls; if tests do not cover them, they will not be found.

Security vulnerabilities: the model may write code with injection, unauthorized access, and other issues, or copy bad examples from its training data. The coding patterns it has learned are themselves mixed with a large amount of insecure legacy code.

Over-trust: the longer and more plausible the code looks, the more likely people are to relax review—and that is precisely the most dangerous point. Neat formatting and logical correctness are two different things, and people easily mistake the former for the latter.

The right positioning is to treat it as "a very capable but error-prone junior engineer": productive and useful, but its output must be reviewed and tested. The correct use of AI coding is "a human-supervised accelerator," not "an unsupervised replacement."

In operational terms, the inputs to code risk control are candidate patches, dependencies, tests, security rules, and reviewers; the outputs are conclusions such as accept, revise, reject, or pending verification items. Compilation, testing, static analysis, and manual diff provide layered checks: compilation blocks hallucinated APIs, testing blocks logic errors that are already covered, static analysis blocks known vulnerability patterns, and manual review blocks problems that machine rules cannot express. One point must be faced honestly: treating the model as a high-output junior engineer is a way of assigning responsibility; it does not mean that human review is naturally error-free—reviewers themselves may also be tired or misjudge, so this review process also needs to be taken seriously.

If a model is strong on small function benchmarks, can we directly infer that it can fix real projects? No, we cannot directly infer that. This reflects a difference between two measurement logics.

Function generation commonly uses the pass@k metric: sample k candidates, and if at least one passes the hidden tests, the task counts as a success. It measures “whether a few tries can hit the target,” and allows trading sampling attempts for success rate. pass@1 focuses only on a single generation hitting the target, without the benefit of a sampling budget. Therefore, for the same model, reporting pass@k versus pass@1 means completely different numerical meanings.

Repository-level tasks have much higher requirements: the model also needs to locate relevant files, understand dependencies, make minimal changes, and pass regression tests. Evaluations like SWE-bench come closer to this real workflow; they test not just “can it write this snippet,” but “can it find the right places to change in someone else's project, change them correctly, and avoid breaking anything else.”

To compare results, several variables must be fixed; otherwise scores are not directly comparable: test set, tool permissions, sampling budget, and whether retries are allowed. If any one differs, the numbers are not comparable.

Final boundary: regardless of pass@1 or pass@k, passing tests only proves that the code passes existing tests; it does not prove that it is safe, meets performance criteria, or fully conforms to user intent. The inputs to the metric are fixed tasks, hidden tests, tool permissions, the sampling count k, and the retry budget; the outputs are pass@1, pass@k, or repository-level task success rate. The scope covered by each is exactly what determines what each cannot prove.

Stringing together the threads from the previous sections reveals a complete causal chain, from “why code suits language models” all the way to “why all tests passing is still not a license to merge without human review.”

The starting point is: AI coding lets models translate intent into runnable code, from completion to implementing features. It succeeds because code simultaneously has three properties—it is text, and models have been trained on massive amounts of open-source code; it can be executed and verified, so correctness can be tested through compilation and tests; its patterns are highly repetitive, making it suitable for pattern-based continuation.

From this, tools evolve step by step along completion, conversational, repository-level, and autonomous programming Agent, and the advanced form is essentially applying an Agent to code. The key partner that truly closes this evolutionary loop is code execution: write, run, read the error, fix, run again—the model self-corrects based on real feedback.

But once you enter large projects, new constraints appear: if the context is not fed correctly, the model writes nonsense, so the core of repository-level programming is Context Engineering. Even when context and verification are done well, risks remain—inventing non-existent APIs, hidden bugs, security vulnerabilities, over-trust—so the output must be reviewed and tested.

Ultimately this chain leads to a conclusion: verifiability makes code one of the most successful applications of large models, but it only verifies “the expressed specification”; repository-level success, by contrast, depends heavily on feeding the right context in the right way. Taken together, these two explain why “all tests passing” is a necessary part, but is never a license to skip human review and automatically merge and release.

The knowledge on this page is not isolated; it builds on several more foundational concepts and paves the way for several deeper topics.

Prerequisite concepts: To understand AI coding, you first need to understand large language models themselves, tool calling, and the basic mechanisms of AI Agents. These three determine "why models write code" and "why the advanced form is an Agent".

The core of this page is executable verification, the write-run-modify loop, the evolution from completion to coding Agent, and context determining success or failure. Together, these four points form the kernel for understanding AI coding.

Closely related extensions are: Code Execution and Sandboxing (where the safe environment for running code comes from), AI Coding Tools (how specific tools implement the mechanisms discussed earlier), Context Engineering (how to systematically feed the right context to the model), hallucination (the root of errors such as fabricating nonexistent APIs), and human-in-the-loop (why a human must be in charge).

Further directions also include: Planning and Task Decomposition, Workflow Orchestration, and evaluation. These are issues encountered only when scaling a single coding task into a larger system, and can be explored after mastering the core of this page.