A production REPL: Yaegi over the composition root

I came to Go from Ruby, and for a long time the one tool I missed most was the one Rails ships with: a console where any confusing production situation can be inspected by hand. bundle exec rails c, Order.find(42), and what you get is not a row in a table but an application object with all of its methods. My Go projects had no such console. What was left was psql, and only if it had been added to the image. You can look at data; you cannot run an application operation. And the question is usually exactly that: why didn’t the feature turn on for this user, even though the flag is set in the table?

What follows: choosing between gore and Yaegi, why “export the services one by one” is a trap, why packages come out undefined in the REPL even when they are registered correctly, and how to make it impossible to wipe the production database from the console by accident. All the code this article relies on lives in cosy-console; the listings below are excerpts from it.

The problem

Supporting production without a console looks like this. A ticket arrives: “order 42 has the wrong status”. Then:

1. connect to the database
2. remember which table holds the status and which one holds its history
3. psql $POSTGRES_URL
4. SELECT ... JOIN ... WHERE order_id = 42
5. look at numbers with no business context: what does status = 7 mean?
6. go read the code to decode it
7. if the use case has to be inspected: write a temporary HTTP endpoint
   or reproduce production state in a local environment, which is
   not always simple in itself

In Rails, one command replaces that entire workflow:

$ bundle exec rails c

order = Order.find(42)
order.status_label           # the status is readable
PaymentsService.retry(order) # run an action from the console

I wanted the same thing in Go. The question is what to build it with.

Act 1: choosing the tool

Googling “go repl” gives two candidates: gore and Yaegi. At first glance both are “a REPL for Go”, but they solve different problems, and that distinction matters here.

gore: about the language, not about your application

gore is an ordinary Go REPL: it starts, waits for statements, and runs them through go run. That gives it two properties, each of which matters for our problem:

property 1: every statement is compiled from scratch (slow, and the
            authors say so themselves)
property 2: it needs the Go toolchain, which production does not have
            and should not have

But the main thing is neither of those. Starting gore in the project directory only gets you as far as importing a project package:

:import cosy-console/internal/domain/orders

And that is as far as it goes: you cannot construct the use case. It needs a *gorm.DB, a connection pool, config from environment variables, clients of external services. In your application all of that is assembled by the composition root, the single place where dependencies are created and wired together, and gore knows nothing about it. What you get is a local sandbox that still has to be turned into a production-ready console.

Yaegi: an interpreter that embeds into your console

The key word is embeds. Yaegi has an API:

i := interp.New(interp.Options{...})
i.Use(stdlib.Symbols) // register packages
i.Eval(`1 + 1`)       // run code
i.REPL()              // an interactive session

The capability that matters to us is passing compiled values into the interpreter. You can hand the REPL a *gorm.DB, a use case, a config, and the interpreted code will call them as ordinary values.

gore:                       Yaegi:

REPL --go run--> code       the application process
                              |
no access to the app        composition root (db, use cases, http clients)
                              |
                              v
                            Yaegi interpreter --> the REPL calls
                                                  compiled objects

One important note about Yaegi before we continue

The latest Yaegi release is v0.16.1, April 2024. Issues keep appearing in the tracker, releases do not. That is acceptable for us: Yaegi is a tool, it does not touch business logic, and if we ever need to, we replace it. What matters is what is actually interpreted here: not business logic, but a single call, App.Orders.Usecases.Order.Get(Ctx, id), and the method itself stays ordinary compiled code.

A human starts the REPL; no user traffic goes through it. The interpreter can crash, but only the session dies with it, not the API process. With that constraint, Yaegi is fit for a console.

Act 2: the console lives in a process, not in an endpoint

The second decision is architectural, and it is easy to fall into a common misconception here.

Discarding the “button in the admin panel” idea right away

A frequent mistake: make the REPL an HTTP endpoint, or embed it into the main process to avoid building a separate binary. What is wrong with that:

Option	What is wrong
REPL inside the http server	an interpreter with state in the process memory: a shared connection pool, shared memory, and anything that kills the console can kill the API
An HTTP endpoint /console	this is remote code execution (RCE) over HTTP; whoever finds the endpoint will get around any protection you put on it
REPL as an endpoint with auth	still RCE, just with a password

We want the same convenience Rails gives. And rails c does not connect to a running server: it starts its own process and loads the same environment (models, config, database, and so on). Its memory is its own, while config and data are the same as production: load and database locks are shared, but the API process does not depend on the console.

A separate process plus a shared composition root

We follow the same model:

                       config + logger
                              |
                    composition root (Container)
                              |
            +-----------------+------------------+
            v                 v                  v
        cmd/api           cmd/worker        cmd/console   <- the new binary
            |                 |                  |
        HTTP server         workers          Yaegi REPL
            |                 |                  |
            +--------- compiled use cases -------+
                              |
                    PostgreSQL / Redis / Kafka
             (the same ones, through the same production config)

cmd/console starts the same composition root as the API, which means:

• the database connection is configured identically;
• the console sees what the application sees: one and the same container graph. There is no HTTP layer in that graph: the router and the handlers live outside the container, and the console does not need them.

The Dockerfile only needs two extra lines: build the binary and copy it into the same image. No separate containers, no separate images:

RUN go build -o bin/console ./cmd/console
...
COPY --from=builder /app/bin/console /app/console

The binary simply sits in the image next to the API. It is invoked from inside the container. It exposes no port or HTTP endpoint.

Locally you can check it like this:

docker compose exec api /app/console

What to export into the REPL: three attempts to get it right

Now the subtle part: which names are visible in a session. We went through three options before we found the right one.

Attempt 1: export everything one by one.

i.Use(interp.Exports{"console/console": {
    "DB":      reflect.ValueOf(db),
    "Orders":  reflect.ValueOf(ordersUseCase),
    "Catalog": reflect.ValueOf(catalogUseCase),
    ...        // 30 lines today, 60 after a couple of sprints
}})

It works, but the list grows with every domain, the names live in two places (the container and the export list), and you find out about the service you forgot in the middle of an exciting incident investigation.

Attempt 2: auto-export by reflecting over the container.

// walk the fields of Container with reflect and register each one
v := reflect.ValueOf(app).Elem()
for i := 0; i < v.NumField(); i++ { ... }

Magic, and dangerous magic. First, it does not always work: an interface field, an embedded struct, a name collision, and reflection trips over each of these in its own way. Second, and worse: you refactor the container, rename a field, move a use case between domains, and everything compiles, all the tests are green: they know nothing about the console. Meanwhile the names in a session have changed: the familiar App.Orders.Usecases.Order.Get(...) is now undefined. The person who finds out is not the author of the refactoring but the engineer debugging the incident.

For a tool used to fix production, “silently” is the worst word there is.

Attempt 3: the whole container as one value.

func exports(ctx context.Context, container *app.Container) interp.Exports {
    return interp.Exports{
        "console/console": {
            "App": reflect.ValueOf(container),
            "Ctx": reflect.ValueOf(ctx),
        },
    }
}

App is the whole composition root as it is, Ctx is the session context for methods that require one. Here is what App looks like:

App (*app.Container)
 |-- Orders
 |    |-- Usecases.Order.Get(ctx, id)  -> (models.Order, error)
 |                      .Cancel(ctx, id)
 |    \-- Repos.OrderRepo.Get(ctx, id)
 |                       .Update(ctx, id, orders.UpdateAttrs)
 |-- Catalog
 |    |-- Usecases.Item.List(ctx, catalog.ItemFilter)
 |    \-- Repos.ItemRepo.Get(ctx, id)
 \-- Shared
      |-- DB   *gorm.DB
      \-- Tx   transaction manager: InTransaction(ctx, fn)

Identifiers in the project are uuid.UUID, so id in the examples below is a parsed uuid, not a number. Access is navigation through that structure:

> App.Orders.Usecases.Order.Get(Ctx, id)
> App.Catalog.Repos.ItemRepo.Get(Ctx, itemID)
> App.Shared.Tx.InTransaction(Ctx, func(...) error { ... })

The reflect in this code is there only because that is the Yaegi API. No magic dependency discovery: whatever is in the container is what is available. Add a use case to the composition root and it shows up in the console automatically. HTTP handlers are not part of the container. The console does not know about them and should not: it needs data and business rules, not a router.

Act 3: symbols, or why you get undefined type with correct code

Let’s look at how Yaegi is built. Starting with the terms:

Symbols are a “name -> reflect.Value” map inside the interpreter. Through it Yaegi sees the compiled world.

Registration (i.Use) writes a package into that map under a path-shaped key. Registering is not enough: a name appears in the session only after binding.

Binding (i.ImportUsed) declares the names of registered packages in the session scope, as if every import had been executed at an invisible start of the session.

extract is a symbol table generator: it reads a compiled package and writes a Go file that fills Symbols in init().

The problem: the REPL cannot construct an argument

App is there in the session, methods can be called, but the very first call trips over an argument:

> App.Orders.Usecases.Order.Get(Ctx, uuid.MustParse("...b2"))
1:28: undefined: uuid

To read an order by id you need the type uuid.UUID, and to the interpreter uuid is a package that does not exist. The same goes for the project’s own structs:

func (r *OrderRepo) Update(
    ctx context.Context, id uuid.UUID, attrs orders.UpdateAttrs,
) (models.Order, error)

the expression orders.UpdateAttrs{Status: &status} has to be compiled first, and orders is unknown to the interpreter too:

> status := "x"
> App.Orders.Repos.OrderRepo.Update(Ctx, id, orders.UpdateAttrs{Status: &status})
1:28: undefined type

Yaegi cannot “just import” a compiled package: the interpreter does not have its types in its own table. That is what yaegi extract, the symbol table generator, exists for.

What extract generates (and what to do with it)

go run github.com/traefik/yaegi/cmd/yaegi extract -name symbols cosy-console/internal/domain/orders

creates a file:

// Code generated by 'yaegi extract cosy-console/internal/domain/orders'. DO NOT EDIT.

func init() {
    Symbols["cosy-console/internal/domain/orders/orders"] = map[string]reflect.Value{
        "ErrNotFound":    reflect.ValueOf(&orders.ErrNotFound).Elem(),
        "UpdateAttrs":    reflect.ValueOf((*orders.UpdateAttrs)(nil)),
        "CreationParams": reflect.ValueOf((*orders.CreationParams)(nil)),
        ...
    }
}

The comment on the first line speaks for itself: the file is generated, there is no reason to edit it by hand.

Another mistake is to run extract over everything in sight, gorm.io/gorm included:

github_com-google-uuid.go               <- 70 lines
gorm_io-gorm.go                         <- 900 lines: gorm.Open, gorm.DB, clause...
cosy-console-internal-domain-orders.go  <- 30 lines
cosy-console-internal-domain-catalog.go <- 30 lines
...

Don’t do that. A gorm symbol table is for the scenario where the REPL user writes import "gorm.io/gorm" and opens connections of their own. In our case the connection is already open, created by the composition root with the production config. It arrives in the REPL inside the container, as App.Shared.DB. There is no reason to open second, unconfigured connections from the interpreter. We deleted the gorm file, and stated the rule as: symbols are for input types, not for infrastructure.

How to decide what to generate: count the input types

We need to know which packages are required. Intuition says “every domain and every model”, and intuition is wrong. The right way to ask is:

which packages must the REPL user be able to CONSTRUCT
in order to call a container method?

Whether a package is needed depends on where its types appear in a method signature:

                  +-- method parameters --> INPUT: the user builds a value
a container method+                         -> the package is needed in symbols
                  +-- return values ------> OUTPUT: the user only reads it
                                            -> the package is NOT needed

Return values are handled by Yaegi itself: a value returned from a compiled method carries its type with it, and its fields can be read without any registration at all:

> o, err := App.Orders.Repos.OrderRepo.Get(Ctx, id)
> o.Status // works, although the models package is registered nowhere

We did not guess the list of input types. We derived it with a script. It takes Container, walks all of its exported methods and collects the argument types; if an argument is a struct, it descends into its fields and collects types from there as well. The script uses golang.org/x/tools/go/packages to load the project: that package is not in the standard library, it lives in a separate module. The answer turned out to be compact:

Package	Why the REPL needs it
cosy-console/internal/domain/orders + models	UpdateAttrs, CreationParams; the model is an argument of Create(ctx, order)
cosy-console/internal/domain/catalog	search filters, pagination options
github.com/google/uuid	uuid.UUID in signatures: uuid.Parse(...), uuid.Nil
cosy-console/pkg/pagination	the WithPage(1) options

Five packages in total instead of “everything”: orders and its models, catalog, uuid, pagination. Checking for what is missing costs nothing: start the REPL, try to construct the value, and if you get undefined, add the package to the list. Finding what is superfluous works from the other end: a package that never turned out to be needed as an input argument gets removed from the symbols.

One nuance about uuid: extract gives you the whole package, including internals you will never call from a console (ClockSequence, SetRand, NodeID), while the application needs a limited set. We kept the minimum: Parse, MustParse, Must, New, Nil, UUID.

yaegi extract

extract is a utility of Yaegi itself. The interpreter cannot “just import” a compiled package: it does not have the types from your binary in its own table. extract is the bridge: it reads a package and generates a name -> reflect.Value map for all of its exported entities. Such a map can be written by hand too, which is how we register App, Ctx and the trimmed-down uuid below, but for a domain package that would be hundreds of lines. Whatever ends up in the map can be constructed and called from the REPL.

How to run it. extract lives in the same Go module as Yaegi. The flags that matter: -name sets the name of the map (ours is symbols), while -include and -exclude filter by regexp if you only need a slice of a package.

What you get. A file in the current directory, one per package; the file name is built from the import path. Run it from internal/console/symbols/ and the files land exactly where they belong:

$ cd internal/console/symbols && go run ... extract -name symbols \
      cosy-console/internal/domain/orders cosy-console/internal/domain/orders/models

cosy-console-internal-domain-orders.go        <- extract of the domain
cosy-console-internal-domain-orders-models.go <- extract of its models

Inside each file is the same map under init() shown above.

Checking that nothing was forgotten. The generated files guarantee nothing by themselves: a package may have never made it into the list, or may have been lost to a name collision. You verify it with a one-off expression in the assembled console, one type from each symbol package: /app/console -e 'orders.OrderFilter{Status: "paid"}.Status' must print a value, not undefined.

Syncing with one command. Symbols are a snapshot of the code, and the snapshot falls behind every change to a signature: add a field to UpdateAttrs and it is not in the REPL until you regenerate. Which means regeneration needs to be a single command, not a sequence of steps that one person on the team remembers. So it is packed into a make target:

# make console-symbols DOMAIN=orders         (a single domain)
# make console-symbols DOMAIN=orders,catalog (several, comma-separated)
# make console-symbols                       (every domain at once)
DOMAINS ?= orders catalog

console-symbols:
	@set -e; for domain in $$(echo $(if $(DOMAIN),$(DOMAIN),$(DOMAINS)) | tr ',' ' '); do \
		echo "==> $$domain: extracting domain + models"; \
		( cd internal/console/symbols && \
			go run github.com/traefik/yaegi/cmd/yaegi extract -name symbols \
				cosy-console/internal/domain/$$domain cosy-console/internal/domain/$$domain/models && \
			mv -f cosy-console-internal-domain-$$domain.go $$domain.go && \
			mv -f cosy-console-internal-domain-$$domain-models.go $${domain}_models.go ); \
		echo "==> $$domain: written $$domain.go + $${domain}_models.go"; \
	done

Three decisions are baked into that target.

First: extract is called with two paths, the domain itself and its models, because that is how our project is laid out. extract does not support ... patterns, and a Go package does not include subdirectories, so one path is not enough: the params would be extracted and the models would not. The models are needed not for reading but for constructing: in Create(ctx, order models.Order) the model is an argument, and without registration models.Order{...} gives undefined type in a session.

Second: extract runs inside internal/console/symbols/. It writes files into the current directory, so this way they land exactly where they belong instead of in the repository root. The two generated files are renamed to <domain>.go and <domain>_models.go: one file per package, with an honest DO NOT EDIT and extract’s own keys.

Third: mv -f. No file, it gets created; a file exists, it gets overwritten whole. No hand-editing of generated code: you change UpdateAttrs, run make console-symbols DOMAIN=orders, commit. The session names are picked up by the domain itself, because they are derived by Exports(), not by the file.

Why everything inside App is available without symbols

Symbols are needed to build a value from scratch. Everything the composition root has already created arrives in the session inside App, and does not have to be registered. The console has two delivery channels:

channel 1: symbols         TYPES AND FUNCTIONS by package name
                           what the user CONSTRUCTS themselves:
                           orders.UpdateAttrs{...}, uuid.Parse(...)

channel 2: App             the whole container GRAPH
                           what the user only CALLS:
                           App.Orders.Repos.OrderRepo.Get(...)

Symbols answer the question “how do I create a value in the REPL?”. App answers “how do I reach something already created?”. The repositories are created by the composition root long before the REPL, so by session time they already exist, inside App.

The Yaegi mechanics: on App.Orders.Repos.OrderRepo.Get(...) the interpreter does not look up a package by name, it walks the fields of App through reflection, and the method is found by the type of the value itself. The value carries its type with it. That is why use cases, repositories and App.Shared.DB all work without a single line of registration: all of them were created before the REPL started.

If you do extract them and call them anyway, order_repo.New(db) in the REPL gives you second instances of the repositories, bypassing the container: a different configuration, a different pool, no relation to what the application actually runs on. And the chain would drag *gorm.DB into the symbols, together with the “open your own connection” scenario we banned a couple of sections ago.

Imports without imports: ImportUsed

The packages are registered now. But that is not enough: after i.Use(...) the package names are still invisible in a session, and every session would start with a dozen imports:

> import "fmt"
> import "time"
> import orders "cosy-console/internal/domain/orders"
> ...10 lines, and only then the first useful call

Yaegi handles this with one call:

i.ImportUsed()

ImportUsed takes every package registered through Use and declares its name in the global scope, as if all the imports had been executed at once at an invisible start of the session:

Use(...)      = "interpreter, remember these packages"   (registration)
ImportUsed()  = "and show their names without imports"   (name binding)

> fmt.Sprintf("%d", 7)   <- fmt is available
> orders.UpdateAttrs{}   <- and so is orders

The one line the console executes on the user’s behalf

ImportUsed covered all the ordinary packages, but one line had to stay in the startup code: import . "console". Here is why:

Step 1: the interpreter only accepts packages. We want to hand two values to the session, the container and the context, but Yaegi will not take a standalone value: a “name -> value” map has to live under some path-shaped key. So App and Ctx are wrapped into a “pseudo-package”, a map under the made-up path console/console. No real package by that name exists; to the interpreter it looks like a package console with two variables inside.

Step 2: without an import the names are unavailable. A pseudo-package is a package like any other: its names appear in a session only after an import. Going through a plain import:

> import "console"
> console.App.Orders...         <- works, but needs the prefix every time

Step 3: the dot removes the prefix. Go has the dot-import: import . "pkg" declares a package’s names directly in the current scope, with no prefix:

import . "console"
// App and Ctx are available directly:
// > App.Orders...

Why didn’t ImportUsed do that itself? It only knows the ordinary import, and it takes the session name from the registration key: everything after the last slash. Our key is console/console, the last slash cuts off console, and the session gets a package by that name, exactly the prefix we are trying to get rid of:

> console.App.Orders.Usecases.Order.Get(Ctx, id)   <- this is what works after ImportUsed
> App.Orders.Usecases.Order.Get(Ctx, id)           <- undefined: App

The dot-import is a separate operation, and ImportUsed does not perform it. So the console performs it itself, with a single Eval:

i.ImportUsed()
if _, err := i.Eval(`import . "console"`); err != nil { ... }

The order is mandatory. The doc comment on ImportUsed says it outright: do not call it twice and do not call it after the first Eval, because the method may rename packages.

Renaming happens on a name collision. In the stdlib two packages claim the name rand, math/rand and crypto/rand, so ImportUsed gives each its own: math_rand and crypto_rand. In a session that looks like this:

> math_rand.Intn(1)
0
> rand.Intn(1)
1:28: undefined: rand

For application packages that mechanism does not save you; we resolve their collisions ourselves in Exports(), covered separately below.

The important part is that renaming does not add a name, it replaces one: ImportUsed removes the old name from the scope. As long as the startup code has a single Eval line, the set of names does not depend on the order, and we checked both variants. But the more lines run at startup, the more code gets to execute before the renaming, and following the rule is cheaper than re-checking every time.

The result: ImportUsed binds the packages, import . unpacks the two values, and the session starts at an empty input line.

ImportUsed problems: the package name is the last segment of the key

After moving to ImportUsed the tests produced a confusing result:

> orders.UpdateAttrs{}         <- WORKS
> pagination.Settings{Page: 1} <- WORKS
> uuid.Parse("...")            <- WORKS
> models.Order{}               <- undefined type
> models.Item{}                <- undefined type

The packages are registered the same way, extract ran over all of them, so why are the domains visible and the models not? The answer is in the source of ImportUsed:

for k := range interp.binPkg {
    name := path.Base(k)        // the name in scope = the LAST segment of the key
    ...
}

Look at the keys. extract builds the key as path + package name, so for the domains the last segment always matches the name. But the models package exists in both domains:

Key	path.Base	Result
github.com/google/uuid/uuid	uuid	matches the package name
cosy-console/pkg/pagination/pagination	pagination	matches
cosy-console/internal/domain/orders/orders	orders	matches
cosy-console/internal/domain/orders/models/models	models	collision
cosy-console/internal/domain/catalog/models/models	models	the same name a second time

Two packages claim one name, models. You would think Yaegi solves this: the doc comment on ImportUsed promises renaming in the spirit of math_rand/crypto_rand. Reading the code shows why that does not fire on our paths:

// Handle collision by renaming old and new entries.
name2 := key2name(fixKey(sym.typ.path))
sc.sym[name2] = sym
if name2 != name {
    delete(sc.sym, name)   // and the simple name disappears
}

fixKey replaces the last slash of the path with an underscore. In the stdlib the paths are short and it all works out: math/rand becomes math_rand, a name you can actually type. Our path is deep: cosy-console/internal/domain/orders/models becomes cosy-console/internal/domain/orders_models, which is not an identifier and cannot be typed in a session. And the plain models has already been deleted.

We confirmed it with a clean experiment: after a collision both packages are unavailable: there is no models, no orders_models, nothing. Which of them would take the name if it were freed is decided by the map iteration order, that is, by chance.

The fix: substituting the keys after generation

Packages have to be registered not under their real paths but under keys shaped like console/<name>/<name>, where the last segment is the name the session will get. extract itself cannot do that: it builds the key rigidly as <import path>/<package name>, and none of its flags change that. So the keys have to be substituted in a step after generation. Here is what they should become:

key from extract                                     key we register it under
cosy-console/internal/domain/orders/orders           console/orders/orders
cosy-console/internal/domain/orders/models/models    console/orders_models/orders_models
cosy-console/internal/domain/catalog/models/models   console/catalog_models/catalog_models

The console/ segment in these keys means nothing: the key has to look like a path, so some segment has to come before the name. Yaegi only looks at the last segment, so the session gets orders, not console.orders. These keys have nothing to do with the console/console pseudo-package that delivers App and Ctx: there console was a real package name, here it is just the start of a substituted key.

As a bonus, distinct names (orders_models, catalog_models) read better in a REPL than one ambiguous models.

Now, how to do this properly. We do not touch the generated files at all: the edits would vanish on the first regeneration and the collision would come back. The first version of the solution kept a hand-written “session name -> extract key” map for the conflicting packages, but that turned out to be the same manual list: add a domain with models, don’t forget the line. The final version removed the list entirely: each package name is derived from the key itself. To make it visible how these files find each other, here are both halves in full. The generated one:

// internal/console/symbols/orders.go
// Code generated by 'yaegi extract cosy-console/internal/domain/orders'. DO NOT EDIT.

package symbols

func init() {
    Symbols["cosy-console/internal/domain/orders/orders"] = map[string]reflect.Value{
        "UpdateAttrs": reflect.ValueOf((*orders.UpdateAttrs)(nil)),
        ...
    }
}

And in the same package we declare the function Exports:

// internal/console/symbols/symbols.go
package symbols

// Symbols is the shared map of the package. It is declared empty and filled
// by the init() of the generated files, each under its own real import path.
var Symbols = interp.Exports{}

// Exports returns every generated table under substituted keys. The session
// name is derived from the key: for a top-level package it is the name of
// the package itself, for one nested in a registered package it is
// <parent>_<name>.
func Exports() (interp.Exports, error) {
    paths := importPaths() // paths of all registered packages
    exports := interp.Exports{}
    owners := map[string]string{}

    for key, syms := range Symbols {
        name := path.Base(key) // "orders", "models", ...

        // nested in a registered domain -> prefix with the parent
        if parent, ok := parentPackage(importPath(key), paths); ok {
            name = path.Base(parent) + "_" + name // "orders_models"
        }

        // two packages resolve to the same session name: name both culprits
        if prev, dup := owners[name]; dup {
            return nil, fmt.Errorf(
                "session name %q claimed by both %s and %s", name, prev, key)
        }

        owners[name] = key
        exports["console/"+name+"/"+name] = syms
    }

    return exports, nil
}

// importPath cuts off the last segment: extract registers a package as
// path.Join(importPath, packageName).
func importPath(key string) string {
    return strings.TrimSuffix(key, "/"+path.Base(key))
}

func importPaths() []string { /* paths from all Symbols keys */ }

// parentPackage finds the nearest registered package that contains the given
// path as a subdirectory: .../orders/models is nested in .../orders.
func parentPackage(pkgPath string, paths []string) (string, bool) { /* ... */ }

Here is what ends up in exports and goes into i.Use:

key in Symbols (from extract)	derived name	key in Exports()	name in the session
cosy-console/internal/domain/orders/orders	orders	console/orders/orders	orders
cosy-console/internal/domain/orders/models/models	orders_models (parent orders)	console/orders_models/orders_models	orders_models
cosy-console/internal/domain/catalog/catalog	catalog	console/catalog/catalog	catalog
cosy-console/internal/domain/catalog/models/models	catalog_models (parent catalog)	console/catalog_models/catalog_models	catalog_models
cosy-console/pkg/pagination/pagination	pagination	console/pagination/pagination	pagination
github.com/google/uuid/uuid	uuid	console/uuid/uuid	uuid

Where the merge happens. Nowhere explicitly: both halves write to and read from one package variable. symbols.go declares Symbols empty, the init() of each generated file puts its own table there under the real path, and by the time Exports is called every domain is in the map. There is no separate “collect everything” step; the language does it. Files of one package see one variable, so the generated files neither import symbols.go nor know about each other. Which is exactly why regenerating any file breaks nothing: the session names are derived from what extract has already generated, not from a list someone has to maintain.

A duplicate name is an error. In stock ImportUsed a collision is resolved by map iteration order, that is, by chance. Here exactly one overlap is possible: two identically named packages with no registered parents (two bare models). For that case owners returns an error naming all the culprits: the console fails at startup and tells you which keys collided, instead of silently dropping half the table. Nobody has to check for duplicates by hand or hope the compiler will.

Act 4: the REPL’s own rough edges

Multi-line input works. This is not obvious, so it is worth saying plainly: the stock REPL buffers an unfinished statement and waits for the rest. As long as a bracket is open, you can keep typing for as many lines as you like, a whole function if you want:

> err := App.Shared.Tx.InTransaction(Ctx, func(ctx context.Context) error {
> order, err := App.Orders.Repos.OrderRepo.Get(ctx, id)
> if err != nil {
> return err
> }
> fmt.Println(order.Status)
> return nil
> })

Usually a REPL hints that the input is unfinished by changing the prompt. That is what psql does: until the query is closed with a semicolon it waits for the rest and says so:

shop=# SELECT id,
shop-#        name          <- the prompt changed to "-#": input continues
shop-# FROM orders;

Yaegi has no such hint, and that is a limitation rather than a design choice. It has one prompt for every situation:

> err := App.Shared.Tx.InTransaction(Ctx, func(ctx context.Context) error {
> order, err := App.Orders.Repos.OrderRepo.Get(ctx, id)   <- the same "> ",
> return err                                              even though input
> })                                                      continues

After every Enter the REPL tries to execute what it has accumulated. If the parser trips over the end of input, the statement is unfinished: the line goes quietly into the buffer, and the next Enter repeats the attempt with it included. There is no separate “waiting for more” state inside, only a failed attempt to execute, so there is nothing to show in the prompt. The REPL in Yaegi is a helper utility, not a product. A proper input loop with state tracking is a separate parse cycle, and it is not in the code. Hence the rule: watch the output, not the prompt. Silence after Enter means the line went into the buffer, keep typing. A result or an error means the input ran.

The exception is a line break after a dot: you cannot split a call chain there.

> s := fmt.
> Sprintf("%d", 7)
> s
3:1: expected selector or type assertion, found '}'    <- there is no '}' in the input at all
1:28: undefined: Sprintf
1:28: undefined: s

The rule: never break a call chain after a dot: write fmt.Sprintf(...) whole, or break after a comma between arguments.

Ctrl+C must have exactly one handler. The stock REPL catches SIGINT itself and cancels the current statement, which is convenient: the prompt comes back, the session lives on. Meanwhile our first main() followed the habitual pattern of any service: signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM). The very first Ctrl+C was caught by both handlers: the REPL cancelled the statement, and NotifyContext cancelled the context we had exported into the session as Ctx.

The result is a zombie session: the prompt responds, the variables are alive, but every App.*.*(Ctx, ...) call dies with context canceled, and it stays that way until a restart. One line diagnoses it: Ctx.Err() returns context canceled instead of <nil>.

And one caveat about Ctrl+C itself: what gets cancelled is the interpreter’s execution, while a call into compiled code that has already started runs to completion. The prompt returns immediately, but a query in the database is only interrupted by statement_timeout or by cancelling the context that was passed into that call.

We made the fix: SIGINT stays with the REPL, the human at the terminal uses it to cancel the current statement, and main() listens for SIGTERM only. You cannot see this from the signature of Run, so the requirement is simple and worth keeping in mind: the context that goes into the session must not die on SIGINT.

err := f() silently takes the first value. Update returns two values, (models.Order, error). In ordinary code err := Update(...) would not compile: the compiler says assignment mismatch: 1 variable but ... returns 2 values. Yaegi accepts it and puts the first returned value into err, that is, the order:

> err := App.Orders.Repos.OrderRepo.Update(Ctx, id, attrs)   <- what you want to write
> err
{0 00000000-... {false 0} ...}                      <- but this is an Order, not an error

The error itself is lost. err holds the order, and whether the call failed cannot be told from it: on a failed call it will simply hold a zero struct. Write both names, res, err := ....

The working solution

Putting it all together: three parts in the repository (the console package, the symbols, the entry point) and two lines in the Dockerfile. The supporting pieces (runEval, Options, importPaths, parentPackage) are left out here; all of it is in cosy-console.

1. The console package: building the interpreter and two names

// internal/console/console.go
func Run(ctx context.Context, container *app.Container, opts Options) error {
    i, err := newInterpreter(ctx, container, opts)
    if err != nil {
        return err
    }

    if opts.Eval != "" {  // one-shot mode: -e 'statement'
        return runEval(ctx, i, opts.Eval, opts.Stdout)
    }

    return runREPL(ctx, i)
}

// newInterpreter assembles the interpreter: stdlib, the application symbols,
// the pseudo-package with App and Ctx, then the name binding.
func newInterpreter(
    ctx context.Context, container *app.Container, opts Options,
) (*interp.Interpreter, error) {
    i := interp.New(interp.Options{
        Stdin:  opts.Stdin,   // nil = the process streams; handy for tests
        Stdout: opts.Stdout,
        Stderr: opts.Stderr,
    })

    if err := i.Use(stdlib.Symbols); err != nil {
        return nil, fmt.Errorf("use stdlib symbols: %w", err)
    }

    symbolsExports, err := symbols.Exports()
    if err != nil {
        return nil, fmt.Errorf("build console symbols: %w", err)
    }
    if err := i.Use(symbolsExports); err != nil {
        return nil, fmt.Errorf("use app symbols: %w", err)
    }
    if err := i.Use(exports(ctx, container)); err != nil {
        return nil, fmt.Errorf("use console exports: %w", err)
    }

    // Declares the names of every registered package, the way yaegi's own
    // CLI REPL does. Strictly before the first Eval: ImportUsed may rename
    // packages.
    i.ImportUsed()

    // The only line executed on the user's behalf: App and Ctx live in a
    // pseudo-package, and the dot-import puts both names into the session scope.
    if _, err := i.Eval(`import . "console"`); err != nil {
        return nil, fmt.Errorf("import console: %w", err)
    }

    return i, nil
}

// The error from i.REPL() is discarded deliberately: it is the error of the
// last statement, the REPL has already printed it to stderr, and a clean
// Ctrl+D exit after a typo must not fail the process.
func runREPL(ctx context.Context, i *interp.Interpreter) error {
    done := make(chan struct{})
    go func() {
        defer close(done)
        _, _ = i.REPL()
    }()

    select {
    case <-done:
        return nil
    case <-ctx.Done():
        // the goroutine remains blocked on stdin until the process exits:
        // there is nothing to interrupt a terminal read with, and the
        // process is about to terminate anyway
        return ctx.Err()
    }
}

func exports(ctx context.Context, container *app.Container) interp.Exports {
    return interp.Exports{
        "console/console": {
            "App": reflect.ValueOf(container),
            "Ctx": reflect.ValueOf(ctx),
        },
    }
}

2. Symbols: a counted list, one file per package

internal/console/symbols/
  orders.go           <- extract of the domain (params, filters, errors)
  orders_models.go    <- extract of its models (models used as arguments)
  catalog.go          <- the same for catalog
  catalog_models.go
  pagination.go
  uuid.go             <- a hand-written minimum instead of a full extract
  symbols.go          <- var Symbols + Exports(): session names from the keys

The regeneration procedure is packed into make console-symbols DOMAIN=<domain>, see the extract walkthrough: extract writes the files straight here, one per package. The keys in the generated files are never edited: the names are derived by Exports().

3. The entry point: flags, signals, and read-only by default

// cmd/console/main.go
func main() {
    write := flag.Bool("write", false,
        "allow writes; by default the session is read-only")
    eval := flag.String("e", "", "evaluate a single statement and exit")
    flag.Parse()

    cfg, err := config.New()
    if err != nil {
        log.Fatalf("could not start console, %v", err)
    }

    l, err := logger.New(&cfg)
    if err != nil {
        log.Fatalf("could not create logger, %v", err)
    }

    // We do not listen for SIGINT: it belongs to the REPL (cancelling the
    // current statement). If we did, the first Ctrl+C would cancel the
    // exported Ctx.
    ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM)
    defer cancel()

    var opts []app.Option
    if !*write {
        opts = append(opts, app.WithReadOnlyDB()) // protection by default
    }

    container := app.NewContainer(&cfg, l, opts...)
    if err := console.Run(ctx, container, console.Options{Eval: *eval}); err != nil {
        if errors.Is(err, context.Canceled) {
            return // SIGTERM: a clean exit
        }
        log.Fatalf("console exited with error: %v", err)
    }
}

How read-only is turned on, and how to make sure it applied

Read-only, once on, forbids INSERT, UPDATE, DELETE, DDL and SELECT INTO. This is not an agreement along the lines of “don’t run console with -write”, which is easy to forget, but a refusal from the server: the write will not go through even if someone asks for it. It is turned on not in the connection string but in code, after the string is parsed. The technique is the same one we used to pin the time zone in the article about three sources of time:

pgxConfig, err := pgx.ParseConfig(dsn)
if err != nil {
    log.Fatalf("failed to parse postgres config: %v", err)
}
pgxConfig.RuntimeParams["TimeZone"] = "UTC"
if options.readOnly {
    pgxConfig.RuntimeParams["default_transaction_read_only"] = "on"
}

This is a session setting, not a check of ours: the parameter travels to every connection in the pool, and the refusal comes from the server:

> res, err := App.Orders.Repos.OrderRepo.Update(Ctx, id, attrs)
> err
ERROR: cannot execute UPDATE in a read-only transaction (SQLSTATE 25006)

But this is a safety catch, not a boundary. default_transaction_read_only only sets a default, and a transaction can override it: SET TRANSACTION READ WRITE inside App.Shared.DB.Begin() allows writes again. It prevents accidental writes, not deliberate ones. The real boundary is set by role privileges (grants on SELECT only) or by connecting to a replica, not by a session parameter.

There is a second caveat: the parameter may not be applied on the server at all. To let connections through PgBouncer come up, startup parameters it does not know are added to ignore_startup_parameters, and from then on the parameter is not rejected but silently dropped: the console works, only without protection. Recent PgBouncer builds paired with PostgreSQL 14 and newer pass it through themselves. But we do not want to depend on the pooler version. So we fail fast:

func validateReadOnly(sqlDB *sql.DB) {
    var value string
    if err := sqlDB.QueryRow("SHOW default_transaction_read_only").Scan(&value); err != nil {
        log.Fatalf("failed to read default_transaction_read_only: %v", err)
    }
    if !strings.EqualFold(strings.TrimSpace(value), "on") {
        log.Fatalf("default_transaction_read_only is %q, expected on: "+
            "the read-only startup parameter was overridden or stripped "+
            "(check the connection string and pooler config)", value)
    }
}

With that query we ask the server for its actual state instead of re-reading our own config. Failing at startup with a clear message beats a quietly running console that is not protected.

The full protection chain:

make console / /app/console        (no flag)
        |
        v
flag -write = false ---> app.WithReadOnlyDB() ---> postgres.WithReadOnly()
        |
        v
RuntimeParams["default_transaction_read_only"] = "on"  <- set the parameter
        |
        v
validateReadOnly: SHOW ---> not "on" ---> log.Fatalf   <- verify it applied
        |
        v
REPL ---> UPDATE ... ---> server: SQLSTATE 25006       <- the server enforces it

This chain covers the database only. If the application works with Kafka, Redis or another external API, those can be written to as well, and each needs its own restriction. The cheapest option is not to create a writing client at all, through the same app.Option as for the database. If the client is needed, the restriction is set by privileges on the service side: in Redis that is a separate user allowed only read commands, in Kafka it is topic permissions without the Write operation.

The default was not chosen by accident. Forgetting -write costs seconds: the first UPDATE fails. Forgetting the opposite flag, -read-only, would cost data. So the safe thing is the default and the dangerous one is enabled explicitly.

How to debug this

A protocol for when the console behaves strangely:

1. undefined type when constructing a domain type. The package is not registered, or it is registered under a key whose last segment is not the package name. Look at the keys in symbols/ and regenerate the extract: make console-symbols DOMAIN=<domain>, the files are overwritten in place (the procedure is in the extract walkthrough).
2. The package is registered, but its name is not in the session. The last segments of two keys collided, and after a collision both names are unavailable; Exports() fixes it (see the section on ImportUsed). A quick check against someone else’s symbol set: take rand from the stdlib, and if it is undefined too, you have hit the same crypto/rand versus math/rand mine.
3. Every call dies with context canceled. The context passed into Run is cancelled by SIGINT (NotifyContext(..., os.Interrupt) in main). One line diagnoses it: Ctx.Err(). SIGINT belongs to the REPL; main listens for SIGTERM only.
4. The symbols went stale after a refactoring. You renamed a field in UpdateAttrs and did not regenerate the symbol file. This is not silent: undefined shows up on exactly the type that changed.

Takeaways

• A separate binary with the same composition root: config and data shared with the API, the process its own. Transport is not part of the container, and the console does not need it.

• Two names go into the session, App and Ctx. Whatever is in the container is what is available, with no reflection over fields.

• Symbols are needed for input types only; output types are read without registration. The list of packages is counted from Container, not guessed.

• SIGINT belongs to the REPL, main() listens for SIGTERM only. Otherwise the first Ctrl+C kills the exported Ctx and the session turns into a zombie.

• Read-only by default is set in the connection parameters, and a SHOW at startup verifies that it applied: a pooler can strip the parameter, and then the console would come up unprotected. It is a safety catch against a slip, not a boundary; the boundary is set by role privileges.

A production session looks like this:

$ /app/console
> id := uuid.MustParse("00000000-0000-0000-0000-0000000000b2")
> o, err := App.Orders.Repos.OrderRepo.Get(Ctx, id)
> o.Status
paid
> o.TotalCents
1290

> res, err := App.Orders.Usecases.Order.Cancel(Ctx, id)   <- an application operation, not a SELECT
> err
ERROR: cannot execute UPDATE in a read-only transaction (SQLSTATE 25006)
                                       <- and this is correct behavior too

That is the Rails console experience we set out to recreate.

References

• cosy-console: the working console from this article;
• Yaegi: a Go interpreter written in Go;
• yaegi extract: the symbol table generator;
• gore: another REPL for Go;
• ImportUsed: collision renaming and the ban on calling it after the first Eval;
• golang.org/x/tools/go/packages: loading and parsing packages;
• startup parameters in the PgBouncer config.