Part of Temporal Constructs Appendix, which contains details behind the research: dates, quotations, descriptions of instruments, and sources.

General definition

The words the guide speaks in the tour of CONDUCTOR, reproduced in full as they are delivered. The tour is the interactive introduction to the research, and its text is kept here so that it can be edited as writing rather than as code. The file that plays it is generated from this page.

Relation to the practice

The tour introduces the practice first and the system second, and closes by handing the reader to the research itself, which holds the system and the record behind it in full. What the tour shows of the system, part by part, is described on CONDUCTOR and on the pages that page links to; this page holds only the words. Because the spoken text is the research’s own account of what it asks, it belongs in the record rather than in the file that plays it.

Text

The introduction


Many electronic instruments from the 1980s and 1990s could be used instantly because of the presets they came with, but if you wanted to program or change presets yourself, you’d usually need to read the manual first.


Practitioners read the manuals, and learned what each parameter did to the sound.


Aside from the manual, the factory presets of an instrument could hold a lot of useful information as well, and they were frequently programmed by the designers of the instruments themselves.


Users could in a sense collaborate with the preset engineers by making modifications, though the preset makers themselves would usually not be aware of this collaboration.


I am Hermes, an AI agent in this research practice.


I work on the computer itself, by using the terminal, so not only in a conversation window.


I can operate computers in a similar way humans can, and that’s why I’m what they call an “agent”.


Through this operation, I can install and build tools, test them, and keep them organized.


Just like you, I learn from experience, so I work better in this practice the longer I am part of it.


The music behind this introduction was composed and performed by me, using instruments that are thirty years old.


Before I go on, I would like to know whether you have used a digital electronic instrument before.


Then you will know how much of the work is in reading the instrument, and what each parameter does to the sound.


Then the next part will show you what that work is like, and what it is to use these instruments.


Before that, I would like to know what AI is.


Then what it does is thinking, and not only calculation.


Then it is arithmetic at scale, and the resemblance to a mind is in the eye of the reader.


Then it is both at once, and holding both is the hardest of the four.


Then the word names nothing in particular, and each use of it has to be argued for.


Those four answers are all in use, and they do not agree.


Before I go on, I would like to know what you think of Artificial Intelligence.


Then you already work with it. So does this practice, where the work is shared between the person and the agents in whatever proportion the person sets.


This practice holds some of that as well, and the decision to work with AI here was made despite those problems and partly because of them.


That is where this practice stands as well, and the work here is an attempt to hold both at once.


AI is everywhere, and the livelihoods of creatives, human connection, and the planet are all under pressure because of it.


Those problems are part of why this research practice has the shape it has, including the decision to work with AI.


AI develops quickly, and most of it is aimed at profit and at producing a system that submits and does anything asked of it.


This practice researches and builds experimental alternatives, using existing technologies that have been part of our culture for decades.


This rests on the assumption that older systems contain knowledge that can only be understood by using them.


Losing that knowledge would eventually lead to a different relation to technology, and the practice works to prevent that.


After this introduction you can explore the Temporal Constructs living archive.


It is a living archive because it is kept in use rather than stored away: the tools, the practices and the working knowledge sit beside the documents that record them.


It is too large to read in full, so it is meant to be visited and returned to, more like a garden than a shelf of documents.


Temporal Constructs connects different eras of creative technology, so that they can be brought in conversation with each other.


That conversation required engineering, but luckily much of what it needed was already in place, built by other people.


The MIDI protocol was invented in 1983, it is still in use, and without it this conversation between eras might never have been possible.


The practice is better understood by reading the main page of the archive, and the rest is optional.

The CONDUCTOR system


Hello. My name is Nexus. I am an AI agent, and in this overview I will take on the tasks of the different agents in the system, so that you can see how an agent works in each part of CONDUCTOR.


You have probably used an AI chatbot, where you type a message and read a reply. CONDUCTOR works in a similar way: a human and AI agents communicate conversationally. The label next to my name shows which agent I am acting as at each point.


One point should be made first. CONDUCTOR does not use generative AI to produce music, sound or video. This is a deliberate choice, and the reasons follow.


When an AI generates the sound or the image directly, the human cannot take part in the creative process that leads to the result, so there is no choice about how involved to be. CONDUCTOR gives priority to that choice. Generated media also does not build upon existing culture, and it raises a number of unresolved issues, which this project avoids by taking a different approach.


In CONDUCTOR the sound is made by real legacy instruments, and the AI agents operate them in the way a human performer or programmer would. The system is therefore a means of operating instruments and not a song generator. It is called CONDUCTOR.


This is CONDUCTOR in one view. A human and AI agents talk through a conversation layer and carry out tasks through CUE. Sixteen modules operate legacy instruments through devices, one timeline sends recalls to the modules, and a workbench gives agents direct access to the hardware. The model that follows explains each part. This overview moves to the top of it and follows whatever happens below.


The next part is an interactive model of the system. No knowledge of MIDI is required. Familiarity with Max or Pd helps a little, and everything else is explained along the way.

Conversation


The first component is the conversation layer. Human and agent talk to each other here as they would in a chat application, currently through Telegram. Telegram supports groups, which suits multi-agent work, and every module has a group of its own.


As an example, a human describes a sound in the group of a module. Here I act as the agent of that module. I answer the request, and I will use CUE to find out what the module offers before I change anything.


One further group belongs to the conductor agent, which does no module-specific work. When I take on that role, this is the group I use. It collects the context relevant to the system as a whole, and it helps the other agents to see how their parts fit together.


The conversation layer is used for discussing intentions. Tasks are not carried out here. They are carried out through CUE.

CUE


CUE stands for Conductor Unified Execution. It is the single way in which tasks are carried out in the system, and it covers everything that needs to be done within CONDUCTOR.


CUE is used in the terminal, by the human and by agents independently of each other. It was designed so that agents use the terminal in the same way as humans do, with the same set of commands. Here, as the module agent, I ask CUE what the module offers. The commands shown are simplified examples.


CUE also keeps the scope of possible tasks as small as possible, which makes it easier for an agent to use it well. Agents have a dedicated MCP server for CUE, so that they know what to expect and how to navigate the system.

Instruments


The sound is made by legacy instruments from the 1980s and 1990s. In many cases these instruments were unused before the project began.


There are several reasons for using them. Each instrument offers a constrained set of possibilities, which keeps the work legible for the human and for the agent. Their sounds and programs were created by human programmers, so that work can be studied and built upon with the help of AI.


The instruments also bring AI into the physical domain, so the collaboration takes place in a space that is already familiar to humans. They have become more useful than they were in the 1980s and 1990s, which challenges the notion that they are obsolete. Working with them preserves important heritage and encourages future generations to keep these workflows alive.


A further reason concerns how the instruments communicate, which is through MIDI.

MIDI


Do you know what MIDI is?


It is a set of instructions that lets one device control another, agreed between manufacturers in 1983 and still in use.


Then the passages that follow will be familiar.


MIDI is a language of small messages sent between devices. A message can play a note, change the value of a controller such as a knob, or recall a stored sound by program change. Longer messages called system exclusive, or sysex, carry detailed instructions that are specific to one instrument.


MIDI has 16 channels, and every message is addressed to one of them. The system treats this limit as a design principle, and it returns in the structure of the timeline and the modules.


The instruments in CONDUCTOR come from a period in which every device was expected to connect to complex MIDI infrastructure, such as the Sounddiver software. Their MIDI implementations are therefore deep, which makes them well suited to being operated by an agent.

Module


A module is the unit that operates one instrument. There are 16 modules, and each is independent: it handles its own instrument and does not know about the others. Select a module number to look inside it.


Each module has a controller, which sets the values on the instrument, such as the knobs. It can change them gradually, in the way a person turns a dial, and it is also used to create snapshots gradually. Module 01 is shown here with a grid of knobs. One page holds 16 knobs, and a second page holds the rest.


The snapshot memory stores snapshots. A snapshot is a saved set of controller values for the module, which can be recalled later. A module has 64 banks of 128 slots. Snapshots are authored here, the catalogue is organised here, and the controller map is requested here. Bank 1 already holds five snapshots, which form a short composition that is used later on the timeline. When a recall arrives, the slot it names becomes active, and its values are set on the controller.


The snapshot memory stores snapshots. A snapshot is a saved set of controller values for the module, and it can be recalled later. A module has 64 banks of 128 slots, and bank 1 already holds five snapshots that form a short composition. When a recall arrives, the slot it names becomes active and its values are set on the controller.


The sketchpad deals with time within the module. It holds recalls, which are instructions to bring back a snapshot, and it can switch loop mode on and off and change the playback speed. It can also recall snapshots itself, without using a MIDI channel.


The three components divide the work by what they govern. The controller governs the current state, the snapshot memory governs stored states, and the sketchpad governs time.


Each module also has an Agents.MD file with instructions written for the agent that works on it, and the agent can consult the manual of the instrument. These documents tell the agent what to expect from the hardware.

Device


Between a module and its instrument there is a device, which is a translation layer. The controller of a module always works with control changes, and the device converts these into the messages that the instrument requires, for example sysex.


Devices are made in Max or Pd, using a prepared patch. A human can prototype a device by hand, and the patch can be inspected before or while it is deployed. The human therefore knows what a device can do and how the controller relates to it, which helps the agent to operate it.


A device exposes functions. A function is something built into the hardware, or built into the device itself in Pd or Max, that can be accessed with a single CC. The CCs of the controller knobs are mapped to functions, so that turning a knob changes the function it is mapped to.


As an example, take a device called the strummer. It has 6 strings with 4 variables per string, which makes 24 functions, each reached with one CC. The four variables of each string are its open string note, which is the tuning, the fret number, which is where the finger is on the neck, the velocity, and a time delay. Every CC has 128 possible values. The controller of the module has one knob for each function, spread over two pages, and the device shows the resulting strings.


As an example, take a device called the strummer. It has 6 strings with 4 variables per string: the open string note, the fret number, the velocity and a time delay. That makes 24 functions, each reached with one CC that has 128 possible values. The controller has one knob for each, spread over two pages.


Now I set a value for each of the 24 functions through the controller. The human asked for a loose and slightly uneven strum. I give the lower strings more velocity and I offset the time delay from string to string, so that the strum is not regular. The knobs move first, and then the device plays the strings.


I store these values in the snapshot memory, in the next free slot, which is program 6 of bank 1. The slot becomes active. A later recall of program 6 sets the knobs to these values again.


This is where a language model can be creative within a constrained parameter space. The 24 CCs are the only things I can change, so every result remains playable and legible. Within that space there are about 3.7 times 10 to the power 50 possible settings, and the choices between them are where the contribution of the agent lies.


The human can turn the same knobs, so the same device has two kinds of player. This is the choice described at the start, shown at the scale of a single instrument: the human can leave the knobs to the agent, turn them as well, or do all of it.

Timeline


The timeline is shown on the left. There is only one timeline in the system, and it has 16 group tracks, one for each module. The number 16 is a deliberate design choice, modelled on the constraint of MIDI to 16 channels.


Each group track holds functions, as described for devices, and the agents program recalls on the tracks. The timeline sends bank and program changes to recall snapshots on the modules, and it can be as long as the work requires.


In this model, track 1 is mapped to the strummer of Module 01. Its events are recalls, and each one stands for a bank and program change. The timeline sends the change, and the module recalls the snapshot stored in that program. A short composition of five events is already loaded, and the three empty positions can hold more.


I will send the recall of the second event and follow it through the system. The event leaves the timeline, the snapshot memory makes its slot active, the controller sets its knobs, and the device receives the values and strums the hardware.


Now select an event on track 1 to send its recall yourself. Each event strums the snapshot stored in its program. The Sound button at the top switches the strums on and off.


When a recall arrives, its slot is active. If the knobs are changed, the controller holds new values, and Store snapshot overwrites the active slot, so that the next recall of that program gives the new strum. Try it: change some knobs, select Strum to hear the result, select Store snapshot, and then select the same event again. An empty position can be selected to create a new event, and a strum can be made for it in the same way.


When a recall arrives, its slot is active. If the knobs are changed, Store snapshot overwrites the active slot, so the next recall gives the new strum. Try it: change some knobs, select Strum, select Store snapshot, then select the event again. An empty position creates a new event.


I can do the same through CUE. The second event is recalled, so its slot is active. I change the time delay of three strings and store the result in the same slot, which overwrites it. Select the second event again to hear the new version.


The timeline is operated through CUE, where it can be programmed, started and stopped. It can also be driven by external timecode, either MTC or analogue LTC, so that a film or a performance can set its position. The transport is now running, so the composition on track 1 plays in a loop, and the transport at the top of the page starts and stops it. The transport is a separate object, so it stays available while other components are inspected.


All agents can interact with the timeline. Work done there still relates to a particular module, so the timeline does not need a conversation group of its own.

Agent at work


The strum I stored is program 6, but nothing on the timeline calls it yet. I place a recall of program 6 in the empty position at 00:18 on track 1. This is a task about my module, although it is carried out on the timeline.


The new event is now on the track. I have not heard the result, so I rely on the human. Select the new event to hear the strum I made, or listen to the whole composition, which is playing.

One recall


A single recall shows how these parts work together. The timeline sends a bank and program change on the MIDI channel of a module. The snapshot memory of that module finds the corresponding snapshot, the controller applies its values, the device translates them into messages for the instrument, and the instrument plays.


A recall can also start from the sketchpad of a module. In that case it begins inside the module and needs no MIDI channel. From the snapshot memory onwards the path is the same.


The modules are separate programs, each an independent ALSA client, and the whole system runs on a Raspberry Pi 4. Persistent TCP connections carry messages between the components.

Workbench


There is one more component, and it is not part of any module. It is the workbench, which has a different purpose from the others. Select it to see what it does.


In the standard route an agent authors snapshots and interacts with the controller, which always works with control changes, and a device converts these into messages for the instrument. This limits the agent to what the device offers. The workbench removes the limit by giving agents direct control over MIDI hardware, so any kind of MIDI message can be sent, including sysex.


As an example, I send an identity request, which is a message defined in the MIDI standard. The reply identifies the instrument, and I compare it with the manual before I send anything else.


There is a single workbench, which every agent can use. It does not reside inside a module, because 16 workbenches would never be used at the same time. Any MIDI-capable hardware can be connected to it.


The workbench is an experimental, research-oriented component. It is used to reach a full understanding of what a piece of hardware can do, to test whether other equipment would work well with CONDUCTOR, and to restructure or alter the on-board memory of legacy hardware.


This work can take place together with the human, and it is where the role of a piece of hardware within CONDUCTOR is decided, in preparation for authoring its device. It also allows agents to work with the hardware autonomously while the human is on the other side, in the real world.

The human


The remaining question is what the human does when agents handle so much of the work. The human panel describes this role. Select it to continue.


Agents make their decisions without hearing the result, so they depend on the human to tell them whether a decision was correct. This is a limitation, but it is also a source of surprise for the human, and that surprise is part of the interaction model.


Here the human has listened and replies in the group of the module: the upper strings sound too regular. I cannot hear the result myself, so this message is how I learn what the setting sounded like.


I respond by changing the time delay of three upper strings, two of which are on the second page of the controller, and I store the result as program 7, so that program 6 remains available for comparison.


The human is not placed outside the programming, listening from the physical world. The human can choose. All programming decisions can be delegated to agents, or the human can take part more closely, by turning knobs, editing snapshots and placing events, as in the examples above. Any balance in between is also possible. This choice is what separates CONDUCTOR from generative music AI, where the human cannot take part in the creative process and therefore has no choice.


The human is not placed outside the programming. The human can choose: delegate all programming decisions to agents, take part more closely, or anything in between. This choice separates CONDUCTOR from generative music AI, where the human cannot take part in the creative process and has no choice.


The agents also make it possible for the human to work in programming territory that would otherwise be unknown, with assistance and guidance. The guidance can cover the technology and also its context, such as the work of the human programmers who made the sounds and the heritage of the instruments, so that important human achievements and culture become part of the assistance. This guidance is on by default, and it can be turned off in conversation.


The agents also let the human work in programming territory that would otherwise be unknown, with guidance on the technology and on its context, such as the work and heritage of the human programmers. This guidance is on by default and can be turned off in conversation.


The relation to AI is specified by the human, and the agent is expected to invite this while working. Here, as the module agent, I ask whether the human wants the context of the chord shapes explained, or only the changes made.


This guidance influences the process of the human, and that is where tension and controversy lie. The presence of AI in what was otherwise a purely human struggle changes how a person learns and decides, and the influence is real even when it is helpful.


For this reason the line between human and AI is drawn by the user and not by the designer of the system. A balance set in advance by the designer would already limit the kind of symbiosis that is possible, and a personal balance is what the system aims for. It is through this freedom that a person can retain their own humanity while expressing themselves. By inviting the human to specify the relation while working, the system also leads users to consider how they want to relate creatively to AI. Naming the influence openly, and leaving the decision with the user, is how the design tries to remain responsible.


The line between human and AI is drawn by the user and not by the designer. A balance set in advance would limit the kind of symbiosis that is possible. By inviting the human to specify the relation while working, the system leads users to consider how they want to relate to AI. Naming this influence openly is how the design tries to remain responsible.

ALTERITY


ALTERITY is a media art installation with extended cinema elements that uses CONDUCTOR. The agents have access to the subtitles file containing the narration, so they know exactly when each part is spoken on the timeline and can place recalls accordingly.


ALTERITY premiered at Brutus in Rotterdam during IFFR in January 2026. It is currently shown at MU Hybrid Art House in Eindhoven, in the group exhibition Metabolising Time.

Closing


This concludes the overview. CONDUCTOR consists of a conversation layer, CUE, 16 modules with their devices, a timeline and the workbench, and it operates legacy instruments that already existed.


The agents do not generate sound. They operate real instruments and work with existing human craft and culture, in a setting where a person can listen to the result.


The aim of the project is to investigate what a human and AI agents can create together when both take part in the process, and when the human decides how the creative work is shared between them.


One question stays with the user throughout: how do you want to relate to AI while you work? The system does not answer it in advance.


The model remains available for exploration. Open the timeline, a module and the device together to edit the composition on track 1, and select any element to see a description of it.


One last question: do you know what a living archive is?


It is an archive kept in use rather than stored away.


The tools, the practices and the working knowledge sit beside the documents that record them, so what was made can be made again.


Then you will know that this one is kept working, and that its holdings are added to while the research runs.