Benchma(r)king

IN 1840, A SURVEYOR took a chisel to one of the outer walls of St. John’s Church, on Old Haymarket in Liverpool, England. They carved a horizontal line into the stone, a couple of inches long, and below this, three radiating lines, getting wider at the base: a broad arrow. The broad arrow had been used in England since the fourteenth century, and later across Britain and its colonies, to mark government property, but the horizontal line was a relatively new addition. Into this crevice the surveyor would have placed an angle iron, or “bench,” used to support a leveling rod, from which elevations could be taken. Thus, the broad arrow and the line were known as a “benchmark.”
 
The benchmark on St. John’s Church, recorded in the records of the government’s Ordnance Survey, its official mapping agency, marked one hundred feet above sea level. Chiseling it into the stone meant that someone else could come along later and set up a leveling rod in exactly the same position and take their own reading, transforming it into a datum line: a reference point for further, and repeatable, surveying.
 
For a few years, the benchmark on St. John’s was the reference datum for the whole country, until it was realized that more accurate measurements should begin at the mean tide level. A pole was set up in the Victoria Docks, also in Liverpool, and the fluctuations of the tide observed for nine days. The resulting mark was used as the fundamental reference until the 1920s, when it was shifted again, to Newlyn, in Cornwall. Benchmarks continued to be carved into churches, civic buildings, milestones, and country walls across the country, all referencing the tidal datum at Newlyn.
 
“Sea level” is actually different everywhere, but in Britain, it remains a function, plus or minus, of ODN: Ordnance Datum Newlyn. This is the level measured within the town’s tidal observatory, constructed on the South Pier in 1915. The observatory takes the form of a hollow tower in which several floats are suspended, connected to a recording device. An inlet pipe allows the tower’s well to fill and empty with the Atlantic tides. Between 1915 and 2015 the mean sea level at Newlyn increased by 1.83 mm/year—and by 3.8 mm/year between 1993 and 2014. Each change in sea level is reflected in an incremental adjustment to elevations across the land.
 
The last official benchmark was carved on a milestone located outside a pub in Loughton, Essex, in 1993, before satellite-based systems took over the datum (today, a GPS antenna sprouts from the top of the tide tower in Newlyn). The Ordnance Survey still maintains a record of over half a million benchmarks, although many more have disappeared. Hiking clubs lead expeditions to find them, and hobbyist benchmarkers check them off their lists. When you start looking for them, you see them everywhere.
 
“Benchmarking” remains in use today as a term for establishing a baseline against which other data can be measured and judged: software performance, product quality, profitability, growth; quality, time, cost. But it’s worth remembering its origins not as a tool for establishing hierarchies, but for leveling; for connecting, rather than separating; and for understanding the whole earth, and everything that happens upon it, as a continuous, indivisible field, stretched across cities, forests, and mountains, and ever susceptible to the rhythm of the tides.

A COUPLE OF YEARS AGO, I found myself stuck in an apartment in Athens, and in desperate need of distraction. I’d been living with my family on a Greek island, where we had space, and time, and had started to build a house; but at the end of 2023 my partner, Navine, became severely ill, and we had to return to the city we’d lived in for five years previously. She spent nine months in hospital; I visited every day, while also looking after our three-year-old son between nursery school and sleep and school again. I had less space, and a lot more stress, but I also had time to fill, and was itching to use my nascent carpentry skills. I needed to make something, with the limited tools and materials I had to hand—anything would do, it just needed to keep my body, and my mind, busy.
 
It was around this time that the current wave of what we’re calling artificial intelligence started to excite, and demand, attention. I actually studied AI at university, twenty years ago: exactly the wrong time, it turned out, to be studying it. The history of AI is one of booms and busts: “AI winters,” they were called, when the field seemed to sputter out after a series of advancements, and twenty years ago it was decidedly chilly. We learned about expert systems and neural networks from aging professors, but all the sensible students were learning about classical logic and statistics as preparation for careers in digital finance. It was another decade before the field would regain its energy, and—this is a controversial take—it would have little to do with conceptual breakthroughs. Rather, the growth of the internet, with its accompanying boom in processing capacity and data storage, would allow those same ideas—primarily neural networks—to be supercharged by graphics cards, digitized text and artworks, surveillance and social media, and the wholesale transference of everyday life to the online world. The AI “revolution,” if that’s what it is, is not a technological breakthrough, but the overwhelming accumulation of power and data in the hands of a few companies: power and data which rightfully belong to, and have been stolen from, everyone. But I digress.
 
I’d been noodling with this idea for some time: What if we used AI for something actually useful? It feels like we’re applying this new tool to all the wrong things: beating us at the games we enjoy, supercharging predatory capitalism, extracting and burning fossil fuels, depleting the planet’s resources, conflating images with art, encouraging extremism, suppressing rights, hallucinating, and so on and so forth. Wouldn’t it be infinitely more interesting if AI helped us do useful things in new and better ways, ways which we could not have imagined alone? Intelligences which we engaged with more like colleagues and collaborators than slaves and masters.
 
The one thing computers have always been really good at, one of the few qualities which genuinely impress me, is optimization. They can take a human-designed thing, an object or an algorithm, and ruthlessly strip it to the bare essentials while maintaining its functionality. Give a clever machine some widget, like a structural bracket or a complex workflow, and it will produce some alien-looking thing, a third of the weight and twice as efficient, which a human mind would not have conceived. That’s cool, that’s interesting. Of course, applied to culture, society, and the wider environment, such an idea of efficiency is a disaster, but it’s pretty neat for objects, as long as a human is keeping an eye on things.
 
I thought I might be able to make use of this particular skill to create something with what I had to hand: By combining my rudimentary carpentry abilities with the machine’s knack for efficiency and strangeness, something interesting might happen. Specifically, I didn’t want to design a process which would ultimately replace me, or take over the rewarding bits of work, but something more like a smart carpenter’s mate, that could do some of the sums and make useful suggestions and leave me to do the fiddling and the sanding. A collaborator, not a boss.
 
I measured up the various pieces of scrap wood on my balcony, left over and saved, Dad-style, from past shelving and cabinetry projects. A sheet of plywood, some pine panels, an L-shaped piece of chipboard, and half a dozen bits of dowel and beam. I had a saw, a drill, assorted screws, and some wood glue. And I gave this information to OpenAI’s ChatGPT—then running the version known as GPT-4, its second publicly accessible iteration—and asked it to design me a chair.

I’D MADE A FEW CHAIRS BEFORE. Chairs are interesting, and trickier than you expect. They have the simplest of functions—keep your arse up off the ground—and infinite ways of achieving it. And as anyone who’s ever sat in one knows, they fail in all kinds of ways. Cheap chairs wobble and buckle, materials snap and wear out; some chairs produce a deep agony after only minutes of sitting in them. Making one that stands up, supports an adult human, and doesn’t hurt is actually quite a feat. I’d tried my hand at making a few chairs in the past, and my score was usually two out of three.
 
My main instructor in chair-making, and much else, is Enzo Mari. Mari was an Italian designer who came to fame in the 1950s and ’60s, in the postwar heyday of Italian industrial design, producing thousands of pieces of homeware and furnishings from the exciting new materials of the time: injection-molded plastics, tubular metal, and mass-produced modular cardboard and plywood forms. But Mari was also a contrarian: “When I design an object and people say: ‘oh, well done!,’ I unfailingly ask myself: Where did I go wrong?” he once told a magazine writer. “If everybody likes it, it means I have confirmed the existing reality and this is precisely what I don’t want.”
 
This “existing reality” was a problem he ran headfirst into when, in 1971, he designed a neat little sofa bed for the highly fashionable Italian furniture company of Maddalena De Padova. Mari aimed squarely at functionality, and his sofa bed resembled a plain padded bench with a backrest which dropped down to make room for sleep. Despite, or perhaps because of, this simplicity, his design was rejected, and when another company took it on, it failed to sell. Mari blamed this failure on the fact that it cost too little: Mass production, he believed, had made people careless of the true utility of things, and “design” had been debased to the mere production of status symbols, the visible representations of consumers’ wealth and position.
 
To repair this lost connection with the real value of things, Mari decided, it was necessary to involve the consumer—the user of things—in the design and making of objects. To do so, it was necessary to reject most of the techniques, processes, and materials of industrial design and go back to basics: “Everybody has at least tried to hammer in a nail,” he thought, and with that skill alone it was possible to produce much of what was sold in high-end boutiques. So, with the help of a few assistants, he set to work—directly, with little or no preliminary design—to make a series of chairs, tables, beds, benches, wardrobes, and bookshelves. The only materials required would be wood and nails, and only when the pieces were completed would technical drawings be produced. Eventually, nineteen designs for chairs, tables, bedframes, a shelving unit, and a cupboard, alongside photographs and simple instructions, were collected into a catalogue, published in 1974 under the title Proposta per un’autoprogettazione, or “Self-design proposal.”

These instructions, which Mari undertook to mail to anyone who asked for them, came with a cover letter: “The author hopes this operation may remain an open process and invites those who will build these furniture pieces, and in particular variations of them, to send photos to his studio.” Over the decades, thousands of people have taken up this invitation. A few years ago, when Navine and I got ourselves our own studio space on the island, we decided to join them.
 
The space we had rented was a former shop in town, glass-fronted, with high-ceilings. The previous tenants, a furniture store, had installed an upper floor, which had been dismantled, leaving rudely cut metal pillars and huge sheets of MDF—thick, heavy, chemically-bonded wood—stacked in the empty space. We had no furniture, but we had a saw, hammers, nails, and screws, and we had Mari’s instructions, in the form of a badly photocopied PDF downloaded from the internet—and so we decided to follow them. We enlisted a few friends, hoping that the experience of setting up the space together would encourage them to stick around in it with us; in a couple of cases, it did.
 
The instructions in Autoprogettazione are pretty rudimentary: a couple of technical drawings with cutting guides for each piece, and a black-and-white photograph of the finished object. But they are simple objects: We cut tabletops to size from the giant sheets of MDF and used the offcuts to make the legs and supports. We broke numerous drill bits making pilot holes in the dense wood; we used screws that were too short, and too thin (electric screwdrivers are a modern blessing, but screws don’t hold like nails do); so we made more holes, and extra holes, and they stood. I’m writing this on one of them now.

The tables are great, but it was the chairs that really did it for us. Mari gives several examples, and we went with the design that appeared to make the most of all the little pieces of wood we had accumulated. This is in line with Mari’s intent, stated in the catalogue, to teach critical appreciation and architectural principles through the basics of carpentry: “Because the economics of the carpenter’s job force him [or her, or them] to completely recoup the material used for a certain construction, he [or she, or ze] soon gets accustomed to using diagonal strengthening parts with a minimum of waste”—in short, one learns how to build by actually having to build.
 
I have a set of pictures from that day: The studio is covered in dust, a scaffolding frame stands in one corner, the place is littered with offcuts. Our friend Alessandro wields our trusty plunge saw, hooked up ineffectively to a vacuum cleaner. In one photo, Navine reclines, proudly smiling and at ease in the chair she built; in another I sit on my creation, safety goggles pushed back on my forehead and a respirator dangling around my neck, with an idiot grin of disbelief on my face. These images accurately represent our characters: Navine knew that we could do it; I am almost overwhelmed with joy that I have not come crashing to the floor. Something that I built, that I put together with my own hands, is holding me up.
 
These pieces of furniture are neither light, nor sleek. When Bela, our Hungarian painter-decorator, came by the studio one evening, he took one look at them and said they looked “like something my uncle would make when he was drunk.” We didn’t care. They’re chairs and tables: They hold your arse up off the ground and give you a place to work.
 
Moreover, these simple pieces of furniture taught us how to build. Mari called them autoprogettazione because he didn’t want people to simply reproduce them, but to develop them, to “self-design” variants on his initial instructions, based on what was actually required. “In 99% of cases,” he later wrote, eternally pessimistic, “it was not understood.” His archive contains thousands of letters from people who built his furniture, but only rarely did they create variants on the original designs. Even when, in 2010, a Finnish company offered one of the chairs for sale, under Mari’s direction, in two editions—as a built object, and as a kit—the kit version did not sell. I’m happy to say, that’s not how it worked for us: We built the chairs, and we built the tables, and then we made benches, and desks, and shelving, and a kitchen unit, in the spirit of Enzo, but to our own designs and specifications. They creak, sometimes collapse, they’re incredibly heavy and fiendishly uncomfortable without cushions—two out of three, I said—but they are infinitely repairable and reconfigurable, and we live and work with them today.

SO, WHAT WOULD CHATGPT MAKE of an autoprogettazione project? It took a few goes to get the machine to understand what I wanted—and critically, that I wanted it to use the materials I had available in the most economical way, with minimal waste. The machine talked a lot, unlike a good carpenter’s mate, in its customary enthusiastic and overconfident style (“That sounds like a great project!”), but it came back with a set of suggestions on how to go about it, and I tried to follow them. I started with a pencil drawing, and then proceeded to cut the wood to length, and to assemble the chair.
 
There are two qualities of the chair that ChatGPT and I built that stand out. The first is that it looks flimsier than it is, but it works. The AI told me to use a sheet of very thin ply for the seat, so thin that I considered laminating two sheets of it together. It gives a little when you sit on it—but it holds. That’s exactly the kind of optimization I was looking for: a decision I wouldn’t have had the confidence to make, but, again, it works. It might even be slightly comfier that way—although I have no way of knowing if the machine actually intended it, structurally or comfort-wise; it might have been going on dimensions alone. Still, the chair stands.
 
The second quality of the chair is that it is unnervingly big. Not too big—it’s actually pretty comfortable for a six-footer like me—but it’s definitely bigger than you’d expect, verging on the uncanny. This I find to be absolutely fascinating.
 
What we have here is, quite literally, a chair designed by someone who has read a lot about chairs but never actually sat in one. They know what a chair should look like, what it should do, the mechanics of how one is put together, the physics of holding you up. But they have no embodied experience of the world, no history with chairs or any other object, no physical memory. They don’t know how it feels, not merely to sit in a chair, but to encounter one in the world, to place it among other things in a home, to live with it, to be around it every day as one goes about the mundane business of life. This is the idea of a chair brought to life; but things—objects, beings, relationships—are not mere ideas, and you certainly can’t get every aspect of them from books. The chair is low stakes, but, once again, you wouldn’t want to run a society, or a culture, or a planet this way.

SINCE BUILDING THAT FIRST CHAIR, others have been created. This is where the benchmarking bit comes in. A few weeks after the ChatGPT chair, I made another one, this time with an AI model called Mistral. Mistral is a French company which made a lot of claims to be more ethical than ChatGPT (whatever that really means), and I’d been told it was more accurate and precise when it came to technical matters. Once again, I described my request, and we went back and forth a bit, and then I tried to do what it told me.
 
Mistral’s chair didn’t stand up at all without some serious modifications: I had to add a bunch of cleats and bracing around the legs to make it stable enough to sit on, which eliminated any purported gains in efficiency. It was definitely more interesting than ChatGPT’s doll’s-house effort, with a short back and slanted armrests. It was also waaaayyy too high. My feet didn’t even touch the floor. So much for Mistral’s accuracy and precision—but points for making something a daring European chair designer might actually have come up with.
 
AI is unfortunately very hot right now. At a time when we could be attending to decarbonizing our societies and building more robust communities to help us through the turbulent years ahead, we seem to be giving most of our attention to a technology which not only burns through energy and other resources, but actively works to make our relationships less secure and our information environment less coherent. The art world is not immune to this tendency, and so when I received requests for AI-themed work for events and exhibitions, I turned the chair-making process into a set of instructions which anyone could follow, and sent it over.
 
The first iteration took place in Tokyo at an exhibition called “AI is Not Magic” in June of 2025. The curators followed my instructions, inviting Anthropic’s Claude AI to make them a chair from a selection of materials in the studio. It turned out a pretty nice chair, but the quality of the finish made me suspect that an experienced carpenter was involved at their end. Conversely, when a conference called ThingsCon in Amsterdam asked Elon Musk’s Grok AI to have a go, the system told them to build a lounge chair and gave very precise cutting instructions, but none of it fit together at all. As the ThingsCon team noted, “The system could produce—confidently, fluently, at length. But it couldn’t understand what it had made.” The chair they eventually ended up with, after much modification, was “grammatically coherent and practically useless. […] It’s a strange object—part furniture, part evidence.”
 
Each chair is indeed a piece of evidence—a benchmark, in the truest sense. Each iteration of the work produces a different outcome, which we can judge and compare. Each AI model reveals its own tendencies, biases, and limits, in physical form, easily appraised by a layperson. We are benchmarking; we are benchmaking; we are literally making a bench (fine, a chair, whatever). I appreciate processes where you learn to do a thing, a thing that you didn’t know how to do before, and at the end of that process you have not only done it, but you can see the results. And, in this case, sit on them.
 
This benchmarking has a deeper meaning as well. Carving a broad arrow into a church wall doesn’t reduce or isolate anything. Rather, it reminds us that every action is made in context; each benchmark refers to every other one, in a continuous, connected line through space and time. When we build, when we think, when we act, we do so in honor of every object, thought, and action which has preceded this one, and we are changed by the process. AI wants us to forget this, by making every output its own miraculous creation. But we owe the world more than that.

MARI SAID THAT the autoprogettazione designs were intended to teach people how to build beyond his instructions, to “self-design” their own furniture. His aim was expansionist and elevating: He wanted to empower people by giving them the skills and appreciation to make better decisions about the thing which surrounded them. This is what happened to me, when I followed his instructions.

When we use AI to similar ends, who is actually doing the learning? Building the AI chairs taught me something about AI, but it didn’t teach me much about making chairs. Ultimately, I was the teacher, taking what I’d learned elsewhere to make the “practically useless” suggestions into something stable and self-supporting. The AI undoubtedly learned more than me from these interactions: how to speak, what interests (some) humans, how to flatter, how to slant and tailor its responses to my demands.

The anarchist anthropologist David Graeber, may he rest in peace, had a good bit on authority. He said that anarchists don’t oppose authority per se, but they study and critique it, and appraise its legitimacy. What he liked, he said, was self-subverting authority, authority which undermines its own basis. He gave the example of a teacher: A teacher knows something useful to their student, and so they have a natural authority; but if they do their job well, at the end of the process the student knows what the teacher knows, and their authority is spent. The hierarchy is flattened, and no further unequal power can accrue. The student is changed, and so is the world.

Of course it’s possible to learn things from AI, but, for me, real intelligence resides not in an ability to extract and dominate, but in the capacity to share, co-exist, and thrive together. There are many ways of being intelligent, as many as there are living beings, each one unique and valuable. The real challenge of AI is not in building ever more powerful systems—this is difficult, but inevitable—the challenge is to find what relationships between different intelligences benefit us all. The challenge is to choose the kind of world we all want to inhabit. Benchmarking—the practical, embodied work of exploration and mediation—is an attempt to find our way towards such a world.