Standard reference peanut butter is an example of the power of measurement. The creation of the US National Institute of Standards and Technology was sold to industry at a price of $1,069 for three 170g jars. The cost is not due to rare ingredients or a complex production process. The contents of each jar have been analysed. The peanut butter has been frozen, heated, evaporated, and shorn so that it can be measured across multiple dimensions. When buying a jar, buyers can be certain of the exact proportion of sugars and fibre in the jar, as well as the prevalence of dozens of different organic compounds and trace elements. The most categorically known peanut butter in existence is contained in these jars. It is also smooth.

NIST created a library of more than 1,300 standard reference materials to meet the demands of industry and government. It is a testament to the importance of unseen measures in our lives and is a bible of contemporary measurement. From the emission levels of a new diesel engine to the optical properties of glass destined for high-powered lasers, the standards against which checks can be made can be found in the SRM catalogue. Concrete and iron are used in the construction trade, while slurried spinach and powdered cocoa are used in the food industry. Ingots of purified elements and pressurised canisters of gases are available in finely graded blends and mixtures. As if they were the creation of an overly zealous bureaucracy determined to standardise even the most peculiar substances. Domestic sludge, whale blubber, and powdered radioactive human lung are available.

Each has a purpose. Environmental agencies use domestic sludge as a reference to check pollutant levels in factories. Scientists use standardised whale blubber to track the build up of chemical contaminants in the ocean. Human exposure to radioactive materials is benchmarked by powdered lung. Cold war fears of nuclear destruction are what triggered it. The samples were created using 70 kilogram of human lung donated by employees of Los Alamos National Laboratory. The donors had been exposed to radiation during their lives and offered their bodies to science after death. The lungs had to be freeze-dried and pulverised into a fine powder to ensure homogeneity in each sample, which makes for some amusingly straight-faced lab notes.

Steve Choquette is the director of the agency's Office of Reference Materials. His job is to make sure that customers have total trust in the agency's measurement. If Choquette has any questions about the measurement, he can walk across the aisle and talk to the experts.

There are various grades of freezers and containment areas for radioactive and hazardous material in the 25,000 sq feet of warehouse space where the SRMs are stored. Depending on its use, each sample falls into one of two camps. Ensuring consistency in industry tests is what validation means. The flammability of fabrics and upholstery was tested with the standard cigarette, which is for $446 for two cartons of 100. According to Choquette, stray smoking materials are the leading cause of deaths by fire in the US.

You can get the Guardian's award-winning long reads every Saturday morning.

There are legal standards for flame- resistant fabrics. For those tests to be consistent across manufacturers, there needs to be a standardised cigarette. That is SRM 1196a. The cigarettes are made by ordinary manufacturers and are no different to regular cigarettes, but their homogeneity has been tested and verified by NIST. The agency provides the sameness as a service. The result can save lives.

Calibration is one of the major uses for SRMs. If you are a food manufacturer, you want to check the nutrition of your product. You can buy machines that test for certain compounds, but how do you know they are accurate? Each of them has been precisely quantified by more thorough and expensive means, which you can find on the NIST website. If your tests match NIST's, you can trust your machinery.

The samples are delicious. The food critic tasted peanut butter from NIST. He noted that it didn't offer the creamy flavours of most consumer brands, that it looked more like industrial paste than actual food, and that it was, all told, an average product. NIST must have been happy with the assessment.

The discipline of measurement was developed before it was possible to remove the bottom of a jar of peanut butter. In ancient Egypt, the first standardised units were deployed around 6,000 years ago, where the length of Pharoah's arm was used to measure out the dimensions of the river valley. In the Middle Ages, the task of regulating measurement to facilitate trade was both privilege and burden for rulers: a means of exercising power over their subjects, but also atrigger for unrest if neglected. In 18th-century France, there were said to be some 250,000 variant units in use, leading to the revolutionary demand for one king, one law, one weight and one measure.

A plaque showing the ‘standard metre’ measurement introduced during the French revolution in Paris, France

A plaque showing the ‘standard metre’ measurement introduced during the French revolution in Paris, France. Photograph: PjrTravel/Alamy

The creation of the metric system by French savants was the result of an abundance of measures. The metre was intended to embody political ideals, and was defined as one ten-millionth of the distance from the equator to the north pole. Its value and authority were derived from scientific calculation and were therefore accessible to all. Units of measurement were designed to create uniformity across time, space and culture, to enable control at a distance and to ensure trust between strangers. Since the time of the pyramids, they often span the entire globe.

International standards like those mandated by NIST and the International Organization for Standardization (ISO) are mostly invisible in our lives. Measurement is intruded by various stripes in education and the workplace. In school, we are exposed to the harsh lessons of quantification, where we are sorted by grade and rank and told that these are the measures by which our future success will be gauged.

Key performance indicators and OKRs are the tests that reappear after we leave school. The number of articles I wrote and the number of online page views they attracted were the main indicators of the value of my work in my early career as a journalist. We were taught to value quantity over quality. I had to unlearn many of the lessons that were taught by these metrics.

The principle that any human endeavor can be reduced to a set of statistics has become one of the most important principles of the 21st century. metric fixation is a concept that is pervasive in both the private sector and the state's less-quantifiable activities.

Muller writes that we live in the age of measured accountability, reward for measured performance, and belief in the virtues of publicising those metrics through transparency. He stresses that measurement is not a bad thing, but excessive measurement and inappropriate measurement will distort, distract and destroy what we claim to value.

The roots of metric fixation can be traced back to the 19th century. Management was emerging as a profession in the US rather than a skill learned from working in an industry. A drive to rationalise the work of managers was part of the transformation of industrial production itself. The American system of manufacturing was centred on the virtues of standardisation, precision and efficiency. The production of consumer goods used to be done by artisans. Manufacturing was turned into a series of repetitive tasks with the advent of machines that could stamp, cut and mold many different components. In the US in the 1850s, a British engineer noted that wherever machinery could be used to replace manual labour, it was universally and willingly used.

At the turn of the 20th century, the system was further augmented by two concepts: scientific management and mass production. Henry Ford's low-priced Model T helped create a prosperous middle class that defined itself by mass consumption. Ford claimed that his assembly lines, which kept workers static while material moved through their stations on conveyor belts, had been inspired by an aide's visit to a Chicago slaughterhouse. The aide observed that there was a row of butchers taking apart pig carcasses, with each individual focusing on a single repetitive task.

Taylorism is a set of working practices advocated by an efficiency-obsessed engineer, who pioneered the scientific management movement. Taylor and his followers observed labourers and broke down the flow of their work into parts that could be standardized. Taylor said the aim was to develop a science to replace the old rule-of-thumb knowledge. The labourers who carried out the work had to transfer their knowledge and power to the managers who oversaw it.

These kinds of controls are also used in institutions such as prisons, armies and schools. French philosopher Foucault wrote about a world in which compliance is strictly defined. The prisoners are given uniforms and numbers, told when and where to eat and sleep, and live in the uncertainty that they are being watched by unseen guards. Foucault said that they internalise this authority and police their own behavior. The aim is not to punish less, but to punish better. The control is dependent on measurement and standardisation.

Gary Wolf wrote about the age of quantification in the New York Times. He wrote that using data to make decisions is the norm in nearly all spheres of life. Corporate executives are facing down hostile shareholders. Wolf said that business, politics and science are all steered by the wisdom of what can be measured. The cozy confines of personal life has resisted the lure of quantification. Wolf said that would soon change.

Standard reference materials from the US National Institute of Standards and Technology

Standard reference materials from the US National Institute of Standards and Technology. Photograph: Mark Esser/NIST

Humans have historically unprecedented powers of self-measurement thanks to new technology, such as the ability to digitise information, the ubiquity of smartphones and the proliferation of cheap sensors. Santorio Santorio built giant scales in order to better understand his metabolism, and he was able to sit on them. Santorio would measure his weight before and after eating. We are rewarded with a lot of comparable information. We can use gadgets and apps to track our health. We have become a beacon of unseen measurement, emitting quantified data as radiation is released.

The potential of this information is huge for Wolf. The movement's origins can be traced back to the 1970s, when enthusiasts cobbled together clunky ancestors of today's Wearable tech. The idea came to the attention of the public after Wolf and Kelly founded a non-profit to spread their ideas.

Descriptions of the quantified self can be used to create caricatures of Gradgrinds who are obsessed with the optimised life while their souls wither on the vine. It's true that many proponents of QS don't do anything to change the image. They claim to shave minutes off their day through rigorous self-surveillance or discovering through sophisticated analyses. The tech critic says that the quantified self is Taylorism within the modern quest for uniqueness and exceptionalism.

Proponents of the movement defend it as a response to the generalities of official knowledge. If the world has become one-size-fits-all rules that do not fit the individual, why not create a set of numbers that better capture the truth? They cite anecdotes of self-trackers whose chronic ailments resisted the cures of mainstream medicine but yielded to their pattern- finding prowess. After months and years of self- tracking, these individuals discover a hidden mechanism in their life, some food or habit that causes them to suffer, and make the changes necessary to live happily ever after. The quantified self seems to be an attempt to return to the personal dimensions of measurement and resist the use of statistics to fit our lives.

In his 2010 article, Wolf said that a century ago we used Freudians to uncover the mysteries of the self, using language and a culture of literary humanism. This is not the world we live in today, so why use outdated methodologies? He never answers the question of how the precision of numbers is supposed to match the complexity of language as a tool for self- exploration. One thinks this is a feature rather than a bug. By limiting the scope of self- investigation to what can be measured, practitioners are assured of finding answers. The therapist's couch has to be returned week after week to deal with the inefficient complexity of language.

I often end up thinking about a single figure: 10,000 steps, because I think about what measurement means in today's society, how it's used and misused, and how we internalise its logic. It is often cited as an ideal daily target for activity, and is built into many tracking apps and fitness programmes. We are told to walk 10,000 steps a day.

You would be forgiven for thinking that this number was the result of scientific inquiry, as it is presented with such authority and ubiquity. Yamasa Clock is a Japanese company that has a marketing campaign. The company needed a name for their new product in 1965, they were promoting a digital pedometer. They settled on manpo-kei. Why was this number chosen? The first character in the product's Japanese name looks like a figure striding forward with confidence. It seems that there was no science to justify 10,000 steps.

An ant carrying a 1mm square microchip in its mandibles at Huddersfield University Precision Technology centre

An ant carrying a 1mm square microchip in its mandibles at Huddersfield University Precision Technology centre. Photograph: Reuters

The 10,000 steps are useful if they are an illusion. 10,000 steps is too low for children and too high for many older adults according to research into how many steps a day we should pursue. It's clear that increased activity is good for us, and that people who do pursue a daily target of 10,000 steps have less signs of depression, stress and anxiety. If you want to reach people, you need to speak in a language they understand.

Hartmut Rosa, a German sociologist, suggests that the 21st-century desire to structure our lives through empirical observation renders our interests and ambitions as a series of challenges to overcome.

The result of centuries of cultural, economic and scientific development, but has been radicalised in recent years by digitalisation and capitalism. As we measure more and more, we encounter the limits of this practice and wrestle with its effects on our lives.

Patient lying down on CT scanner

The search for a cure for a mystery illness.

My interest in the history of measurement began as a simple question about the origin of some units. Why is a kilogram an inch? If measurement is the mode by which we interact with the world, it makes sense to ask where these systems come from, and if there is any logic to them.

The answer I have found is that there isn't any. As with the 10,000 steps, it is as much the result of accident and happenstance as careful deliberation. The unit of length was defined hundreds of years ago by measuring the planet we live on. They made mistakes in their calculations, and so the metre is 0.2mm short: a minute discrepancy that has been perpetuated ever since. It is the way it is because we say so. Measures are both meaningful and arbitrary, and iron guides in our lives that can be changed if we want them to be. If they don't work, if they don't measure up, then they can be replaced.

The edited extract from Beyond Measure: The Hidden History of Measurement is available at guardianbookshop.com.

You can follow the Long Read on social media, listen to our podcasts, and sign up for the weekly email here.