You Get What You Measure
Thoughts on a general principle I've noticed, relating to the role an environment plays in shaping things within it, including teams, projects, code, and psychology.
I’ve noticed an unfortunate development habit of mine. I’m not the only one, so chances are if you’ve avoided this habit yourself, you’ve at least seen it elsewhere. This habit is over-secrecy in a project’s development. “Nobody can see it until it’s perfect!”, or, “I don’t want to spoil the surprise!”, or, “I don’t want early impressions to shape the project!”
Now, to be fair, these statements are not entirely ridiculous. There are marketing reasons to avoid releasing a project too early (or pushing a rough alpha version out the door), and marketing is important. Sometimes, early impressions can take a project in bad directions, and ideas need to come to fruition behind closed doors. Sometimes, you just don’t have the time to deal with bug reports, and you don’t want to release something that people begin depending on, just so your free time is evaporated.
But there’s a flipside to all of that. The above utterances can also be mere rationalizations, rather than careful and strategic plans. That raises the question—if statements like the above are used for rationalization, what are they rationalizing for? What is the source of over-secrecy?
I don’t fully know, and obviously I can’t make any generalized claims, but in myself I’d claim the source is a sort of anxiety.
When I work on a project—especially in my spare time—it’s because I deeply care about it. There are only so many spare hours in a day, and if I spend them on a project, it’s because I see the project’s fruition as worthwhile. Particularly, I see it as more worthwhile than something else I’d spend those same hours on.
But there is, of course, an uncomfortable danger with such a passion project. It might be the case that nobody else agrees about the project. Perhaps the project doesn’t tangibly matter at all to anyone, and it might just be a big waste of time.
And let’s say that your project isn’t just a spare-time passion project, and upon it rests your livelihood. You might have a wife and kids dependent upon your project’s success. Failure, in that case, is not just disappointing, but destabilizing, and perhaps personally catastrophic. And, furthermore, in that case, success with others is not just an irrelevancy (like it may be with a personal passion project), but instead is a requirement.
Modern culture—especially in the technical world, and very especially in the game development world—doesn’t help navigate this issue. Instead, it seems that everyone has found it more important to throw pity-parties and promote reality-denial. This phenomena can be observed if you navigate over to Twitter (Heaven help you). You might find developers posting nauseating tweets about such anxiety, in a transparent and embarrassing effort to externally-validate their own beliefs about their work’s value. Such tweets often receive hundreds of likes and replies from other developers, encouraging the poster that what they do is worthwhile, valuable to others, and meaningful—almost to the point of fellatio.
While that whole game may be gratifying in the moment, all it achieves is the avoidance of an uncomfortable truth—ignorance, after all, is bliss. The uncomfortable truth is that some ideas are stupid. In fact, some is an understatement—most ideas are stupid. Most thoughts are stupid. Most projects are stupid.
The production of value—ordering in the face of Time, which drags everything toward chaos through entropy—is difficult and expensive. It requires rigorously walking along a narrow path—and chances are, your first best guess at that narrow path will be way off.
In a world of idiotic monetary policy and politically-expedient economics, it may be the case that the narrow path will not be actively selected by markets. Cheap money may flood your stupid company, and fund your stupid project, and keep your stupid job afloat.
But, at some point, the party’s over. Reality will be back from lunch, and tear down everything you worked so hard to build.
So what’s the solution? In my mind, it has to do with directly engaging with this uncomfortable reality. And, in doing so, ideas and projects will be shaped by reality, rather than divorced from it. It begins with simply asking the question—“are you measuring your project against markets—against the perceptions and values of others—or aren’t you?”
And in that lies a general principle I’ve noticed elsewhere: you get what you measure.
If you never measure memory usage in your program, it’s far more likely that your program will have wasteful memory usage. If you never measure a frame’s performance in your game, it’s far more likely that your game will drop frames. If you never measure the size of your codebase, it’s far more likely that it grows indefinitely. And if you never measure your idea against the values and needs of others, it’s far more likely that—to everyone but you—it’s a big waste of time. And if it’s a big waste of time for everyone but you, then your project does not deserve—for instance—anyone else’s money.
In other words, a product is necessarily shaped (at least partly) by its environment. Shaping that environment is the game. That environment—if the producer chooses—can include scrutiny, beta-testing by those who would ostensibly use the product, and exposure to reality. It can also not include any of that.
I’ve noticed that this even arises in shaping one’s own habits and psychology. Let’s say you’re hoping to work out and obtain a nice summer bod. Are you exposing yourself to the harsh reality of the gym, or aren’t you? There isn’t some point at which the “gym bod” switch will be turned on, and the calories from all the burgers you’ve eaten will magically disappear—speaking from experience here! Instead, the right path—it seems to me—is to actively contend with the reality that you hope to conquer. And, as I’ve learned more, I’ve seen that same principle arise not just in personal habits, but in projects as well.
Starting my writing on this site was partly a first step in exposing myself more often to this reality. Not every sentence needs to be crafted to perfection, and not every word must perfectly encapsulate my thinking—the important thing is simply that I’m actively writing, thinking, and engaging with an audience, even if it’s imperfect, sloppy, and uncomfortable.
I hope to use the same strategy in my projects, and I hope that you will too—from what I can tell, it’s best to expose yourself to the realities with which you contend earlier, rather than later.
If you enjoyed this post, please consider subscribing. Thanks for reading.
-Ryan