Tag Archives: AI

Opportunity Costs in Civilization 6

One of the most important concepts I learned in economics is the idea of opportunity costs.  Every action we take has a cost, not just the cost of the action itself, but the cost of *now not being able to do something else* with either the time or money or both that we just spent.  

A simple example: a company only has 100$ to invest in a machine.  If they buy the machine that makes blue widgets, they can’t also buy the machine that makes red widgets.  Thus, buying the blue widget machine doesn’t *just* cost 100$, it also has the *opportunity cost* of not buying the red widget machine.

There are also opportunity costs with time, if you decide to go to Europe for your holiday vacation, you can’t also go to South America at the same time.  So the cost of going to Europe isn’t just the cost of the time and the tickets, it’s also the opportunity cost of not going to South America (or anywhere else) as well.

As an aside, this is why for some people it can make sense to NOT go to college EVEN IF college were totally free.  The cost of going to college includes the *opportunity cost* of not having a full-time job (if you’re a full-time student).  

A comedian once made a joke that, after graduating college he couldn’t find any work “because the dropouts already had the jobs.”  A funny joke, but it demonstrates a point:  You spend 4 years getting a degree, but if that degree doesn’t measurably increase your employment prospects, you could have been better off spending those 4 years getting work experience at a full-time job.  You could not only get the money that a full time job gives you, but the experience itself would increase your employability and ability to get better jobs.

So the education and degree you’re seeking needs to increase your employability *more* than just doing 4 years of work.  If not, then it’s a net loss *even if your education was free* because you had the *opportunity cost* of not getting those 4 years of work experience.

But I didn’t want to blog about college, I wanted to blog about Civilization again.

The non-gamers in my audience may be tired of my gaming blogging, but I’ve spent a lot of this holiday season playing Civ IV and Civ VI with friends, so I’ve been thinking about this.

I complain that in Civ VI, some of the leaders seem to have traits that are utterly worthless, they don’t feel like they improve your Civ’s abilities any more than having a vanilla Civ with *no* traits.  Eleanor of Aquitaine is one of these, her ability to culture-flip cities feels very underpowered and completely useless, and doesn’t make her any more powerful than a Civ that doesn’t have any abilities at all.

My friend shoots back at this by saying that if you put a lot of resources into it, you can set up a situation in which you culture-flip whole continents in an instant.  And yes, this is theoretically possible.  Does that mean Eleanor is “very powerful in the right circumstances?”  No.  Because of *opportunity costs*.  

See, the cost of putting all your resources into Eleanor’s culture-flipping ability is that *you can’t put those resources into other things*.  You can’t research technologies or build military units if you are instead spending your entire GDP on culture buildings.  Culture isn’t free, and it doesn’t just cost what it costs to produce it, it has the *opportunity cost* of not doing anything else with that money and production.

So in any situation where Eleanor can “culture flip a continent” by spending an absurd amount of resources on culture, any other Civ could just use those resources to win the game with military, or science, or even diplomacy.  

Eleanor’s ability is useless not because you *can’t* use it to do things, but because the amount you have to spend to make her ability not-useless could instead be better used to win the game in *any other way at all*.  Her ability has an *opportunity cost* in that if you try to use it to its fullest, you are by definition not using those resources on better strategies that will win you the game more easily.

And that’s what I feel about a lot of the Civ VI leaders.   Some  leaders have abilities so minor they don’t feel impactful.  Some have abilities that completely change the nature of the game.  And some like Eleanor have these abilities that are actually traps, because the opportunity cost of trying to use their ability to its fullest makes you worse off than if you’d ignored their ability and played the game normally.

People didn’t like Civ IV’s leader system because every leader drew from a limited pot of abilities.  Gilgamesh is Creative/Protective, while Catherine the Great is Creative/Imperialistic.  From my perspective, that’s unique, *no one but Gilgamesh has that specific combination of traits*.  From other people’s perspective though “they’re both creative so they’re too similar to be cool.”  

These people who say this seem to really be drawn to Civ VI because every leader has *completely unique abilities* not seen anywhere else.  But from my perspective “many of those abilities are worse to use than just ignoring the ability and playing the game normally.”  Leaning into your “special ability” can have an opportunity cost, and no one but me seems to recognize this.

So in the future, please think about opportunity costs, both for college and for your video games.  Making a nation have a super special ability isn’t actually cool if leaning into that ability makes you worse off than if you’d ignored it and played the game normally.   Opportunity costs are real, even if not everyone understands them.

Why does the Civ VI AI feel so incompetent?  Part 2: Examining how it was made. 

When I was last writing about the Civilization series, I was complaining about how the AIs in Civ VI feel much stupider than the AIs in Civ IV.  I encourage you to read that post, because this one is a direct follow-on. 

In brief, there were a lot of ways AIs could threaten you in Civ IV.  They could send their military to attack you, they could use their production to build wonders before you could, they could use their culture to steal the hearts and minds of your people, making your own cities flip to their side in the process. 

In theory, all these methods still exist in Civ VI, but the AIs are very incompetent at executing them.  None of the Civ VI AIs can threaten you with their military, wonder-building, or culture the way AIs could in Civ IV.  And I think the reason is one of Civ VI’s biggest selling points: unstacking the map. 

See, Civ IV militaries came in “stacks,” where 20 to 100 different units could all sit on one tile together and attack wherever they wanted.  Defeating these stacks meant you had to have a stack of units all your own, and some people complained that this made warfare just a numbers game without any tactics.   

I think those complainers were dead wrong, but regardless Civ V was the first game to “unstack” the military, forcing 20 units to all sit on 20 different tiles instead of stacking together to attack you.  Civ VI continues this trend, and coincidentally Civ V and Civ VI have the same problem in which warlike AIs are incredibly bad at war.   

But while Civ V was the first to unstack the units, Civ VI went further in “unstacking the map.”  In Civ IV and Civ V, your city could have any number of buildings in it that you wanted, built at any time.  So you could build a Forge for +25% production, a Library for +25% Research, a Market for +25% gold.  The question then becomes, which buildings should you build, and in what order? 

If you already know you’re going to build all 3, then you should build the Forge first.  It’s bonus of +25% production will speed up how fast you build the Library and the Market after its finished.  But maybe you are in a severe economic crunch, and you just NEED GOLD NOW.  In that case, maybe build the Market first, and then maybe skip on the library and forge so you city can focus on producing wealth and not spend its scarce resources building infrastructure. 

Or maybe your city produces a lot of science, but almost no production or gold.  Is it worth building the Market and Forge in that case?  Maybe you should *just* build the library and be done with it. 

These are all simple ideas, and you can easily see the AI thinking of the game like an excel spreadsheet and just trying to maximize its values at the end.  The AI sees its running out of gold, it builds markets in response.  It sees a city with high science, it builds a library there.  It sees a city with good everything, it builds Forge first, then Library and Market after.   

The AI in Civ IV is really just deciding what order to build things in, and when.  Its goals can be thought of as simple profit-maximizer functions, and it can be coded in the same way.  The programmers who actually built this AI then had a straightforward job in front of them: adjust how the AI weights each one of its goals until you find a system that makes the AI play reasonably well.   

You can downweight Libraries if your playtesting reveals that the AI is going bankrupt by building those instead of Markets.  You can upweight Forges if the AI is foregoing them to focus only on science and gold.   

Up- and downweighting just chances where the AI puts its build orders in the city queue, and while there’s a lot more to build in Civ IV than just Forges, Markets, and Libraries, the build queue itself is quite simple to grasp. It’s easy to visualize the build queue by just writing it out, and it makes sense that you could try to use it to improve the AI’s intelligence while sitting in front of your computer trying to program the game. 

But with unstacking the cities, there’s no longer just a build queue.  It isn’t just about *when* you build things, but also *where*.  Even explaining this system through text or a spreadsheet is difficult, and you’ll see what I mean.  And I believe that this difficulty made it harder to program a “good” AI.  Because instead of a simple build-queue that can be thought of as a profit-maximizing function, you’re suddenly solving a *graphical* problem instead. 

So here’s an example of unstacking the cities.  In Civ VI you’ll still build the equivalents of Forges, Libraries, and Markets.  Only now Forges give bonus production for being near mines and quarries, Libraries give bonus science for being next to mountains, and Markets give bonus gold for being on a river.  Each building can’t stack on top of another building, so you can’t place a Library where you already put your Forge. 

Let’s say we have a city that’s just south of a river, near a mountain range immediately to its west, and has some mines on the opposite side of the river near the mountains (so northwest from the city).   

Well if you put down the Forge near the mines (so across the river), you invalidate using that spot for your Market.  If you then put your Market down on this side of the river, you no longer have any room to place your Library near those mountains.   

Is this easy to visualize in your head?  Do you think it’d be easy to try to program an AI to maximize its bonuses in this system?  I don’t think so, and I think this might be a fundamental problem with the Civ VI AI: it can’t think in terms about graphical problems, it only seems to think about functional problems.  And I think that’s because the programmers programming it also had trouble solving the graphical problems because translating a graphic problem into code isn’t something most people are used to. 

And I think this is the case because Civ IV’s AI *also* had a fundamental difficulty of solving graphical problems.  Most of Civ IV’s gameplay was like those profit-maximizing functions I talked about above: what do you build or research and in what order.  But *where* to place your cities is a more graphical problem, and it was one problem the AI was unusually bad at. 

Here’s an example of Civ IV’s graphical problem: where to settle your city?  You’re playing as Egypt, and Egypt’s special unit is the War Chariot, which requires Horses.  You see there is a Horse resource a ways east of some Wheat, and to the northeast of the Horse resource is Fish.  Wheat and Fish both provide a lot of food, and food is the most important resource of all in Civ IV (as it is in real history).   

So you want to maximize your food AND get the Horses, but how can you get all 3 of these together in a single city?  Settling closer to the Wheat gives you a city that’s off the coast and can’t get to the Fish.  Settling closer to the Horses means you have to wait until borders expand to get either the Fish OR the Wheat.  Settling closer to the Fish means you have to wait until borders expand to get the Horses. 

Again, this problem of where to settle cities is probably very hard to visualize.  And while a skilled player will quickly learn to solve this problem, it seems the Civ IV programmers couldn’t get the AI to solve it.  The AIs will regularly settle cities in terrible spots where they can’t get any resources or can’t get as many resources as they *should* get. 

Again, I think the graphical problems of Civ IV were harder for programmers to visualize and program for than the profit-maximizing problems, and that’s why Civ IV is worse at the game’s graphical problems, like settling cities, than it is at the profit-maximizing problems, like when to build its Forge, Library, and Market. 

I think as the games’ problems have become more and more graphical, the programmers who are used to coding functions haven’t been able to keep up.  And that leads to a severe disconnect between how the programmers want the AI to behave an how it actually does. 

I think my final piece of evidence for this is the 2021 patch for Civ VI/ 

In the Civ VI 2021 patch, the Devs tried their damndest to finally make the AI smarter.  They did this by making the AI overemphasize science to a ridiculous degree, hoping that if the AI could have a tech lead against the player than all its other problems would fall into place. 

This didn’t work because the AI was still building Libraries in terrible places, it was just now building more of them and invalidating good locations for Markets, Forges, and everything else.  The huge overemphasis on libraries created AIs that would blow through the early-game research before stalling out due to a lack of money and production to build buildings in the later eras.  The AIs still couldn’t win technology victories, or even beat the player in technology, but when you captured their cities you’d find tons of libraries built in spots that should have had a Market or Forge. 

It sounds like the Devs faced exactly the type of graphic problem I’ve described, but tried to solve it with a profit-maximizing solution.  The AI can’t research well?  It’s very hard to teach them *where* to place libraries, so just tell them to build *more* of them.   

I don’t know what can be done to fix this, maybe force the devs to have a copy of the game running on a second monitor as they program, or introduce some training about how to translate a graphical problem into a code-able solution.  But I think this difficulty of solving graphical problems is why the Civ VI AI is so much dumber than the Civ IV AI, all the biggest problems in Civ VI are graphical. 

Why does Civ VI AI feel so incompetent? Part 1: Examining the AI in its natural habitat.

I’ve talked before about Civ IV and Civ VI, two great entries in the much-beloved Civilization series of video games.  I’ve talked before about how the Civ IV AIs feel like they’re a lot “better” at playing Civ IV than the Civ VI AIs are at playing Civ VI.   

Civ IV AIs aren’t smart, they make dumb mistakes, but they are competent and threatening both to the player and each other.  Civ VI AIs are incompetent and unthreatening, they simply don’t know *how to win* even if they are OK at surviving and acting as a speed bump.   

Let me get deeper into how the AIs could “threaten” you in Civ IV.  I don’t know if “threaten” is the right word, but we’ll go with that.

The most obvious way an AI can threaten your empire is they could go to war with you.  A warlike leader like Alexander the Great could just build military units nonstop and attack you.   

But that’s not the only thing AI leaders could do, they could also build wonders that you wanted to build.  In Civilization, there are these unique buildings called “Wonders” which can only be built once in the entire world.  Think of the Pyramids, the Great Wall of China, or the Statue of Liberty.  Every Civ in the game gets a chance to build these wonders, but whoever built it *first* gets the wonder and all the benefits of that wonder, while everyone else gets a crummy consolation prize.   

These wonders gave powerful benefits, The Great Wall for instance would completely stop barbarians from entering your territory.  You might really want that wonder to protect yourself.  So let’s say you start building the Great Wall, but another Civ across the map finishes their own Great Wall mere moments before you were about to finish yours. They get the Great Wall with all its benefits, you get no Great Wall and a crummy consolation prize, AND you invested a lot of production into that wonder that you could have spent on something else.   

An industrious leader like Rameses had the perfect traits to outbuild you in wonders.  So if he was on your map, you had to really plan and strategize how you were going to beat him to get those wonders for yourself. 

AI leaders could also threaten you culturally.  Civ IV had an elegant way of using culture, in that culture decided what parts of the map your empire controlled, and thus what parts you could extract resources from. 

Consider two AI leaders, Julius Caesar of Rome and Louis XIV of France.  They settled their cities right next to each other, and between the Roman and French cities lies a gold resource.  Gold is incredibly valuable, not only does it give you money in Civ IV, but it also counts as a luxury resource that makes every city in your empire happier.  Controlling that gold is key to building a wealthy and powerful nation. 

Caesar is a warlike leader though, he’ll be building non-stop military units in his city.  Louis is a more cultural leader, he’ll build libraries, theatres, that kind of stuff.  These cultural buildings put cultural pressure on the people living between the two Empires, those people will start to adopt more and more French fashion, language, taste, and more and more of them will call themselves French and not Roman.  Because they call themselves French, they’ll work for the French Civ and not the Roman Civ, thereby giving France control over the gold.   

So through the power of culture, France will control the gold and Caesar won’t.  And since Caesar never builds anything but military, he won’t put out the cultural pressure needed to counteract the French culture pressure.  Eventually, French culture might be so strong that the people Rome might get converted into being French, they’ll want to join the French Civ rather than remain Roman because French culture is so dominant.  It will take a lot of military police for Caesar to keep the his people in line, and even then they may revolt out from under him. 

Which is why Caesar usually declares war on cultural Civs that settle next to him. 

But anyway, this cultural pressure is *yet another way* for Civ IV AIs to threaten you.  It’s not enough that you settled powerful cities in good spots, you also have to keep your citizens happy and build then some cultural buildings.  If you don’t, an AI like Louis can settle on your border and convert them all out from under you. 

All these three things: wonders, culture, military, are ways that the AI in Civ IV could affect and threaten you.  You weren’t just playing a game all on your own, Civ IV had AIs on the board who would mess up your every plan at the slightest opportunity, with their military, their wonder-building, and their culture.

Military, wonder-building, and culture all still exist in Civ VI, but the AI can’t really use them to affect a human player. 

Let’s go back to our war example with Alexander.  In Civ IV, Alexander’s main mode was to declare war by marching a force across his enemy’s border that was twice as large as their entire army.  All of these military units could move and attack together, so 20 units could move right next to an enemy city and attack the single archer that was guarding it.  With such a large force, Alexander was basically guaranteed to conquer several cities in his path before his enemies could mount a counter-attack. 

In Civ VI, Alexander is still a warlike AI who likes building units.  But Civ VI has 1-unit-per-tile (abbreviated 1UPT), so all those 20 units are spread out across a very wide area, and they get in each other’s way when they try to move.  If the unit at the front is attacking a city, every unit behind it is blocked from moving forward, and they have to all awkwardly shuffle around to find their own vectors of attack.   

Rather than overwhelming his enemies 20-to-1 like Civ IV Alexander, Civ VI Alexander has his units attacking piecemeal, one-at-a-time, because he can’t get them all into the same place at the same time.  You’d think his 20-to-1 advantage would still ensure he eventually wins, but Civ VI has so many defender advantages, and so many ways to heal units, that his attacks end up petering out in most cases. 

Civ IV Alexander would conquer Civ after Civ until he faced someone with enough of a technology edge to counter his numerical edge.  Civ VI Alexander rarely even takes border cities, and almost never conquers entire Civs.  

How about that wonder example from earlier?  In Civ IV, wonders require a certain technology in order to unlock them, and can be built faster if you have a special resource like Marble or Stone.   

Rameses’s MO was therefore to bee-line for technologies that let him build wonders, try to grab any Marble or Stone he could find, and build his wonders in whatever city he had the most production in.  That was usually enough to net him most of the wonders, and you’d have to bee-line those technologies yourself and outpace him in raw production if you wanted to get any. 

In Civ VI, Rameses is still in the game, still obsessed with building wonders, but he is now MUCH worse at it.  The thing is that wonders now have a lot of specific requirements in order to build them.  You can’t just build the Colosseum in whatever city you choose, you can ONLY build it on FLAT land NEXT TO an entertainment district that ALSO has an Arena in it.   

AIs are really bad at building districts, they always seem to have way fewer than they should and often those districts are placed nonsensically.  The AI also doesn’t plan ahead with their districts, they will happily place their entertainment district in a spot surrounded by hills and mountains so that they have no flat land to build the Colosseum.   And even if the AI builds an entertainment district next to flat land, there’s no guarantee they’ll eventually build the Arena in that district that is required to build the Colosseum.

Many of the wonders in the game have strict requirements like this, so aside from the few wonders with very loose requirements, Civ VI Rameses is just structurally incapable of building wonders.  The Colosseum unlocks in the classical age, and it is a very powerful building, you’d think Rameses would want to build it.  But I can still lazily pick it up in the industrial age *centuries later* because AIs like Rameses will simply *never satisfy the requirements to build it*.   

In Civ VI I don’t need to bee-line technologies, or have super high production.  I just need to be mindful of the wonder’s requirements, and I can build almost any of them at my leisure. 

Finally let’s talk about Culture.  Louis XIV isn’t in Civ VI, but Eleanor of Aquitaine is.  When Eleanor leads France, they should be a cultural powerhouse just like under Louis, right?  Not really. 

See, there’s no cultural struggle in Civ VI like there was in Civ IV.  France can’t settle next to you and steal your gold tile away with culture.  Instead Civ VI works on a first-come-first-served basis, if you get the gold tile first, it’s yours forever barring some unbelievably rare circumstances.   

And in fact, the map is so open in Civ VI that you’ll rarely see a Civ next to you at all.  Civ IV was a mad dash to settle the map before anyone else.  If you were slow, all the good resources (like the gold) would already be taken before you could get to them, leaving you with no resources of your own.  At that point, the only way to get your resources in Civ IV would be either war (like Alexander) or culture (like Louis). 

But Civ VI has more resources than it knows what to do with, I often stop settling cities not because there’s no more room but because I no longer want to have another city to manage.  If someone does take a gold resource, well that sucks, but I can probably find another gold resource somewhere close by.

So my cities very rarely are right on the border with another Civ’s, meaning that even if she wanted to, Eleanor couldn’t steal my tiles like Louis could.   

And besides, the AI can’t build culture any more than it can build wonders.  As I said, the AI doesn’t build enough districts, and they certainly don’t produce enough culture from those districts to matter.  You can’t culture flip tiles, but you can still culture-flip cities, and Eleanor’s special ability in Civ VI is supposed to let her better at this than anyone else.  She’s so good, Civs can’t even use their military to keep cities in check the way Caesar could in the Civ IV example.   

But when I’ve played against AI Eleanor, she never has any success with culture-flipping.  She doesn’t produce enough culture districts, she doesn’t produce enough culture, and her cities are usually so far away from mine that her culture-flipping couldn’t happen even if I ignored culture entirely and went for a pure military victory. 

I wanted to make this point about how the AIs in Civ VI don’t seem to play their game as well as in Civ IV.  I’ve harped on this point a lot over the years, but I wanted to bring in some specifics because in my next post, I’d like to tackle the *why*.  I don’t know for sure, but I think that a very important change in the Civ series made coding AIs for it a MUCH bigger headache, and that has led to stupider AIs overall. 

Stay tuned… 

What does it mean to think? 

It may surprise you to know, but I was once a philosopher.  To be more accurate, I was once a clueless college student who thought “philosophy” would be a good major.  I eventually switched to a science major, but not before I took more philosophy classes than most folks ever intend to. 

A concept that was boring back then, but relavent now, is that of the “Chinese Room.”  John Searle devised this thought experiment to prove that machines cannot actually think, even if they pass Turing Tests.  The idea goes something like this: 

Say we produce a computer program which takes in Chinese Language inputs and returns Chinese Language outputs, outputs which any speaker of Chinese can read and understand.  These outputs would be logical responses to whatever inputs are given, such that the answers would pass a Turing Test if given in Chinese.  Through these inputs and outputs, this computer can hold a conversation entirely in Chinese, and we might describe it as being “fluent” in Chinese, or even say it can “think” in Chinese. 

But a computer program is fundamentally a series of mathematical operations, “ones and zeros” as we say.  The Chinese characters which are taken in will be converted to binary numbers, and mathamatical operations will be performed on those numbers to create an output in binary numbers, which more operations will then turn from binary numbers back into Chinese characters.   

The math and conversions done by the computer must be finite in scope, because no program can be infinite.  So in theory all that math and conversions can themselves be written down as rules and functions in several (very long) books, such that any person can follow along and perform the operations themselves.  So a person could use the rules and function in these books to: 1.) take in a series of Chinese characters, 2.) convert the Chinese to binary, 3.) perform mathamatical operations to create a binary output, and 4.) convert that binary output back into Chinese. 

Now comes the “Chinese Room” experiment.  Take John Searle and place him in a room with all these books described above. John sits in this room and recieves prompts in Chinese.  He follows the rules of the books and produces an output in Chinese.  John doesn’t know Chinese himself, but he fools any speaker/reader into believing he does.  The question is: is this truly a demenstration of “intelligence” in Chinese?  John says no. 

It should be restated  that the original computer program could pass a Turing Test in Chinese, so it stands to reason that John can also pass such a test using the Chinese Room.  But John himself doesn’t know Chinese, so it’s ridiculous to say (says John) that passing this Turing Test demonstrates “intelligence.”   

One natural response is to say that “the room as a whole” knows Chinese, but John pushed back against this.  The Chinese Room only has instructions in it, it cannot take action on its own, therefore it cannot be said to “know” anything.  John doesn’t know Chinese, and only follows written instructions, the room doesn’t know Chinese, in fact it doesn’t “know” anything.  Two things which don’t know Chinese cannot add up to one thing that does, right? 

But here is where John and I differ, because while I’m certainly not the first one to argue so, I would say that the real answer to the Chinese Room problem is either that “yes, the room does know Chinese” or “it is impossible to define what “knowing” even is.” 

Let’s take John out of his Chinese Room and put him into a brain.  Let’s shrink him down to the size of a neuron, and place him in a new room hooked up to many other neurons.  John now receives chemical signals delivered from the neurons behind him.  His new room has a new set of books which tell him what mathematical operations to perform based on those signals.  And he uses that math to create new signals which he sends on to the neurons in front of him.  In this way he can act like a neuron in the dense neural network that is the brain. 

Now let’s say that our shrunken down John-neuron is actually in my brain, and he’s replaced one of my neurons.  I actually do speak Chinese.  And if John can process chemical signals as fast as a neuron can, I would be able to speak Chinese just as well as I can.  Certainly we’d still say that John doesn’t speak Chinese, and it’s hard to argue that the room as a whole speaks Chinese (it’s just  replacing a neuron after all).  But I definitely speak Chinese, and I like to think I’m intelligent.  So where then, does this intelligence come from? 

In fact every single neuron in my brain could be replaced with a John-neuron, each one of which is now a room full of mathematical rules and functions, each one of which takes in a signal, does math, and gives an input to the neurons further down the line.  And if al these John-neurons can act as fast as my neurons, they could all do the job of my brain, which contains all of my knowledge and intelligence, even though John himself (and his many rooms) know nothing about me.   

Or instead each one of my neurons could be examined in detail and turned into a mathematical operation.  “If you recieve these specific impulses, give this output.”  A neuron can only take finitely many actions, and all the actions of a neuron can be defined purely mathematically (if we believe in realism).   

Thus every single neuron of my brain could be represented mathematically, their actions forming a complete mathematical function, and yet again all these mathematical operations and functions could be written down on books to be placed in a room for John to sit in.  Sitting in that room, John would be able to take in any input and respond to it just as I would, and that includes taking in Chinese inputs and responding in Chinese.  

You may notice that I’m not really disproving John’s original premise of the Chinese Room, instead I’m just trying to point out an absurdity of it.  It is difficult to even say where knowledge begins in the first place.   

John asserts that the Chinese room is just books with instructions, it cannot be said to “know” anything.  And so if John doesn’t know Chinese, and the Room doesn’t know Chinese, then you cannot say that John-plus-the-Room knows Chinese either, where does this knowledge come from? 

But in the same sense none of my neurons “knows” anything, they are simply chemical instructions that respond to chemical inputs and create chemical outputs.  Yet surely I can be said to “know” something?  At the very least (as Decarte once said) can’t I Know that I Am? 

And replacing any neuron with a little machine doing a neuron’s job doesn’t change anything, the neural net of my brain still works so long as the neuron (from the outside) is fundementally indistinguishable from a “real” neuron, just as John’s Chinese Room (from the outside) is fundementally indistinguishable from a “real” knower of Chinese. 

So how do many things that don’t know anything sum up to something that does?  John’s Chinese Room  is really just asking this very question.  John doesn’t have an answer to this question, and neither do I.  But because John can’t answer the question, he decides that the answer is “it doesn’t,” and I don’t agree with that.   

When I first heard about the Chinese room my answer was that “obviously John *can’t* fool people into thinking he knows Chinese, if he has to do all that math and calculations to produce an output, then any speaker will realize that he isn’t answering fast enough to actually be fluent.”  My teacher responded that we should assume John can do the math and stuff arbitrarily fast.  But that answer really just brings me back to my little idea about neurons from above, if John can do stuff arbitrarily fast, then he could also take on the job of any neuron using a set of rules just as he could take on the job of a Chinese-knower. 

And so really the question just comes back to “where does knowledge begin.”  It’s an interesting question to raise, but raising the question doesn’t provide an answer.  John tries at a proof-by-contradiction by saying that the Room and John don’t know Chinese individually, so you cannot say that together they know Chinese.  I respond by saying that none of my individual neurons know Chinese, yet taken together they (meaning “I”) do indeed know Chinese.  I don’t agree that he’s created an actual contradiction here, so I don’t agree with his conclusion. 

I don’t know where knowledge comes from, but I disagree with John that his Chinese Room thought experiment disproves the idea that “knowledge” underlies the Turing Test. Maybe John is right and the Turing Test isn’t useful, but he needs more than the Chinese Room to prove that.

Ultimately this post has been a huge waste of time, like any good philosophy.  But I think wasting time is sometimes important and I hope you’d had as much fun reading this as I had writing it.  Until next time. 

The need for data, the need for good data

Another stream of consciousness, this one will be a story that will make some people go “no shit sherlock,” but it’s a lesson I had to learn on my own, so here goes:

My work wants me to make plans for “professional development,” every year I should be gaining skills or insights that I didn’t have the year before.  Professional development is a whole topic on its own, but for now let’s just know that I pledged to try to integrate machine learning into some of my workflows for reasons.

Machine learning is what we used to call AI.  It’s not necessarily *generative* AI (like ChatGPT), I mean it can be, but it’s not necessarily so.

So for me, integrating machine learning wasn’t about asking ChatGPT to do all my work, rather it was about trying to write some code to take in Big Data and give me a testable hypothesis.  My data was the genetic sequences of many different viruses, and the hypotheses were: “can we predict which animal viruses might spill over and become human viruses?” and “can we predict traits of understudied viruses using the traits of their more well-studied cousins?”.

My problem was data.  

There is actually a LOT of genetic data out there in the internet.  You can search a number of repositories, NCBI is my favorite, and find a seemingly infinite number of genomes for different viruses.  Then you can download them, play around with them, and make machine learning algorithms with them.

But lots of data isn’t useful by itself.  Sure I know the sequences of a billion viruses, what does that get me?  It gets me the sequences of a billion viruses, nothing more nothing less.

What I really need is real-world data *about* those sequences.  For instance: which of these viruses are purely human viruses, purely animal viruses, or infect both humans AND animals?  What cell types does this virus infect?  How high is the untreated mortality rate if you catch it?  How does it enter the cell?

The real world data is “labels” in the language of machine learning, and while I had a ton of data I didn’t have much *labelled* data.  I can’t predict whether an animal virus might become a human virus if I don’t even know which viruses are human-only or animal-only.  I can’t predict traits about viruses if I don’t have any information about those traits.  I can do a lot of fancy math to categorize viruses based on their sequences, but without good labels for those viruses, my categories are meaningless.  I might as well be categorizing the viruses by their taste, for all the good it does me.

Data labels tell you everything that the data can’t, and without them the data can seem useless.  I can say 2 viruses are 99% identical, but what does that even mean?  Is it just two viruses that give you the sniffles and not much else?  Or does one cause hemorrhagic fever and the other causes encephalitis?  

I don’t know if that 1% difference is even important, if these viruses infect 2 different species of animals it’s probably very important.  But if these viruses infect the same animals using identical pathways and are totally identical in every way except for a tiny stretch of DNA, then that 1% is probably unimportant.

Your model is only as good as your data and your data is only as good as your labels.  The real work of machine learning isn’t finding data, it’s finding labelled data.  A lot of machine learning can be about finding tricks to get the data labelled, for instance ChatGPT was trained on things like Wikipedia and Reddit posts because we can be mostly sure those are written by humans.  Similarly if you find some database of viral genomes, and a *different* database of other viral traits (what they infect, their pathway, their mortality rate), then you can get good data and maybe an entire publication just by matching the genomes to their labels.

But the low-hanging fruit was picked a long time ago.  I’m trying to use public repositories, and if there was anything new to mine there then other data miners would have gotten to it first. I still want to somehow integrate machine learning just because I find coding so enjoyable, and it gives me something to do when I don’t want to put on gloves.  But clearly if I want to find anything useful, I have to either learn how to write code that will scrape other databases for their labels, create *my own data*, or maybe get interns to label the data for me as a summer project.  

Stay tuned to find out if I get any interns.

My kingdom for a venv

I’ve never enjoyed using Python. I think my feelings on it can be summed up by this video. But for whatever reason, Python is unavoidable if you want to do anything with AI/machine learning. And so as someone wanting to get into AI, I have no choice but to use it.

But I don’t have to learn to code it of course, because all the tools you need for AI area already written and available. ChatGPT is of course easy to use on the web. But what if you wanted to have a version of ChatGPT that was snarkier, or wrote better jokes, or was in whatever way tuned specifically for your needs and wants? In that case, you can always make a fine-tuned language model and use it yourself.

But that’s where Python rears its ugly head. I wanted to fine tune a language model. So I installed LLaMA, downloaded a simple model from huggingface, and got to work. 

To fine-tune a model for your own needs, you need to have data and you need to annotate that data. No time to explain how annotations work, but there are programs that make it easy. There is a program called Label Studio that I thought I could use. The instruction say to just download python, make a venv (virtual environment) and have pip (a python installer) install Label Studio. Sounds easy, right? Just 3 lines of code.

The trouble started almost immediately because despite Label Studio telling me it was available for Windows, the install instructions were actually written for Linux. I realized this and corrected it, but the trouble didn’t stop. Once I created the venv, I tried to install Label Studio, but one of the dependencies failed to install so the whole process failed.

Uh… what? Why is this program, which is available as a paid enterprise product by the way, failing to install itself due to a dependency issue? I find the missing dependency and try installing it directly to the venv, hoping that fixes the issue. But no, it still errors out. What am I missing?

So it turns out that when I directly install that dependency, it installs the latest version of it. But Label Studio is looking for a specific older version, so it still tries to install the older version when installing itself. I tried to install the specific older version, and that fails too. Apparently I can install the new version with no issues, but not the old version.

Reading the message closely, it says that to install the old version I need to have another python module installed and also add that other module to the system path. Now we’re getting into part of why I hate venv. The thing about Python is that if you install itself outside of a contained environment, it infects your computer and doesn’t get out. Ask an amateur pythonist how to remove an old version of Python, and see the blank look on their face. Just deleting the folder doesn’t fix it.

And this old version/new version bs can mess you up something fierce, because some other python module will start looking for what it needs, and find the old version instead of the new version. Or it will be sent to where the old version used to be, but finding nothing there it will error out. Venv is supposed to fix all this so you only install things into designated containers where they can’t escape.

But I can’t do that, because to install something into this venv, I have to install another package and add it to the path of my entire Windows system. So the venv isn’t even doing what it’s supposed to do!

So I gave up. I hate having to use python like this, normal programs will just come to you as an executable or a zip and you use them. Python always needs to install itself everywhere and then usually fails even then. So I won’t use Label Studio and will look for another tool instead.

If anyone knows of a good annotation tool for LLM data, hit me up.

Good idea: financially supporting workers displaced by AI. Bad idea: taxing companies when for displacing workers with AI.

AI is again the topic of the day and people are discussing what to do about the coming “job-pocolypse.” It seems AI can do anything we humans can do better and so 30% or more of jobs will be destroyed and replaced by AI. Leaving aside how accurate that prediction is, if 30% of all jobs will be impacted then it does warrant a public policy response. Everyone’s got their own personal favorite, but one I see come up again and again is that companies should face a hefty tax any time they replace a worker with AI.

To be blunt, taxing companies for replacing workers with AI is a terrible idea. Let’s leave aside the argument of “how do you prove it,” and cut straight to the fact that the government should not be taxing technological progress. Just to start with some history, how many farmers were displaced by tractors? Millions. In 1900 40% of Westerners worked on farms, now it’s less than 5%. Tractors meant that a single farmer could do the labor of tens or hundreds of men, and so they could fire many of their farm hands to be replaced by tractors. But does anyone reading this wish nearly 1/2 of us were still farmers? Should the government have heavily taxes tractors to preserve the idyllic rural farm life?

The argument in favor of taxing companies that replace workers with a machine is that the company is becoming more profitable at the expense of the worker, and they should pay it back. The current hullabaloo is about being replaced by AI, but in the 20th century similar calls were made when factory workers were being replaced by robots. The problem with this argument is that ignores society. The worker and the company are not the only 2 pieces of the equation, society in general benefits when companies become more efficient. Technology is deflationary, and it has allows many products to drop or price or not increase as rapidly as wages in general. Food today costs less as a percent of annual income than at nearly any time in history, and a large part of that is because the cost of food is decoupled from the cost of labor. So farm hands being replaced by tractors helped all of society by giving us cheaper food, and all of society would have been harmed if taxes had been instituted to prevent tractors from becoming commonplace.

Are the workers harmed when their jobs are replaced by AI? Yes of course. But society itself is helped and so all of society should bear the costs of helping the workers. We should of course offer unemployment benefits and job retraining to those affected. We should not let them go by the wayside the way we did to blue collar factory workers in the 20th century.

But neither should we shoot society in the foot by blocking technological progress that will help all of us. AI replacing jobs will mean products will become cheaper relative to wages, just as what happened with food. A lot of people also spread nonsense that unemployment will skyrocket as the displaced workers can’t find other jobs. They misunderstand economics, there will always be demand for more jobs. The price of some goods will decrease thanks to AI, but that means that people can buy more of those goods or buy more of others goods that they put off buying because they were forced to choose and only had so much money. As prices fall, demand will rise, raising demand for labors in other areas, and a new equilibrium will be reached. Those jobs lost due to AI don’t mean the workers will be forever jobless, any more than 35% of the population displaced by tractors meant that unemployment skyrocketed in the 20th century. Time and time and time again technology has replaced the jobs of workers, and the workers have found new jobs. It will happen again with AI.

If the weavers get replaced by machines, who will buy the clothes?

I’ve seen way too many articles about AI casting doom and gloom that it will “replace millions of jobs” and that this will lead to societal destruction as the now job-less replacees have no more money to spend.  The common refrain is “when AIs replace the workers, who will buy the products?”

This is just another fundamental misunderstanding of AI and technology.  AI is a multiplier of human effort, and what once took 10 men now takes 1.  That doesn’t mean that 9 men will be homeless on the street because their jobs are “replaced.”  The gains reaped from productivity are reinvested back into the economy and new jobs are created.

When the loom replaced hand-spinning weavers, those weavers were replaced.  But they could eventually find new jobs in the factories that produced looms, and in other factories that were being developed.  When computers replaced human calculators, those calculators could now find jobs programming and producing computers.

For centuries now, millenia even, technology has multiplied human effort.  It used to take dozens of people to move a single rock, until several thousand years ago someone had the bright idea of using ropes, pullies, and wheels.  Then suddenly rocks could be moved easily.  But that just in turn meant the demand for moving rocks shot up to meet this newer, cheaper equilibrium, and great wonders like the Pyramids and Stonehenge were suddenly built.

The same will be true of AI.  AI will produce as many new jobs as it creates.  There will be people to produce the AI, people to train the AI, people to ensure the AI has guardrails and doesn’t do something that gets the company trending on Twitter.  And there will be ever more people to use the AI because demand is not stable and demand for products will rise to meet the increase in supply generated by the AI.  People will want more and more stuff and that will lead to more and more people using AI to produce it.

This is something that people get hung up on, they think that demand is stable.  So when something that multiplies human effort gets created, they assume that since the same amount of products can be produced with less effort, that everyone will get fired.  Except that demand is not stable, people have infinite wants and finite amounts of money. 

Technological progress creates higher paying jobs, subsistence farmers become factory workers, factory workers become skilled workers, skilled workers enter the knowledge economy of R&D.  These new higher paying jobs create people who want more stuff because they always want more stuff, and now have the money to pay for it.  This in turn increases demand, leading to more people being employed in the industry even though jobs are being “replaced” by machines.

To bring it all back to weavers, more people are working in the textile industry now than at any point in human history, even though we replaced weavers with looms long ago.

AI will certainly upend some jobs.  Some people will be unable or unwilling to find new jobs, and governments should work to support them with unemployment insurance and retraining programs.  But it will create so many new jobs as well.  People aren’t satisfied with how many video games they can purchase right now, how much they can go out to restaurants, how much housing they can purchase, etc.  People always want more, and as they move into higher paying jobs which use AI they will demand more.  That in turn will create demand for the jobs producing those things or training the AIs that produce those things. 

It has all happened before and it will happen again.  Every generation thinks that theirs is the most important time in the universe, that their problems are unique and that nothing will ever be the same.  Less than three years ago we had people thinking that “nothing will ever be the same” due to COVID, and yet in just 3 short years we’ve seen life mostly go back to normal.  A few changes on the margins, a little more work from home and a little more consciousness about staying home when sick, but life continued despite the once-a-century upheaval.

Life will also continue after AI.  AI will one day be studied alongside the plow, the loom, and the computer.  A labor-saving device that is an integral part of the economy, but didn’t lead to its downfall.

The AI pause letter seems really dumb

I’m late to the party again, but a few months ago a letter began circulating requesting that AI development “pause” for at least 6 months. Separately, AI developers like Sam Altman have called for regulation of their own industry. These things are supposedly happening because of fears that AI development could get out of control and harm us, or even kill us all in the words of professional insanocrat Eliezer Yudkowsky, who went so far as to suggest we should bomb data centers to prevent the creation of a rogue AI.

To get my thoughts out there, this is nothing more than moat building and fear-mongering. Computers certainly opened up new avenues for crime and harm, but banning them or pausing development of semiconductors in the 80s would have been stupid and harmful. Lives were genuinely saved because computers made it possible for us to discover new drugs and cure diseases. The harm computers caused was overwhelmed by the good they brought, and I have yet to see any genuine argument made that AI will be different. Will it be easier to spread misinformation and steal identities? Maybe, but that was true of computers too. On the other hand the insane ramblings about how robots will kill us all seem to mostly amount to sci-fi nerds having watched a lot of Terminator and the Matrix and being unable to separate reality from fiction.

Instead, these pushes for regulation seem like moat-building of the highest order. The easiest way to maintain a monopoly or oligopoly is to build giant regulatory walls that ensure no one else can enter your market. I think it’s obvious Sam Altman doesn’t actually want any regulation that would threaten his own business, he threatened to leave the EU over new regulation. Instead he wants the kind of regulation that is expensive to comply with but doesn’t actually prevent his company from doing anything it wants to do. He wants to create huge barriers to entry where he can continue developing his company without competition from new startups.

The letter to “pause” development also seems nakedly self-serving, one of the signatories was Elon Musk, and immediately after Musk called for said pause he turned around and bought thousands of graphics cards to improve Twitter’s AI. It seems the pause in research should only apply to other people so that Elon Musk has the chance to catch up. And I think that’s likely the case with most of the famous signatories of the pause letter, people who realize they’ve been blindsided and are scrambling to catch up.

Finally we have the “bomb data centers” crazies who are worried the Terminator, the Paperclip Maximizer or Roko’s Basilisk will come to kill them. This viewpoint involves a lot of magical thinking as it is never explained just how an AI will find a way to recursively improve itself to the point it can escape the confinement of its server farm and kill us all. In fact at times these folks have explicitly rebuked any such speculation on how an AI can escape in favor of asserting that it just will escape and have claimed that speculation on how is meaningless. This is of course in contrast to more reasonable end-of-the-world scenarios like climate change or nuclear proliferation, where there is a very clear through-line as to how these things could cause the end of humanity.

Like I said it I take this viewpoint the least seriously, but I want to end with my own speculation about Yudkowsky himself. Other members of his caucus have indeed demanded that AI research be halted, but I think Yudkowsky skipped straight to the “bomb data centers” point of view both because he’s desperate for attention and because he wants to shift the Overton Window.

Yudkowsky has in fact spent much of his adult life railing about the dangers of AI and how they’ll kill us all, and in this one moment where the rest of the world is at least amenable to the fears of AI harm, they aren’t listening to him but are instead listening (quite reasonably) to the actual experts in the field like Sam Altman and other AI researchers. Yudkowsky wants to maintain the limelight and the best way to do so is often to make the most over-the-top dramatic pronouncements in the hopes of getting picked up and spread by both detractors, supporters and people who just think he’s crazy.

Secondarily he would probably agree with AI regulation, but he doesn’t want that to be his public platform because he thinks that’s too reasonable. If some people are pushing for regulating AI and some people are against it, then the compromise from politicians who are trying to seem “reasonable” would be for a bit of light regulation which for him wouldn’t go far enough. Yudkowsky instead wants to make his platform something insanely outside the bounds of reasonableness, so that in order to “compromise” with him, you’ll have to meet him in the middle at a point that would include much more onerous AI regulation. He’s just taking an extreme position so he has something to negotiate away and still claim victory.

Personally? I don’t want any AI regulation. I can go to the store right now and buy any computer I want. I can to go to a cafe and use the internet without giving away any of my real personal information. And I can download and install any program I want as long as I have the money and/or bandwidth. And that’s a good thing. Sure I could buy a computer and use it to commit crimes, but that’s no reason to regulate who can buy computers or what type they can get, which is exactly what the AI regulators want to happen with AI. Computers are a net positive to society, and the crimes you can commit on them like fraud and theft were already crimes people committed before computers existed. Computers allow some people to be better criminals, so we prosecute those people when they commit crimes. But computers allow other people to cure cancer, so we don’t restrict who can have one and how powerful it can be. The same is true of AI. It’s a tool like any other, so let’s treat it like one.