A pattern governing supply and demand changes

[Scouse Gits]

That confirms my observations - Dye is always demanded, but when the screen is refreshed the Dye becomes Hides.

---------------------

SG(2)

[samson]

Solo - Okay.
What if you use Reveal Map on all these Dye-hungry cities? Does Dye go away? Or does it stick?

[solo]

One small correction first.

In the SP game, all the AI were given dye, but the human player went through the first few cycles without getting it.

Cheat mode can not be used to reveal the whole map in a hot seat game, but the map is pretty much revealed to all civs at this stage due to map trades.

In my hot seat game all cities except for a few of the newest, yet to be "cycled", now have dye in demand slot #1.

SG, I am not understanding what you mean by refreshing the screen.

Samson, I will try and locate and reload my SP test game and reveal the map to see if all the AI still demand hides.

[solo]

Okay, I am back. Reveal map got rid of all the dye, but the trade demand lists no longer correspond to city lists. Attached is the save.

[Scouse Gits]

solo ... I'm probably using the wrong jargon but this is what I mean:

In cheat mode > Reveal Entire Map > Create Settler > Build City > Monitor Supply/Demand on first viewing the city screen. Then come out of new city to the map of the world (refreshing) and return to cheat city. Commodities have changed. Dye demand (in early years) is usually Hides.
Typically:
Supply: Silk Beads Salt
Demand: Dye Copper Wine

changes to:

Silk Dye Beads
Hides Copper Wool

-----------------------

SG(2)

[William Keenan]

Solo, here are the vb files you need.

[samson]

Sg - Thanks for the confirmation of the "Dye effect". I strongly believe that Reveal Map is showing the "correct" S/D values. Solo's save confirms this, too. However, it is unsettling to hear that the solo-cycle doesn't produce the same fix as Reveal Map.

As for map exploration, I don't believe it is a factor in Supply and Demand. And I don't think Reveal Map simulates exploration. Reveal (Entire) Map, I believe, does exactly that. And part of the revealing involves updating the Supply/Demand stuff when you go into City Display.

Solo - thanks for the save. Very helpful.

Another trigger event for S/D updates is the delivery of a carvan to help build a Wonder. Food caravans work just as well as commodities. This could be a good tactic, using food caravans to update S/D in a city when you know the supply is set to change.

[solo]

William,

Thanks a bunch for those files. I will download and give them a try with civcity.

Samson,

I'm a little confused by your statement that reveal map shows "correct" S/D values, as it appears to me that its use replaces the correct values (i.e. current values attained so far by game play) with arbitrary ones that do not even tally with the Trade Advisor S/D screens.

I do agree that use the reveal map cheat does not simulate total map exploration, since when I trade maps, the demand lists of AI capitals do not instantly change the way they always do when reveal map is used.

I agree about food caravans as another trigger. I have forgotten about terraforming, which we can addd to the list of independent triggering events, now consisting of:

1) 16 turn cycles
2) caravan deliveries (may affect Source or Destination, and possibly both)
3) food contributions to wonders (include ordinary food deliveries to cities, too)
4) terraforming (with added chances of a change when the tile contains a special)

[samson]

Solo -

The S/D values shown by Reveal Map and not arbitrary at all. They are "correct" in the sense that they are the values the city will take on after the next trigger event occurs, with the exception of the DYE problem with have discussed. Demand for Dye works strangely under gameplay but shows quite regular behavior when Reveal Map is used. Of course, gameplay behavior is what matters and so will have to be accounted for at some point.

Terraforming is not a trigger event. I think at one point I said it was, but it isn't. Changes to terrain show immediate S/D results under Reveal Map but not during normal gameplay. They need a trigger event to take effect.

Caravan deliveries can act as a trigger for either the Source or Destination city, but I have not yet seen them affect both at once. Odd.

[solo]

Samson,

Okay, I see what you mean by "correct", in that reveal map shows the next change, which is predictable, and not arbitrary. So am I right in now assuming that reveal map acts as an instant trigger of the next S/D change for each city?

Terraforming can come off the trigger list. I thought had remembered a previous post by you where you had altered the timing of an S/D change by changing the rate of terraforming a tile.

I am pumping out caravans now in my test game and will continue to monitor how often and where deliveries act as triggers. Source and Destination changes at the same time should be possible, if both cities have been enabled by independent causitive events.

More later.

[samson]

From the thread "Ancient Words of Wisdom"

Originally posted by debeest
In the game I just finished, I traded dye profusely for a long time, but somewhere around 1500-1700 the demand began to diminish. By the end of the game in 1800, no city, domestic or foreign, had demanded dye for quite some time, and it seemed as if dye was the only commodity any of my cities would produce. During that time, I grew from maybe 40 cities to maybe 80; the AI dropped from about 35 to 0. Partly republic, partly democracy.

The thing that caught my attention was: when I would save the game, quit, and come back later, if I checked the trade adviser, there were dozens of cities demanding dye -- but as I clicked on those cities to examine them, they didn't really want dye, and then they weren't listed that way by the trade adviser anymore.

I've heard of commodities having volatile demand, but that's going a bit too far!

It would seem that the false demand of Dye is a common phenomenon.

[Elephant]

There does seem to be a "difference of opinion" between what the Trade Advisor lists as Demand and what a click on the map or a glimpse of the CityScreen lists. This is a high-priority issue as it is what mostly frustrates people into thinking the Trade system is broken. In the example I listed earlier, the CityScreen listed one thing (a change), but the Trade Advisor listed something else (no change), and returning to the CityScreen the S/D was back to normal.

Here's a snip from the beginning of one of my test games, using SG's large map start where the English got Trade as a freebie:

1el_b4000
free techs:Alph/Bro/Cer/CoL/Cur/Hor/Map/Tra

2 settlers started on Plains; moved 2 spaces to Grass.
3900- founded LONdon (c9/101,15) on GS, working Whale
__Supplies Silk/Salt/Dye
__Demands Hide/Wine/Bead
founded YORk (c9/106,16) on GS, working Forest
__Supplies Silk/Salt/Dye
__Demands Hide/Cop/Bead

TradeAdvisorCheck:
(Hmmm- no one wants Dye...)
Hides: Cardiff/Zimbabwe/Crewe/Beijing/LON/YOR/(Delhi)
Wool: (Delhi)
Beads: Cardiff/Zimbabwe/Madrid/LON/YOR
Cloth: Cardiff
Wine: Zimbabwe/Crewe/Beijing/LON/(Delhi)
Spice: Madrid/Beijing
Gems: Crewe/Madrid
no: Wool/Coal/Dye/Silver/Gold/Oil/Uranium

so looks like
C01=Cardiff (Hide/Bead/Cloth)
C02=Zimbabwe (Hide/Bead/Wine)
C03=Crewe (Hide/Wine/Gem)
C04=Madrid (Bead/Spice/Gem)
C05=Beijing (Hide/Wine/Spice)
C06=LONdon
C07=YORk
3800- Delhi (new add- added to lists above in parens)
C08=Delhi (Hide/Wool/Wine)
MONARCHY; start Write
3700- Warrior in LON & YOR, start MPE in LON, Settler in YOR
3550- Delhi now Wool/Dye/Wine
3500- LON now Supplies Silk/Salt/GEMS, same Demand
3450- Beijing now Beads/Dye/Wine
3400- Madrid now Cloth/Dye/Gems

Notice that there is NO demand for Dye at start; Delhi, the eighth city, got it first in 3550BC. The commodity sequences are NOT correct; I was interpreting them from the TradeAdvisor, not from map clicks or hot-seating. Come to think of it, if we suspect the TradeAdvisor is not correct, or there is some sort of trigger event with using it, these could be tainted data. I think we need to resolve the "reliability" issue of TradeAdvisor lists (once Trade is discovered) versus CityScreen lists (shows Demand and Supply lists IN SEQUENCE, but to get them all you need to run a hot-seat game) versus map clicking. Otherwise we'll get inundated by questionable data points.

[samson]

Elephant,

You raise a number of good points. I'll give you my current understanding of them.

1) The Trade Advisor itself is not a trigger event. Using it does not cause changes to supply and demand commodities. It merely displays the saved S/D values of each city when it was last updated.

2) There are three clear trigger events. The 16-turn solo cycle; the delivery of caravans (commodity or food) for trade or Wonders; and having the Reveal Map command on in the Cheat Menu. The last of these doesn't cause changes directly, but forces all cities to update their S/D lists when clicked on.

3) The algorithm for the calculation of the demand of Dye has a bug in it, in my opinion, which causes many cities to demand Dye that don't really "want" it. Since the supply and demand lists are a result of comparing the SQs and DQs of all commodities, a bug in one formula such as Dye, can falsely promote or demote other commodities as well. This means other items on the lists can be "incorrect" also.

Now, don't misunderstand. "Incorrect" doesn't mean that the city won't pay the demand bonus for a commodity it says it demands. It will. It's just that the demand of Dye is ephemeral and will disappear when a caravan delivery triggers an update to the city's S/D lists. If the Dye demand was indeed false, both lists are likely to change dramatically.

4) The Demand list shown by clicking on a visible foreign city is in the same sequence shown in its city display, as investigated by a diplo.

[solo]

I will check the Trade Advisor in my hot seat game, to see if I can find differences from the actual lists I see in each city.

A delivery of dye has been made in my game already to a city "addicted" to it. Nothing happened to the S/D list of the Destination city, but the Source city, Washington, had two supply AND two demand changes. Washington was able to supply and build this dye only because the commodity was built on a cycle turn, where a glitch resulted in momentary supply.

Elephant, in an SP game, so far I have observed that all the AI want dye, and only the human supplies it. Thus, it is no surprise to me to see demand disappear as they AI cities were destroyed or captured.

And William, thanks for supplying those .DLL's. The only thing I wish I had more than that right now would be a copy of the Rise of Nations beta being circulated among reviewers.

[samson]

Originally posted by solo

A delivery of dye has been made in my game already to a city "addicted" to it. Nothing happened to the S/D list of the Destination city, but the Source city, Washington, had two supply AND two demand changes. Washington was able to supply and build this dye only because the commodity was built on a cycle turn, where a glitch resulted in momentary supply.

I have seen a caravan delivery to a foreign city cause its false Dye demand to disappear immediately. This was in an SP game. So there's evidence on both sides of this question.

I'd like more info on this glitch in Washington. Can you give me the S/D lists before and after the event, plus the supply list that disappeared? Also, if you would, the coordinates of Washington.

[samson]

An update.

Caravan deliveries act as a trigger event in both the source and destination cities. Of course, this doesn't mean it always cause an S/D list change. Both when conditions are right, changes in both cities will be made visible.

[Elephant]

My question about TradeAdvisor versus CityScreen was because they were giving different lists on the same turn, and the CityScreen changed back to agree with the TradeAdvisor AFTER I cross-checked with the TradeAdvisor.

In checking my notes I missed the fact that the city had promoted from 1 citizen to two that turn. Could that have been the "trigger" for a S/D change?

Here is some of the log:

pre-first-move:
Hides-Card,Zimb,Crewe
Beads-Card,Zimb,Madrid
Cloth-Card
Wine-Zimb,Crewe
Spice-Madrid
Gems-Crewe,Madrid
C01-Cardiff (Hide,Bead,Cloth)
C02-Zimbabwe (Hide,Bead,Wine)
C03-Crewe (Hide,Wine,Gems)
C04-Madrid (Bead,Spice,Gems)
(NOT sequenced correctly)

4000 - founded LONdon (c9/103,17) on Plains
__supplies Silk,Dye,Gold
__demands Hides,Beads,Wine
start Monarchy
3950-
TradeAdvisor Demand:
_Hides- Card,Zimb,Crewe,LON,Beijing
_Beads- Card,Zimb,Madrid,LON
_Cloth- Card
_Wine- Zimb,Crewe,LON,Beijing
_Spice- Madrid,Beijing
_Gems- Crewe,Madrid
so C06-Beijing is Hide,Wine,Spice
3900- Demand: same
3850- worked square has new road; Demand: same
3800- new city C07-Delhi: Hides,Wool,Wine
3750- Demand: same
3700- Demand: same
3650- HUT (104,22): WarCode
Demand: same
3600- Demand: same
3550- Demand: same
3500- Demand:
Hides- Card,Zimb,Crewe,LON
Wool- Delhi
Beads- Card,Zimb,Madrid,LON,Beij
Cloth- Card
*Dye- Beij,Delhi
Wine- Zimb,Crewe,LON,Beij,Delhi
Spice- Madrid
Gems- Crewe,Madrid
so Beij now Bead,Dye,Wine
and Delhi now Wool,Dye,Wine
(could Delhi have changed last turn AFTER me, then Beijing this turn?)
3450- LON size 2, disorder (fixed with Elvis)
__LON change:
__Supplies Silk,Gold,Gems
__Demands Dye,Hides,Beads
TradeAdvisor Demand: same
!!!after Trade Advisor, LON returns to S:Silk,Dye,Gold; D: Hide,Bead,Wine
3400- Madrid now Cloth,Dye,Gems

Could the temporary Disorder on getting to size 2 have caused a S/D change, which then reverted when I corrected it? Have you tested for city size or disorder effects yet?

[solo]

Here's the dope on Washington:

coordinates (75,51)
supply wildcard - beads
demand wildcard - salt

2250 s: silk,salt,gems d: dye,hides,beads
1850 s: hides,silk,salt d: dye,beads,wine
1050 caravan builds dye, options were hides,silk,dye
1050 s: hides,silk,salt d: dye,copper,beads
875 dye(d) delivered s: silk,beads,cloth d: dye,salt,wine

Attached is a hot save file in case you would like to check out terrain around Washington. Save is for the turn just before 1050.

Elephant,

My Trade Advisor lists look okay. City growth (i.e. changes in size) can cause S/D changes. I am not sure about disorder, but it's something I could test for:

[samson]

Elephant,

Okay, I loaded up the Trade start you are working with and saw the problem you're talking about. And I think I can explain what is happening here.

First of all, London is the 5th city founded, not the 9th as your log indicates. This means that London's first solo-cycle year occurs in 3450BC. Now, before that turn London's S/D list looks like this:

Supply: Silk, Dye, Gold
Demand: Hides, Beads, Wine

The wildcards are: Supply = Beads; Demand = Beads.

Curiously both the wildcards are the same and also (as it turns out later) the same as the true 2nd place Demand item. Thus, we are looking at actual S/D rankings here without wildcard substitutions.

In 3450BC London goes into disorder and doing Zoom-to-City brings up a City Display with these S/D lists:

Supply: Silk, Gold, Gems
Demand: Dye, Hides, Beads

Again, no wildcard substitutions occur because Beads is already on the Demand list. However, the "Dye bug" has produced an artificially high demand for Dye. So much so, that Dye's DQ is now greater than its SQ and it jumps from the Supply list to the top of the Demand list. This pushes Hides and Beads down to 2nd and third place. On the Supply list, the vacancy created is filled by Gold moving up to 2nd place and Gems joining the list.

Now, after you exit the Zoom-to-City display and return to the map screen, you looked at the Trade Advisor. It shows what it thinks are the correct S/D values in its display (exactly what you saw in the Zoom-to-City display. The Trade Advisor doesn't update anything, it just displays the last stored values for each city's S/D lists. The Trade Advisor is working fine.

However, if you then return to London and enter the City Display, the S/D lists are back to what they originally were. These are the "correct" values. After viewing the City Display, you return to the Trade Advisor which now shows the original values again.

This is all a manifestation of the "Dye bug". The problem is that for each different way that the S/D list updater is invoked, the results for Dye's Demand Quotient can be different. The ways we know about are:

1) 16-turn cycle. S/D lists are updated automatically at start of turn.
2) City Display. If they on a solo-cycle year, a city's S/D list is updated every time the City Display is entered.
3) Zoom-to-City during the pre-turn on a solo-cycle year.
4) Reveal Map used with City Display.
5) Caravan deliveries.

Apparently, the interface between each of these events and the S/D list updater is slightly different and can effect the calculation of Dye's Demand Quotient. Because of the interrelated nature of commodity calculations, this single bug can affect other commodities on both the Demand and Supply lists, as your example illustrates.

The Trade Advisor appears blameless in all this.

[solo]

Sorry for uploading the wrong hot file. Here is Washington just before dye is built:

[samson]

Solo,

It looks like the "Dye bug" operates differently on the Gold edition and 2.4.2. Using your "Dye.sav" on 2.42, I can send caravans into Dye-demanding cities and it triggers a change in S/D lists, removing Dye from the demand lists. But on Gold Edition it doesn't. It seems that on Gold, Dye-demanding is the persistent state and non-Dye-demanding is transitory. On 2.4.2 it's just the opposite.

Your Washington glitch and Elephant's London glitch seem to be the same thing. In your case, on Gold, the non-Dye-demand state occurs briefly during the pre-turn phase when you build a caravan on a cycle year. Then the Dye-demand state returns. In Elephant's example just the opposite happens in 2.4.2.

A couple explanations:

1) The dye bug manifests differently on Gold and 2.4.2
2) The dye bug has been "fixed" on Gold, and Dye-demanding is the intended result. (A slight glitch still occurs on cycle years.)
3) The Trade system was revamped for Gold Edition and there are other surprises in store.

I think #1 is the most likely although #2 is possible. My testing with S/D formulas is being done on 2.4.2 and I sure hope things are not too different on Gold.

[solo]

Samson,

Hmm, I was thinking the differences were between MP and SP, regarding dye, but if dye deliveries are always triggering changes on Destination demand lists in 2.42, then there must be some difference between versions, too.

I have calculated all the wildcards for the cities in my MGE hot seat game, and there is only one city I have to check out a little more for errors on my part while doing this. All other cities are showing behavior consistent with what you describe for wildcards, so I would bet that the treatment of individual commodities will remain consistent between versions of the game, too.

Armed now with the knowledge of the masking effect of the middle positions by wildcards, I want to do a little more analysis on commodity trends, while taking wildcard action into account. I'll be back later with those results.

[samson]

Solo,

Thanks for checking the Wildcard stuff on MGE. It's a big help. I was beginning to think I'd have to do all my testing over again, if the versions were too different.

[solo]

My latest thought about dye, is that this extra demand could be on purpose to make the shifting of commodities on the demand lists even less noticeable.

Some news about dye, which I will add here with an edit. After a beads(d) delivery to Leipzig in 725 BC, copper replaced dye on Leipzig's demand list. Leipzig is now size 5, and this was the second time it has received a delivery. Unless copper has become supercharged, the way dye seemed to be before, it appears that dye's lead over other commodities was not as commanding as it first seemed. Either that, or something happened recently to make cravers of dye come to their senses.

So dye is behaving the way you described in your 2.42 game, where deliveries also were taking it off the demand list. A few more details:

before: hides,(silver),(wool) dye,salt,beads
after: hides,(silver),(wool) copper,salt,(beads)

Leipzig wildcards are silver and salt.

Since dye is amenable to change, it makes me back off more from the glitch theory, since with a glitch, I would expect a much greater difference between dye's quotient than those of its nearest competitors on the demand list.

[samson]

Solo,

Interesting that you mention Copper. I have been testing Copper today. I agree its DQ does exhibit some of the same behavior as Dye's, but only after the city's size grows above 3. The fact that two commodities are showing this "supercharged" demand makes it more likely to be by design. Also, Dye is more well-behaved on MGE than it is on 2.4.2. So it's possible that some kind of glitch was fixed in that version.

What really bothers me about Dye, is not the huge DQ is seems to have but the flakiness it seems to introduce into the whole Trade system, as in Elephant's example and some of your own.

[solo]

One more note about dye, is that ALL cities were demanding it at the time this delivery happened. No new cities had been founded for enough turns to allow all of the newest ones to be "dyed" on their first cycle turns.

[samson]

My latest thoughts on Dye.

There are actually 3 commodities with "supercharged" demand. Dye, at the start of the game; Copper, after cities reach size 5 (you were right about that, it's size 5 not size 3 that triggers the "copper rush"); and Uranium, after Nuclear Fission is discovered. The existence of three such strong demanders is evidence against the "bug" theory of Dye. [Although I still think 2.4.2 has a bug that was fixed in MGE.]

Since we have 3 commodities at different stages of the game that produce remarkably strong demands, I'm starting to think that this is a designed feature. Perhaps the fact that I have found little correlation of demand strength to terrain or other factors just means that I just don't know what the demand of these items is based on. Here's what I do know about the demands of these commodities:

Dye: demand is reduced by Chemistry and Mass Production.

Copper: demand starts at size 5, is increased by Electricity, reduced by Computers.

Uranium: demand starts with Nuclear Fission, increased by # of techs and city size.

Today, I'm going to look at Uranium a bit closer, see if I get some clues there.

[solo]

Yes, copper is starting to show a lot of strength. I will monitor it as more cities grow to size 5. Copper is even climbing lists on smaller cities at this time.

One thing your tests might not be simulating are phases of the game when the addition of new cities slows down and even stops for periods of time. Dye may be used for keeping track of this, since it is immediately added to fresh cities on their first cycle turns. This always happens, except to the human player in SP games, so it seems quite deliberate to me, since most new cities do not start off with dye as #1 in demand.

It could be that copper gradually starts to take the place of dye when this stage of a game is detected.

This last turn, all 3 caravan deliveries have triggered S/D changes, in both the Source and Destination cities, although one Source had its cycle turn at the same time.
Each cargo happened to be hides and was demanded.

[samson]

Solo,

You're right. The turn number (or game year) could be a factor that alters supply and demand in any commodity. I haven't tested for that at all. Another thing you mention is the "human player". All my testing as been done with "human" cities. Some formulas may work differently with AI civs.

If you look at the latest formulas I've added to the list, most of the supply sides are done, but I have lots of demand missing. It is much easier to crack Supply formulas because the basis I use for comparison in order to adjust all formulas to the same scale is COPPER SUPPLY. It is the simplest fomula I've found, based solely on Hills and Mountains with the same coefficient. Originally I used a "copper=1" scale, but got a lot of fractional coefficients in other formuals. Currently, I'm using "copper=5" and have mostly integer coefficients to deal with now.

To scale Demand Side formulas, I've got to get a commodity to appear on the Supply side first (at approximate equality to Copper) and then flip it to the Demand side by pumping up what I think affects its demand. The point at which it jumps to the Demand side is its point of approximate equality to Copper's SQ. Then it's a matter of finding and measuring all factors and doing the algebra.

[solo]

Yes, I can see how supply would be more likely to be tied to terrain, which is more of a constant, and much easier for conducting systematic tests.

Demand is probably quite more dynamic, changing more often, and tied more to the progress of a civ through the different stages of the game, something harder to simulate easily while conducting tests. Because of this, I am making saves of my game after every player turn, in case the game becomes of use for demand tests, later.

One thing I have noticed in all my games is that the Germans are always wanting wine, much more so, on average, than other nationalities. Demands may be linked more to individual civs (not just colors). I know you are testing for this, but are all civs being included in a systematic way?

I have looked over the additons to your commodity lists, and see a lot of progress. When you have your best estimates in place for all supplies and demands, I will want to be checking these against what actually occurred in my test game. This may be a good way of fine tuning the formulae, or detecting new influencing factors.

For now, it's on with my game, in which it is taking a lot of time to make progress in, since I don't want to omit what might turn out to be critical information.