726 replies to this topic

[jimdahl]

We're only a month or so away from the Bracketology articles starting to get interesting and after a few weeks of playing around, I think I'm getting close to some useful ways to present PWR forecasts.

So, without further ado:

Predicting the PWR… The impact of the Denver series
For those who can't be bothered to follow a link -- surprise, surprise, this is the biggest series of 2009 so far for PWR.  A sweep will land UND an average of 8.5 spots higher in the PWR than getting swept.  Other series this weekend that could move UND more than 1 spot: Northern Michigan over Alaska, Bemidji State over Niagara, SCSU over CC

[AZSIOUX]

thx, i just asked you about this on the denver thread. thx

[jimdahl]

AZSIOUX, on Jan 22 2009, 08:07 PM, said:

thx, i just asked you about this on the denver thread. thx

Yeah, busy week.  Now that I think I'm getting close to some readable, useful analysis of the forecasts, the next step will be to automate it so its not dependent on me having the time to run the simulation and type up the results.

[Goon]

jimdahl, on Jan 22 2009, 09:10 PM, said:

Yeah, busy week.  Now that I think I'm getting close to some readable, useful analysis of the forecasts, the next step will be to automate it so its not dependent on me having the time to run the simulation and type up the results.

Two words: Just Win... And Go Sioux.

[AZSIOUX]

Goon, on Jan 22 2009, 07:18 PM, said:

Two words: Just Win... And Go Sioux.

isnt that 5 words ?  

[Goon]

AZSIOUX, on Jan 22 2009, 10:23 PM, said:

isnt that 5 words ?  

The two words are just win.

Go Sioux.

[AZSIOUX]

Goon, on Jan 22 2009, 07:44 PM, said:

The two words are just win.

Go Sioux.

ummm, yeah i got that  

[Goon]

AZSIOUX, on Jan 22 2009, 11:04 PM, said:

ummm, yeah i got that  

I didn't study math.

[siouxweet]

other than knowing the exact formula that goes into the rpi and how it is figured out I'm not sure why people say the pwr rankings are so confusing to figure out.  they are pretty simple.  match up each team and see who wins the most catagories.

[AZSIOUX]

siouxweet, on Jan 22 2009, 08:45 PM, said:

other than knowing the exact formula that goes into the rpi and how it is figured out I'm not sure why people say the pwr rankings are so confusing to figure out.  they are pretty simple.  match up each team and see who wins the most catagories.

yeah, just takes tons of time. thankfully jim has a nice system going on to help us all out

[Goon]

Because of people like Jim we don't have to. For that I am thankfull.

[jimdahl]

Having last week's chart in the thread keeps me honest, but I declare victory on that, with 3 points we landed between the thick parts of the forecast for a sweep and a split.  (Keep in mind that when I give a "90% chance" range, we are by definition going to land outside of that range 1 out of 10 times).

From Forecasting the PWR… a look ahead at SCSU:


One point of clarification -- the numbers on the bottom are the PWR ranking, not the number of comparisons won.  Being further left (closer to 1) is better.  That's why the far left curve is UND sweeping and the far right curve UND getting swept.

[sagard]

Nice analysis Jim.  For your prediction are you actually running all possible outcomes (skipping ties) of every other game and running the PWR for each or are you doing some expected results using KRACH or something similar?

Either way it's pretty impressive.

[Big A HG]

Looking at the DU series, your chart was pretty spot on.  Assuming that 3 points puts us right between a sweep and a split, it's right on 14.


So, there is a method to his madness.  

[AZSIOUX]

thanks jim, these charts are great!!

[jimdahl]

sagard, on Jan 29 2009, 11:57 AM, said:

Nice analysis Jim.  For your prediction are you actually running all possible outcomes (skipping ties) of every other game and running the PWR for each or are you doing some expected results using KRACH or something similar?

Either way it's pretty impressive.

The techniques are explained a little more deeply here and here, but in short:
* I run 1,000,000 trials of the set of games in interest (either the remaining season or, as above, the next weekend)
* Outcomes are either random (to simulate possible outcomes) or KRACH-based (to simulate likely outcomes)
* Calculate the PWR for each trial, save, and aggregate

Simulating every combination of outcomes would be tough.  With 49 games this weekend, that would be 2^49, or about 562,949,953,421,312 possibilities.  I've done some informal sensitivity analysis, and 1,000,000 trials does a pretty good job of producing consistent results.

For the chart embedded above, I use KRACH-based likely outcomes (hence no ties) with fixed UND outcomes because I think that's the best determinant of what UND's PWR is likely to be after the weekend.  I can also do the same thing for the entire season, as shown in the original blog post (the first "here" above).

The cool thing is, a single run produced similar data for every team, so once I make it an automated process that runs itself and produces a web page, I should be able to make a page for every team that includes: outlook for the team for the weekend and key non-team games this weekend (see my blog post for an example).

[AZSIOUX]

just pulling this from your blog as i know many wonder to who to cheer for , thanks again jim  

UND's PWR on 2009-Feb-01 based on outcomes vs. St. Cloud

Help for those who don’t like reading graphs:

Sioux sweep: 67% chance of landing at #12 or #13, 94% chance of landing between #11-#14.

Split: 77% chance of landing #14 or #15

Sioux swept: 60% chance of landing #17 or #18, 91% chance of landing between #16-#19.

Who else to cheer for?
Of teams playing just 1 game:

Vermont win over New Hampshire gives UND an average rise of .472

Of teams playing 2 games:

A Bemidji St sweep over Bob Morris gives UND an average rise of .607 (a single win gives UND a rise of .277)
An Alaska sweep over Ohio State gives UND an average rise of .518 (a single win gives UND a rise of .247)

[The Whistler]

Jim has just taken away the whole point of playing the games.  We'll just go by his simulator from now on.

[AZSIOUX]

The Whistler, on Jan 29 2009, 12:37 PM, said:

Jim has just taken away the whole point of playing the games.  We'll just go by his simulator from now on.

no, jim has just dumbed the pwr down for everyone    thank goodness, it really shows the importance of these late series before being played.

sioux sweep

[jimdahl]

The Whistler, on Jan 29 2009, 02:37 PM, said:

Jim has just taken away the whole point of playing the games.  We'll just go by his simulator from now on.

Your implied point that the games haven't been played yet and outcomes can be whatever the team makes them is an important one, and actually a key element of the forecasts.  That's why the output is multiple distributions of possible PWR outcomes instead of a single PWR.  They're not predictions of game outcomes, they're "what ifs" that forecast what various potential outcomes would do to PWR.

A simulation that predicted game outcomes by some existing ranking (e.g. KRACH) would produce a set of PWR rankings consistent with current KRACH expectations.  In early January KRACH predicted that UND would lose 3 of the 4 vs. Minnesota and Denver.  Those predictions were based on performance to date, and if true would have precluded UND from making the tournament (which a lot of people at the time were guessing was impossible for the Sioux, based on intuition instead of actual calculations).

Instead, I simulate across all the possible game outcomes to help answer the conditional questions (including those that deviate from current KRACH predictions) that people are continuously asking about PWR -- if UND defies expectations and sweeps the Gophers, what will that do to PWR?  If UND wins out 70%, can it climb to the top 12 in PWR?  If UND sweeps this weekend, what could its PWR be? etc...