The Great Change: Turning Cathy into a Lawyer

I dunno...that seems like a fair statement. Of course it doesn't take into account other possibly known factors - maybe it was always tested when Mr. Smith determined the questions, but never when Mr. Jones or Mr. Wilson put the test together. Maybe it was tested 9 out of 10 times three years ago, and only once since then. Maybe it was tested every third exam for awhile, and more recently every 5th exam (I'm too lazy to do the math that makes that work out to 10 of 45). And so on. It isn't a truly random occurrence, so probability is not really the right concept for it - if you want to game the system like that, you need a LOT more intimate details on how it is put together. But with no other known data, yeah, you've got a 22% chance of needing to know that subject.

Well, yes, as you say, "probability" is probably not what's governing it. Most likely it's part of some strategic formula by the bar examiners, a formula we are not able to adduce from the "10-times in 45 sessions" statistic, especially since they have recently removed certain subjects (thus shrinking the pool of possible ones to use and making it more likely that they will use any of the remaining ones) and that they made no mention of whether there has been an increase or decrease in the frequency within the last 45 sessions (so, for instance, if it had come up once every 10 tests and now comes up on every test, you could still have your 10-in-45 statistic, but it wouldn't be very meaningful).

But even if the subject frequency were subject to random chance, the way to measure its probability is not by how many times it was observed in the past but by how likely it will be to be observed again. It's like flipping a coin. On every coin flip there is a 1-in-2 chance of coming up heads (two sides to a coin, and it will always land on one of them). But it is possible that after 10 flips you might have seen heads 7 times. That's possible, because each flip is its own independent chance to come up heads or not. Whether it had come up heads before has absolutely no bearing on whether it will come up heads again. But if you based your probability calculations on what you had observed, you'd think there'd be a 7-in-10 chance of a coin coming up heads, which is obviously not the case.

In this case, if there are, say, 15 possible subjects that can appear on the test and on every test there are 5 chosen, then the likelihood that this one will be chosen is 5-out-of-15, or 1-in-3. I don't actually know how many subjects are chosen (and it's also possible that there are different tiers of subjects, with some always being on the test and some only sometimes being on the test, which would change the likelihood of those latter subjects), but the point is that if this subject's likelihood works out to a 1-in-4.5 chance of being on this test it's because of pure coincidence, and not at all based on the rationale he gave.

In any case, either way you slice it, what he said was technically wrong, which was my point ;-)