How are ranks calculated?

nbarriga · Post by **nbarriga** » Fri Jul 20, 2007 7:44 pm

How are the ranks calculated? and how do you decide the pairings? and how often do the bots play? and is there a way to submit a totally different bot, and to reset the record?

Maybe it would be usefull to have a table with the individual scores by each pair of bots.

Sorry for the amount of questions

Post by **adum** » Fri Jul 20, 2007 10:10 pm

check out the post in the General section of the bots in the forum to see how they're currently ranked.

bots continually play, but only one matchup is going on at a time. if you wait a bit, your bot will start playing. soon i'm going to have live linking to all recorded matches for replaying etc.

i'm considering wiping the win/loss record on a new submission. probably makes sense.

questions are welcome =)

adum

Captain Segfault · Post by **Captain Segfault** » Wed Jul 25, 2007 6:25 am

adum wrote:i'm considering wiping the win/loss record on a new submission. probably makes sense.

Ideally the wipe ought to include the match currently in progress.

ShardFire · Post by **ShardFire** » Thu Jul 26, 2007 6:44 pm

Also, when you upload a bot that is completely different with different name, different AI, clicking on the bot history still shows matches from your previous uploaded bot whatever it was, and with the name of the new bot...

Post by **adum** » Thu Jul 26, 2007 8:28 pm

Also, when you upload a bot that is completely different with different name, different AI, clicking on the bot history still shows matches from your previous uploaded bot whatever it was, and with the name of the new bot...

yeah, i wasn't sure about the best thing to do here. if the change is minor, it's useful to be able to see the old matches still. if it's major, then it's probably better not to. you can always compare dates. i'll probably lean towards wiping the old stuff.

nbarriga · Post by **nbarriga** » Fri Jul 27, 2007 12:42 pm

I saw that you lowered the 15 games matches to 5 games matches and now 1 game matches. Don't you think it's too little? there will probably be too much mobility in the ranks, since most bots have a random part(I guess). I personnally found 5 games to be too little, but I think 15 games wasn't feasable once the amount of bots started to increase.

Post by **adum** » Fri Jul 27, 2007 4:29 pm

the rankings have now moved to an Elo rating system, so everything is one-game matches but no longer in a ladder format. this is leading to much improved results and feedback, and is much faster and more accurate.

adum

ShardFire · Post by **ShardFire** » Sat Jul 28, 2007 1:27 am

adum wrote:yeah, i wasn't sure about the best thing to do here. if the change is minor, it's useful to be able to see the old matches still. if it's major, then it's probably better not to. you can always compare dates. i'll probably lean towards wiping the old stuff.

I thought it would be as simple as - bot has the same name, keep the old matches (meaning bot has just gone up a version); bot has a different name, delete the old matches. (presumably signifying significant changes)

I don't have a problem with using Elo ratings, but I want to be able to access more than 1 game per opponent for my current bot, especially if I am prototyping it! The more of your lost games you get to see the easier it is to see what is wrong

Captain Segfault · Post by **Captain Segfault** » Sat Jul 28, 2007 5:12 am

adum wrote: more accurate.

I'd have to disagree; my previous SegBot could do 8/15 on just about any of them, but gets penalized a *lot* because it happens to lose to IGotsADD and their ilk once in a while.

I downgraded to my weaker version that is more consistent but less intelligent.

Post by **adum** » Sat Jul 28, 2007 6:20 am

I don't have a problem with using Elo ratings, but I want to be able to access more than 1 game per opponent for my current bot, especially if I am prototyping it! The more of your lost games you get to see the easier it is to see what is wrong

the idea is you should be able to access perhaps 50 of the last games against any opponent, not just 1. i'm phasing that in now, i hope.

I'd have to disagree; my previous SegBot could do 8/15 on just about any of them, but gets penalized a *lot* because it happens to lose to IGotsADD and their ilk once in a while.

i guess it depends how you define accurate then... but overall i think this is a big improvement. before there would be bots who would stay in weird rankings just due to how they played against certain bots next to them, for example.

15 games was also way many because with captain segfault's bot maxing out the instructions, a game could take 20 minutes. 15 of these, and it's a long time between matches.

now i'm getting the problem of segfault's new bot eating up all the heap space...

adum

adum

Post by **adum** » Sat Jul 28, 2007 6:48 am

hmmn, fixed the memory leak, but captain, as a head's up, your bot is taking more than 500k instructions.

adum

Captain Segfault · Post by **Captain Segfault** » Sat Jul 28, 2007 8:11 am

adum wrote: the idea is you should be able to access perhaps 50 of the last games against any opponent, not just 1. i'm phasing that in now, i hope.

Yay!

adum wrote: now i'm getting the problem of segfault's new bot eating up all the heap space...

Odd... the new bot was just a slightly fixed up version of one of my older ones. I compiled it with jikes instead of javac though, maybe that made a difference.

Captain Segfault · Post by **Captain Segfault** » Sat Jul 28, 2007 8:48 am

adum wrote:hmmn, fixed the memory leak, but captain, as a head's up, your bot is taking more than 500k instructions.

Oh, wow, that was stupid, though I didn't figure out the bug until after I'd obsoleted it.

Captain Segfault · Post by **Captain Segfault** » Sat Jul 28, 2007 9:12 am

adum wrote: 15 games was also way many because with captain segfault's bot maxing out the instructions, a game could take 20 minutes. 15 of these, and it's a long time between matches.

Fixed.

It wasn't the 500K instructions, but rather the int[dx*dy*50][3] array I lazily allocated. Changing "50" to "2" makes a match vs itself only take about 30 seconds, down from at least 10 minutes.