MATLAB Central Spring Contest
in [Matlab]
|
From: OkinawaDolphin on 7 May 2008 05:31 > The most obvious sign for tweaking is the introduction of > randomness into the algorithm. Not only that parameters of > the code are varied, we even don't know which parameters > that are! In my opinion the use of random generators (rand, > randn, randperm etc.) should be forbidden. > > Any other thoughts? If randomness is forbidden, genetic algorithms, simulated annealing or even a random walk are forbidden, too. Only deterministic algorithms are allowed in this case, no matter how inefficient they might be. Tweaking can be prevented by removing or deactivating the functionality for reading and editing existing code. However, tweaking is allowed in this contest and it is done by people who understand the code others wrote.
From: Matt Butts on 7 May 2008 07:54 Contest ends at 12PM Eastern today, correct?
From: Nathan on 7 May 2008 08:10 Hi all In my opinion, wholesale tweak-bombing, on a scale large enough to monopolise the queue for hours, is contrary to the spirit of the contest. Arguably, it's a form of hacking - "Entries that compromise the contest machinery are no longer allowed". There is probably no technical solution that would prevent it (and anyway it's unfair to expect the Mathworks team to work as enforcers). However, I agree with Markus that a captcha would be a workable and useful measure. In principle, I would like to see random functions outlawed - however, competitors could easily circumvent it with handwritten random number generators (they wouldn't have to very good ones) which would lead to less intelligible code. In general I'm in favour of a light touch with rules and regulations. New restrictions will act as targets to be hacked and evaded. Fun and a social spirit should stay at the centre of the contest. Nathan PS Although Alan defeated the online diff function, CSdiff on windows (or presumably a unix diff) will reveal what he is up to :) "Markus Buehren" <mb_matlab.REMOVE(a)gmxTHIS.de> wrote in message <fvrot9$gsi$1(a)fred.mathworks.com>... > Alans bombing as well as his analysis both speak for adding > a "captcha" to the submission site (which I think is even > better than an IP limit). This would prevent the > over-over-over-specification of the solver to the test suite > which occurs due to the many parameter variations that are > submitted. > > I have found some algorithmic improvement yesterday, which > dramatically boosted the score on the test suite we are > having at home. However, as the leading code is so > over-specified to the actual test suite, my change even lead > to a worse score on it. I think the parameter tweaking by > numerous automated uploads takes away the room for > algorithmic improvements. > > The most obvious sign for tweaking is the introduction of > randomness into the algorithm. Not only that parameters of > the code are varied, we even don't know which parameters > that are! In my opinion the use of random generators (rand, > randn, randperm etc.) should be forbidden. > > Any other thoughts? > > Markus >
From: Markus Buehren on 7 May 2008 10:13 > If randomness is forbidden, genetic algorithms, simulated > annealing or even a random walk are forbidden, too. Only > deterministic algorithms are allowed in this case, no > matter how inefficient they might be. In my view, you compare two different things here: Genetic algorithms and such are tools for finding an optimal parameter set. The solver algorithm itself is then used for solving the problem, using the parameters found in a step before. If you look at the entries using randomness in this contest, there is not the slightest sign that something like a genetic algorithm is applied there. > Tweaking can be prevented by removing or deactivating the > functionality for reading and editing existing code. No, there are simple ways to upload submissions automatically. You can even do it simply from within Matlab! I know which Matlab function one can use for it, but I won't tell it to anyone now :-) > However, tweaking is allowed in this contest and it is done > by people who understand the code others wrote. No!! For tweaking, you don't need any clue about how the code itself works! Just play around a bit with the parameters, let one solver run more than once and select the best result, or combine two different solvers and choose the better result and sometimes you will find an improvement. You don't need to understand the algorithms themselfes for that. Markus
From: Luigi Sorbara on 7 May 2008 10:18
Thank you Nathan, well said. I would assume the majority of people participate in this contest out of interest and challenge (and maybe the opportunity to work for matlab :) ). I have limited time in the evening once I get home from work and do my husbandly chores .. to actually participate. There is nothing more frustrating then waiting over 40 minutes to see how a submission faired whilst looking at hundreds of tweak bomb entries. Helen .. any chance we can extend the contest one more night for us true lovers of the game :) "Nathan " <nathoqXXX(a)yahooXXX.com> wrote in message <fvs66t$8l4$1(a)fred.mathworks.com>... > Hi all > > In my opinion, wholesale tweak-bombing, on a scale large > enough to monopolise the queue for hours, is contrary to the > spirit of the contest. Arguably, it's a form of hacking - > "Entries that compromise the contest machinery are no longer > allowed". There is probably no technical solution that would > prevent it (and anyway it's unfair to expect the Mathworks > team to work as enforcers). However, I agree with Markus > that a captcha would be a workable and useful measure. > > In principle, I would like to see random functions outlawed > - however, competitors could easily circumvent it with > handwritten random number generators (they wouldn't have to > very good ones) which would lead to less intelligible code. > > In general I'm in favour of a light touch with rules and > regulations. New restrictions will act as targets to be > hacked and evaded. Fun and a social spirit should stay at > the centre of the contest. > > Nathan > > PS Although Alan defeated the online diff function, CSdiff > on windows (or presumably a unix diff) will reveal what he > is up to :) > > "Markus Buehren" <mb_matlab.REMOVE(a)gmxTHIS.de> wrote in > message <fvrot9$gsi$1(a)fred.mathworks.com>... > > Alans bombing as well as his analysis both speak for adding > > a "captcha" to the submission site (which I think is even > > better than an IP limit). This would prevent the > > over-over-over-specification of the solver to the test suite > > which occurs due to the many parameter variations that are > > submitted. > > > > I have found some algorithmic improvement yesterday, which > > dramatically boosted the score on the test suite we are > > having at home. However, as the leading code is so > > over-specified to the actual test suite, my change even lead > > to a worse score on it. I think the parameter tweaking by > > numerous automated uploads takes away the room for > > algorithmic improvements. > > > > The most obvious sign for tweaking is the introduction of > > randomness into the algorithm. Not only that parameters of > > the code are varied, we even don't know which parameters > > that are! In my opinion the use of random generators (rand, > > randn, randperm etc.) should be forbidden. > > > > Any other thoughts? > > > > Markus > > > |