Teamwork in Contests


  • 4
    Y

    Prompted by other discussions on this forum and some surprising contest results I tried to investigate whether contest entrants are teams or individuals. One result is shown in the chart below,

    0_1490571419730_upload-65d31637-1596-439d-9a8f-010fd498d9f3

    This is a histogram from Contest 23 of the time taken to correctly submit Q1 and Q2.
    In orange bars are the times of for each entrant to answer the first question. For example 4 contestants submitted their first correct answer in the 6th minute.
    In grey bars is the time between each entrant's first and second correct submission, this is the time to answer the second question after doing the first question.

    For the first question there are no correct submissions in the first few minutes, then the rate builds up. For the second question there are many submissions starting from straight after the first question. In fact 73 second submissions take less than 5 minutes and only 2 first submissions take that time.
    Since Q1 is "Easy" and Q2 is "Medium" we could reasonably expect that the second question would on average take longer.
    A likely explanation is that the best performers are teams who can work on Q2 in parallel with Q1 and submit whichever question is completed first, which is often around the same time.

    It's important to be aware of this and individuals shoudn't get disheartened by their rankings, competing only against other individuals they would be higher.
    Also any employer using a contest ranking as a basis to interview or hire needs to be very careful.

    Very happy to hear comments.

    A bit more on the methodology (maybe boring for most so stop here and just take away the points above)

    • I also looked at contests 21, 22 and 24. There were always more second questions than first questions answered within 3 minutes. No first question was answered in less than 2 minutes.
    • Contest 23 is the best example and the shape of the graph above is different for the different contests but it always starts the same - more grey than orange.
    • The times exclude penalties for wrong submissions, they are just times to make a correct submission.
    • The total count of entrants who correctly submitted at least Q1 and Q2 was 872
    • First submission in 87% cases is Q1 but I allow "First qn" to be Q2 and and second Q1
      in the chart. Note that I only look at these, ignoring Q3 and Q4.

  • 0

    I'm repeating your analysis now and I think you have some mistakes. For example I find four people who solved their first question in the tenth minute: Mrsuyi, ycc0602, laughting and seven777.


  • 0
    Y

    @StefanPochmann yes you are correct, thanks for that I corrected my post above. Thankfully small changes and the broad picture is similar


  • 0

    Still wrong, now you're missing for example @sohelH solving the first problem in the fifth minute (at 0:04:55).

    Here are my charts for contests 21 to 24:

    0_1490440889690_contests_21_22_23_24.png

    But that gray "one minute from first to second" also includes cases like @zwu13 submitting first and second problem at 1:38:45 and 1:39:24 (and the third and fourth at 1:39:10 and 1:41:56). Surely that doesn't imply teamwork. So I did the analysis again, this time removing "first to second" times where the first time was over ten minutes. Looks a lot different:

    0_1490441875882_contests_21_22_23_24.png


  • 0

    To further analyze my remaining small gray bars with one or two minutes, here are the contest 21 to 24 times of the competitors having any "first to second" in one or two minutes where the first was in the first ten minutes.

    Edit: To clarify:

    1. I'm simply looking at the extreme cases here, and my point is that even these extreme cases don't look like teamwork to me. I'm not suggesting that these might be teams, I think the data rather suggests the opposite, that they're not teams.
    2. I chose two minutes as the cutoff purely because that gives me four people while choosing three minutes would give me 12 or 13 more people (from contest 22) and I didn't feel like doing that.
    3. The times in each row are sorted (so for example the first time in a row isn't necessarily for the first problem of the contest). I'll probably change this now...

    @awice Consistently submitting the first two "together" in the last three contests rather looks like a strategy.

    Contest 21    8:13 13:14 14:36 24:13
    Contest 22    5:41  5:53 21:38 23:22   12 seconds difference
    Contest 23   13:00 13:32 25:53 31:01   (32 seconds difference, but first was over 10 minutes)
    Contest 24   17:03 17:19 40:03 53:04   (16 seconds difference, but first was over 10 minutes)
    

    @cuiaoxiang

    Contest 21    3:44 12:24 15:35 36:48
    Contest 22    9:08 11:00 18:04 25:32   112 seconds difference
    Contest 23   16:12 19:47 29:06 42:12
    Contest 24    5:48 17:51 21:18 31:23
    

    @saffah

    Contest 21    3:12  8:08 11:27 17:56
    Contest 22    7:49  9:21 13:01 20:17   92 seconds difference
    Contest 23     -     -     -     -  
    Contest 24    5:18  5:49 13:28 18:38   31 seconds difference
    

    @y0105w49

    Contest 21    6:40  7:09 10:54 19:14   29 seconds difference
    Contest 22     -     -     -     -  
    Contest 23   14:21 14:46 21:33   -     (25 seconds difference, but first was over 10 minutes)
    Contest 24    2:30  6:20 15:14 22:18
    

  • 1

    @StefanPochmann

    I suggest you taking the following facts into your analysis:

    1. It is a common strategy to open all the problem links when they are available at the beginning of the contests to save the page loading time; and people might use the strategy to glance through the first few problems if the statement is short, which means it is possible that they could already have the implementation details in mind for problem 1 and 2 before actually writing code.
    2. Due to network issues the contest page isn't available even 10 mins after contest starts e.g. in Weekly Contest 23, my first problem was submitted in 16:12, while the second problem was submitted in 19:47. This is because I can't see the problem until 13 mins after the contest begins.
    3. Looking at my codes for Contest 22, problem 2, I don't see why it's get into your radar for teamwork. Seriously, I can type that code snippet in 30 seconds if I have the implementation in mind.

    BTW: my strategy is opening all problem links at the beginning. Read problem 1, solve problem 1; Read problem 2, solve problem 2; ...


  • 0

    @cuiaoxiang said in Teamwork in Contests:

    I don't see why it's get into your radar for teamwork

    Not sure what you mean with "radar for teamwork". I wouldn't call it that. I just took a deeper look at what @yorkshire showed and try to find explanations. So thanks for providing more explanations.

    You're in my analysis because you're one of the extreme cases as described. And my point is that even looking at these extreme cases, it doesn't look like teamwork to me.

    Typing that solution in 30 seconds is impressive. I tried a few times and need about 60 to 70 seconds now. But I'm not used to typing C++ and I'm not the fastest typer anyway (but I can type my Python solution in 16 seconds :-). Do you just type that fast, or do you get help from an editor or so? (Would be totally fine in my opinion, I'm just curious.)

    How big is page loading time for you? I never thought of that as a factor, it's fast for me (though I can't participate in the contests, so I don't know about loading times there).


  • 1

    @StefanPochmann
    I wouldn't chime in this thread if I didn't get named.

    Speaking of the extreme case, it could be more scientific. Take Contest 22 problem 2 as an example, I guess my 112 seconds performance is above the set bar (sorry for the radar metaphor if that sounds offending) for suspicious teamwork in your analysis. So let's take a look at how suspicious it is when comparing horizontally with other contestants.

    1 0:07:49 0:09:21 0:13:01 0:20:17 (1min32sec)
    2 0:05:52 0:08:34 0:17:45 0:27:35 (2min42sec)
    3 0:04:59 1 0:07:17 0:12:59 0:19:19 (2min18sec)
    4 0:08:19 0:11:34 0:19:28 0:30:02 (3min15sec)
    5 0:04:18 0:07:27 0:22:46 0:31:18 (3min9sec)
    6 0:05:51 0:08:40 0:15:39 0:21:25 1 (2min49sec)
    7 0:08:39 1 0:03:18 0:21:37 0:13:26 (3min18sec)
    8 0:03:43 0:05:45 0:34:43 0:17:11 (2min2sec)
    9 0:09:08 1 0:11:00 0:18:04 0:25:32 (1min52sec) <--- me
    10 0:05:23 0:07:31 0:17:08 1 0:25:43 (2min8sec)

    My point is if a solution takes significantly less time than even the top contestants, that's a more convincing evidence for teamwork. I would not post user IDs in my thread, that's rude. I would directly click Report Cheating if I strongly suspect someone is cheating in the match.

    I type in web pages.

    Page loading time is an issue for me only once, usually it loads in 30 sec, but not instantly. It's true that at the beginning of the contest, it takes some time to load considering so many people fire requests at the same time.


  • 1

    I do not work as a team. The reason I submit staggered sometimes is because I am passively thinking about my answer and whether I want to risk the 10m penalty while solving other problems. I have been burned so many times by negative input, fractional input, null input, and other corner cases. If "run code" data was available instead of just final submission, you would see something like this, for example:

    [working on Q1]
    6:00 Q1 run code
    6:30 Q1 run code
    [working on Q2]
    12:00 Q2 run code
    12:30 Q2 run code
    12:40 Q2 submit
    12:45 Q1 submit

    This is available to admins so they can easily verify this. I have no other way to my knowledge, to conclusively prove I don't cheat.


  • 1

    @cuiaoxiang Yes, as described I picked two minutes as the bar. The reason was that with three minutes, I would've had to analyze 12 or 13 more people, and I didn't feel like doing that. I felt four people would be enough. And yes, while I already analyzed more deeply than @yorkshire, one can of course look even more deeply. Thank you for doing that and pointing out that the top ten competitors all had similarly fast times for that problem. Sorry if you feel that it was rude to post user IDs. I don't think it was. I think it just makes communication easier. Not sure why you're telling me what you would do if you strongly suspect someone is cheating. I don't suspect anyone of cheating, let alone strongly. As I already explicitly said, it doesn't look like teamwork to me. But I'll add another note up there to prevent confusion. I was hoping the data would speak for itself, but apparently it doesn't.

    @awice Thanks for your response as well, good to see further explanation. As I already said, what you were doing rather looked like a strategy to me (instead of teamwork). Partly because your overall statistics don't look suspicious to me, but mostly because I already knew you from many good posts from you on this forum. I had actually thought you maybe held back the submission in order to not give others a clue how easy the first problem is, but your actual reason makes even more sense.


  • 0
    Y

    @StefanPochmann amended (again) thanks


Log in to reply
 

Looks like your connection to LeetCode Discuss was lost, please wait while we try to reconnect.