Advisor tries to reproduce my results to ensure my honesty. Is it the norm?

    7
    118

    I am a M.Sc. student in Machine Learning, about to finish my thesis this month. My advisor wants me to meet him in person to reproduce my results under his control. It is obvious that the main reason for this action is to be confident that I didn’t manipulate the diagrams, etc.

    I didn’t do any cheating and I have no problem with that. But it is a little annoying for me that my advisor doesn’t trust me. I want to know whether this is the norm in academia or is my adviser just a little skeptical of me?

    I should mention that I am the lone M.Sc. student in the lab and I am nearly sure that the advisor doesn’t do that with PhD students in the lab.

    • Grzegorz Wierzowiecki

      I would say you are lucky!

      The main goal of science is to deliver something insightful to others!
      Therefore, when discovering something, make such results reproducible to them. Otherwise, it’s “an accident” that happened only to them. Science is about understanding reason and the cause, therefore about reproducibility of processes. As the goal is to make it reproducible by others, the best test is to try out if others can reproduce it.

      I am personally a great fan of “reproducible research” (there is a whole movement around this, just to mention: http://reproducibleresearch.net/ , https://reproduciblescience.org/ , and open science in general). Making sure that others can reproduce your effects, confirms that you’ve transmitted over paper all artifacts important to achieve your goal. Communication (also over papers), relies always on a big pile of assumptions. One might have to skip some details, steps, obvious to her/him in a paper, while actually not that obvious to others, then making whole work sadly lost and non useful to anyone else. Therefore, if someone with the skill/knowledge level of the desired audience is not able to reproduce it, maybe the paper should consist of more/better references, or more introduction to concepts/procedures/tools used. Again, all serves the purpose of transmitting over paper to others, to humanity, how to do something not done earlier, therefore to move the whole of humanity forward. Checking if it’s reproducible by others is the best way to check whether it is reproducible by others :).

      After all, mentioning in a paper, or notes for reviewers, that results were successfully reproduced by others, could IMHO even increase the value of the paper. Making notes (even blog post) by those others, would be even better to confirm.

    • Peutch

      While I agree with other commenters that it might be for other reasons, we cannot invalidate that this person might just be checking that you did not cheat. Think about the recent Michael LaCour scandal: he falsified his data, and his senior co-author (Green) did not catch him; this obviously raised some eyebrows.

      After that, I’m sure many senior academics have taken a closer look at their supervisees data/process.

    • mathreadler

      People willing to spend time on you is almost never bad. It proves that you or what you have done is important somehow. Try to unlearn that being checked is bad and try and view it as you are important enough for someone to care about. It is when no one cares any longer that you should be concerned, not the other way around ! This sure can be confusing for some personalities with high personal integrity.

    • Billy Jean

      In part, you are correct, but more than likely it’s a lot of other factors. Once you leave the gates of college, whether it’s your peers, your colleagues, or your professional community. If there is ever anything new or of significant value being claimed, than it MUST be reviewed. So definitely get used to people behaving skeptical initially.

      In part it’s to confirm your results. It can also be to potentially mimic results for future research or education. At the end of the day, his reputation lies on signing off on your work as “valid”. Or at least to make sure that what you did is not mimicked work with minute, insufficient contributions to your field. Plus, if there’s good stuff in your results, he might try and assist in finding you some serious grant funds or can help assist with private research funds.

    • Cliff AB

      In the field of machine learning, there is a plenty of ways to mistakenly do things incorrectly that make your results look better than they really are. An easy example is cross validation: if you do model selection based on the results of your validation dataset, you are going to be downward biasing the error in your validation results.

      In this light, your advisor is likely somewhat surprised at your results. Before just completely accepting what you’ve done, they want to verify this wasn’t a result of a mix up. If your advisor watches what you do, and everything is correct, then that’s great and you’ve probably got some great results on your hands. If you are doing something that’s not quite right, then your advisor will presumably help fix the error.

      You shouldn’t take this as criticism, but rather as being critical. Being critical is what makes science reliable and is vital to the academic process.

    • Peter K.

      There may be another reason your advisor wants to do this: perhaps the work can be continued in the future by another of their students, and your advisor wants to be able to explain the details to that (potential) future student.

      Regardless, I think it’s praise that your advisor is giving you.

    • DMlash

      As many have pointed out in the comments (Nate Eldredge, Dan Romik, Per Alexandersson, etc.), the point is not to catch you cheating (as if you were intentionally trying to manipulate your results), but rather to verify your process of obtaining such results. We all should be so lucky as to have an adviser take the time out of his/her schedule to verify our processes.

      I would also like to add that — and this assumes your thesis could/will lead to an academic publication at a journal or conference — that your adviser’s reputation is potentially on the line by attaching his/her name to such a document. In other words, by becoming a co-author on your [future] publication, your adviser is essentially saying, “Yes, I helped work on this, and I am sure of the methods and the results contained within.”

      The bottom line (tl;dr) is that you shouldn’t take this as a personal attack. You should be thankful to have someone who can invest the amount of time necessary to ensure the correctness of the work you’ve done.