Done with Economics Rankings!
https://drive.google.com/file/d/1j97BpC7Jc9-h7H6Y1wmSmLjeW9C9d3GL/view?usp=sharing
Thank you all for your input, without which creating rankings would have been impossible.
When I started the exercise, I had thought that devising the rankings would take me a maximum of two weeks. However, weeks passed and I ended up investing close to 3 months. Hopefully, next year it will be an easier exercise.
The rankings methodology changed significantly over the months. Initially, I had been assigning a fixed score or a rank-based penalty to postdocs, which would underestimate the schools with a higher postdoc %. Overall, I realized that the next-career move for postdocs needs to be considered. As I began to investigate the data, realized that a significant portion of the postdocs do not transition out of the role. Hence, it became imperative to correctly assign a score to those cases. There is an inherent bias in terms of recent cohorts having a higher share of non-transitioned cases. This implies that one needs to add a penalty to the final resolved placement for a given postdoc; where the penalty is dependent on the time they had spent as a postdoc.
But how to assign a score to the cases, where we don't have a next career move? For those cases, used a CatBoost model to estimate the expected scores using the transitioned postdoc data. All of these have been explained in detail in the methodology section.
Even though the modeling approach is a step in the right direction, there is scope for improvement. For instance, placement data is inherently biased (due to biased reporting); hence, one needs to tackle such cases. Currently, we use fixed-tier rankings for placements at Central Banks and International Organizations. Maybe more sophisticated approaches can be brought into the picture.
Initially, I would consider placements in think tanks and government agencies via a fixed tiered scoring method. However, I realized that government placements can be all over the place and not necessarily in research roles. Initially, I did try to segregate the research ones and assign a fixed-tier score. With the government fixed-tier score, the rankings seemed off.
Eventually, I had to drop rankings for international schools. I had spent a considerable amount of time to come up with an objective ranking. Unfortunately, even after 3 months, I could come up with no method that would objectively place Erasmus Rotterdam over the University of Amsterdam. I have no preference for any school, but quite a number of folks suggested that Erasmus is a better school compared to other schools among the Dutch programs.
Given that all the objective strategies fail to place Erasmus over other Dutch schools, I think there is something systematic that might be happening. I have looked at data countless times and yet the issue persisted.
I am more than happy to share the data for free (under a data sharing agreement) in case anyone wants to do research using them.
As PandaInUniv grows, you will continue to see richer and richer data. The data quality and standardization have improved significantly over the last 6 months. We will soon start reporting advisor names against doctoral students, suggesting an average advising quality based on placements.
The goal of PandaInUniv is to bring transparency to academia. For many reasons, there is an inherent opacity in academia, and our goal is to reduce that as much as possible. We also want to voice student concerns: in case you think you were discriminated against or harmed by university policies, please do reach out to us. We will give you a platform to raise your concerns (anonymously, if you prefer). For industry, we have glassdoor. Where is the equivalent of that in academia?
I plan to send the rankings to school admins on Monday. Hopefully, they get forwarded to you. I do hope you guys continue to support the endeavor. I can't emphasize enough how much your support means to me.