Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Overview of U.S. Science and Engineering Ph.D. Program Requirements

Summary
This article is one particular attempt to summarize what it takes to earn a Ph.D. degree. It might be helpful for people who are deciding whether to pursue a Ph.D. Specifically, I describe what it takes to earn a Ph.D. in computer science from Stanford University circa 2012. Ph.D. students in different universities, majors, and time periods will have drastically different experiences.

Everyone knows the rules of the “academic game” for high school, college, and almost all types of graduate schools such as law, business, or medical school. You “win” (get a degree) when you pass all of the required classes. And how do you pass those classes? Turn in your homework, study, pass exams, and maybe do a final term project. Every school has its own fixed class requirements, and as long as you pass those, then you get your degree.

The Ph.D. is the only type of graduate school where the exact rules of “the game” are ambiguous and often unstated. In general, you earn a Ph.D. as soon as you can convince a group of three to five professors in your department to grant you a Ph.D. degree. That's all it takes; every other requirement (e.g., classes, qualification exams, thesis proposals) is insignificant.

So how do you convince three to five professors to grant you a degree? By demonstrating that you have made an original contribution to academic research in a specific field of study.

What constitutes original academic research, and how do you convince your professors that you have done enough to graduate? The exact criteria vary widely based on your field of study, your university, and even the time period when you are attending school.

This article describes my own experiences of getting a Ph.D. in computer science from Stanford (a well-known U.S. university) from 2006 to 2012. Thus, the less your Ph.D. program has in common with mine in terms of field, university type, or time period, the less relevant this chapter will be to you. For example, if you are getting a humanities Ph.D. from Europe, then the requirements are probably very different.

To earn a computer science Ph.D. from Stanford circa 2012, you need to either:

  • publish 2 first-author papers in a top-tier conference,

  • or publish 3 to 4 first-author papers in a second-tier conference,

  • or publish some roughly equivalent mix of top-tier and second-tier papers (e.g., 1 top-tier and 2 second-tier).

Note that these are estimates based on observation; there is no official department paper quota required for graduation. Some of my fellow students have graduated with fewer published papers. And some other schools and departments let their students graduate without publishing at all. However, the more papers you publish, the easier it is to convince your professors to let you graduate. (In the next section, I will define what I mean by publish, first-author, and conference.)

As soon as you publish “enough” papers, the three to five professors on your thesis committee will give you permission to combine your papers together into a book-length document called a dissertation. Then they will sign the proper forms and grant you a Ph.D. degree.

You get to choose all of the members of your thesis committee, so a strategic choice can help you graduate more easily. The most important professor on your committee is your Ph.D. advisor, who usually works with you to formulate research ideas and to write papers.

The published papers that comprise your dissertation must be related to a coherent theme, although there is significant leeway as to what “coherent” means. The papers need to be closely related enough so that they make sense as chapters in a technical book (your dissertation). For reference, I graduated after publishing 1 top-tier conference paper and 3 second-tier conference papers that were somewhat related to one another.

How long does it take to earn a Ph.D.? In my department, most students get their degree in five to seven years (I graduated in six years). However, there is no upper limit; some students can take eight or ten years to graduate, and others never graduate.

The faster you can publish “enough” papers for your dissertation, the faster you can graduate. Most students in my department publish the first paper of their dissertation during their fourth year. The first three years are usually spent taking classes and learning to do research by helping out on other people's projects.

The Publication Game

By now, I've made it clear that you probably need to publish papers in order to earn your Ph.D. Specifically, in my department, you must publish a few first-author papers in top-tier or second-tier conferences. I will now define these terms in detail.

Peer-reviewed academic publishing

First, what does it mean to publish an academic paper? In general, you do some original piece of research, write up your ideas and results in a paper, and then submit your paper for review. (The exact format of an academic paper varies widely by field, and even by sub-field.) Your paper gets peer-reviewed by three to five volunteer experts—usually professors or research scientists—who critique its merit. Researchers volunteer to peer-review papers because it's part of their professional service duties. In theory, these reviewers are your “peers,” but in reality, they are much more senior than you. If the reviewers deem your paper worthy of publication, then it gets published. If not, then you must revise your paper and resubmit it for review at a later date.

The purpose of peer review is to ensure that all published papers are up to a certain level of acceptable quality, as determined by the scholarly community. Thus, from a practical standpoint, publishable academic research is simply what your senior colleagues deem worthy of publication, since they are the people who review your papers. This is a simple but very important point! The word “research” colloquially means creating or discovering something new in the world. However, for your work to count as publishable academic research, your senior colleagues must say that it counts. This vetting process is imperfect but absolutely necessary, since there needs to be some arbiters of “goodness” to filter out crackpot claims.

I've seen many Ph.D. students get disillusioned when their supposedly ultra-creative, ingenious, off-the-wall ideas cannot get published (and hence they cannot graduate). What I tell them is that the “Ph.D. game” is not about being brilliant; it's all about being able to publish! To earn a Ph.D., you must do the type of research that your senior colleagues approve for publication.

A professor once said, “Academic research is like pornography. It's hard to define, but you know it when you see it.” Thus, the best way to get a sense of what is considered acceptable academic research in a particular field is to read a few dozen published peer-reviewed papers in that field.

What is considered acceptable or “good” research also varies over time. For example, the kinds of work that were deemed publishable in the 1970s are probably not publishable today, because the field has advanced and a new generation of Ph.D.s are now serving as paper reviewers. Similarly, the kinds of media deemed as pornographic also varies by geographical location and time period. The same is true for all other creative endeavors—e.g., good art is what the establishment of current artists and critics deem as good, and that varies by location and time.

Computer science conference publishing

In modern-day computer science research, the primary venue for publication is the conference. In most other science and engineering fields, the journal is the primary venue, and the word conference means something quite different. Also, academic conferences are very different than industry conferences or corporate trade shows.

Each computer science conference is usually held once per year and focuses on a particular sub-field. Here is how the conference publishing process works:

  • The conference issues a call for papers with a specific due date (e.g., March 1st of each year).

  • Researchers write up and submit their papers by the deadline (e.g., March 1st) using an online form. Each submitted paper contains roughly 30 to 40 pages of double-spaced 12-point font text. A conference typically receives 100 to 300 paper submissions.

  • The conference program committee (PC), consisting of 10 to 20 volunteer researchers in the given sub-field, splits up the submitted papers and reviews them. Each paper is reviewed by three to five people, who are either PC members or volunteer external reviewers solicited by PC members. This process takes about three months.

  • After everyone on the PC is done with their reviews, a PC meeting is held to discuss all paper reviews. The PC then decides which papers to accept and which to reject.

  • The PC emails all authors to notify them of whether their papers have been accepted or rejected, and attaches the reviews to the notification emails. For a conference with a March 1st submission deadline, authors will probably be notified by June 1st.

  • The conference event occurs a few months after acceptances have been sent. Authors of accepted papers attend the conference to give a 30-minute talk on their paper. All accepted papers are published in a book (now usually online) called the conference proceedings.

  • Authors of rejected papers may revise and resubmit their paper to another conference, or to the same conference in the subsequent year.

Rejections, resubmissions, and re-rejections

Computer science conferences vary in reputation. Each sub-field usually contains two top-tier conferences, and three or four second-tier conferences; everyone in the sub-field knows what those are, just like how everyone knows which consumer brands are top-tier or second-tier. The most visible difference between top-tier and second-tier conferences is their selectivity:

  • Top-tier conferences usually accept 8 to 16 percent of submitted papers.

  • Second-tier conferences usually accept 20 to 30 percent of submitted papers.

When your paper is under review for one conference, you are not allowed to submit it to another conference. Therefore, since reviews usually take three months to complete, you can submit a paper at most four times per year. But since conference submission deadlines don't perfectly align, you can realistically only submit a paper three times per year.

Let's now do some math. For simplicity, assume that ...

  • after you start a new research project, it takes 1 year to get enough results so that you can write a respectable paper submission to a conference in the relevant sub-field.

  • your research is fairly good but not spectacular, so your paper's chances of acceptance is 20% for a top-tier conference and 40% for second-tier.

  • it takes 4 months to receive your paper reviews and revise accordingly, so you can only submit your paper 3 times per year.

Let's now estimate how long it takes for your paper to be accepted for publication:

  • You first submit to a top-tier conference since you are ambitious. With 20% probability, your paper is accepted to this conference, which occurs 1 year, 4 months from when your project began (1 year of work time + 4 months of review time). However, with 80% probability, your paper will be rejected.

  • Let's say your paper gets rejected. You read the reviews, revise your paper, and resubmit it to another top-tier conference. Again, your paper has a 20% chance of acceptance. So with 36% probability, your paper is accepted within 1 year, 8 months of project inception time (20% + (80% * 20%) = 36%).

  • If your paper gets rejected again, then you revise and resubmit. But you're tired of getting rejected, so you aim lower for a second-tier conference with an estimated 40% chance of acceptance. So with 62% probability, your paper is accepted within 2 years from project inception time (20% + (80% * 20%) + (80% * 80% * 40%) = 62%).

  • Now if your paper gets rejected yet again, so you resubmit to the next second-tier conference. With 77% probability, your paper is accepted within 2 years, 4 months from project inception time. (Just trust my math!)

  • Now if it gets rejected again and you resubmit, then with 86% probability, your paper is accepted within 2 years, 8 months.

  • Let's try once more. With 92% probability, your paper is accepted within 3 years.

The purpose of these estimates is to show how it often takes longer for a paper to get published than the amount of the time you originally spent working on the project! According to these numbers, if you spent a year working on a project, there is a 38 percent probability of taking more than two years before your paper gets accepted (100% – 62% of acceptance within 2 years). And there is a small chance that it will never get accepted.

Student publication challenges

The above odds are generous estimates, because they assume that your research is good enough to be competitive at the professional level, and that you are submitting each paper three times per year. As a Ph.D. student, your paper submissions are competing directly against those written by professors and entire teams of industry/government research scientists with far more experience than you. It's not as though you are submitting papers to student-only conferences. In contrast, for all other types of school, you are competing against only your fellow students to pass your classes, not against professionals who have already graduated.

Since conference acceptance rates are so low, it's common for computer science Ph.D. students to have each paper be rejected three, four, five, or even six times before it gets published. Rejection notices are some of the most demoralizing and painful experiences for Ph.D. students. Many students are deeply frustrated by the fact that long after a project is already done, they still need to spend significant amounts of time rewriting and resubmitting their paper with the hope of getting more favorable reviews the next time around.

Because each conference has a different set of reviewers, each with their own human biases, you often need to reframe and reword your paper submissions to suit reviewer tastes, with no clear idea of whether you've increased or decreased its chances of acceptance. Sometimes submitting to a conference in a different sub-field can improve your paper's chances of acceptance if your topic or style is more “in vogue” in that sub-field. These marketing-related activities have nothing to do with the actual content of your research, but you must excel at this sort of “salesmanship” if you want to publish and eventually graduate.

A skeptical reader might be wondering: Surely reviews are not so arbitrary. If you do good work, then your paper will definitely be accepted! After all, isn't academia a pure meritocracy?

In 2009, Thomas Anderson, a well-respected senior professor at the University of Washington, studied reviewing trends at top-tier computer science conferences and wrote up his findings in a paper entitled “Conference Reviewing Considered Harmful” (View PDF). This paper quantitatively confirmed what many researchers already know from experience: that despite the sincere best efforts of reviewers, it's almost impossible to clearly differentiate the majority of submitted papers into accepts or rejects. A tiny fraction of submissions to every conference are so strong that they will definitely be accepted, and a tiny fraction are so terrible that they will definitely be rejected. But for the vast majority of papers, the decision to accept or reject is effectively random, since they are so close in quality that reviewers cannot possibly make an objective clear-cut distinction between accept and reject.

Thus, it's not uncommon for a paper to be rejected from one top-tier conference only to be accepted with spectacular reviews at another top-tier conference (often with almost no changes to the paper's contents). Given this empirical evidence, my simple model of paper submissions having a fixed percentage chance of acceptance is quite realistic.

This now begs the question: Why are acceptance rates so low? Why don't computer science conference program committees simply accept more papers? I don't have enough experience in academia to know the true answers; all I know is that these are the current rules of the “publication game,” so Ph.D. students must play by those rules.

Ph.D. students in my department who want to graduate in six years need to start submitting first-author conference papers by their third or fourth years. It usually takes at least three years before students can formulate an original research project, get meaningful results, and write a paper that has a reasonable chance of acceptance.

Also, students need to work on several projects concurrently, always preparing a new paper while waiting for reviews on older paper submissions. Without wise planning of submissions and inevitable resubmissions, it's difficult to publish the 2 to 4 conference papers that are implicitly required to graduate.

I didn't submit my first dissertation-worthy paper until the beginning of my fifth year, so I was very worried that it would take me seven or eight years to graduate. Thankfully, I did some meticulous planning and had great luck during my fifth and sixth years, so I published enough papers to graduate by the end of six years.

Paper author order

The final publishing-related issue I want to mention is author order. Recall that only first-author papers help you to graduate. Each paper usually has multiple authors, and the order in which authors are listed actually matters. The author whose name appears first is the project leader who usually does far more work than all subsequent listed authors. All other authors are project assistants—usually younger students or distant colleagues—who contributed enough to warrant their names being on the paper. Ph.D. students often list their advisor as the last author, since the advisor helps with idea formulation, project planning, and paper writing.

Papers where you are not listed as the first author usually don't help you to graduate, but they can still be valuable learning experiences. In fact, the best way for early-stage Ph.D. students to prepare for their dissertation is to help out on senior students' or professors' projects. During my first three years of Ph.D., my name appeared on three papers where I wasn't the first author. Those experiences taught me a great deal about doing academic research and about the “publication game.”

Funding

Now that I've covered what is required to get a Ph.D. in my field, I want to discuss money. Who pays for your education? Unlike college and all other types of graduate schools, you do not pay to attend a Ph.D. program. Instead, you get paid a small salary (called a stipend) while working on your Ph.D. Why? Because you are working a full-time job to produce publishable research that can benefit your advisor and university in terms of academic prestige (and sometimes intellectual property such as patents). Your stipend ranges from $15,000 to $30,000 per year in U.S schools, which usually pays for your living expenses but not much else. In sum, for most types of school, you are paying tuition to consume knowledge, but for a Ph.D. program, you are being paid a stipend to produce new knowledge.

Funding typically comes from a combination of these sources:

  • Department: Your department usually funds first-year Ph.D. students who have not yet found an advisor.

  • Grant money: If your advisor obtains research grant money from government funding agencies (and sometimes from companies), then those grants can pay for you. The main downside of grant funding is that your project needs to be on a topic that is related to the type of research that the grant is intended to fund. Grants are the most common source of Ph.D. student funding in my department.

  • Teaching: You can be paid for being a teaching assistant for classes in your university. The main downside of teaching is that it can take a lot of time away from your research. Depending on the class, teaching duties can take anywhere from 10 to 25 hours per week.

  • Fellowship: The best source of funding is a fellowship, which is a scholarship that funds a few years of your Ph.D. studies with no strings attached. Fellowships are extremely hard to win, with acceptance rates around 3 to 10 percent. If you can win a fellowship, then you have much more freedom in choosing your own research projects, since you are not bound by a specific grant.

I was fortunate enough to win two fellowships: the NSF (National Science Foundation) and NDSEG (National Defense Science and Engineering) graduate fellowships. When combined, these two fellowships paid for five years of my Ph.D. studies. My sixth year was paid for by a combination of teaching and my advisor's grant money. Most fellowship winners get only two to three years of funding, so I was very lucky to have received five years. Since I was funded primarily by fellowships, my research agenda wasn't bound by grant restrictions, and I didn't have to teach many classes.

The majority of students in my department are funded by a combination of advisor grants and teaching. The students whose advisors don't have enough grant money need to teach more classes, which sometimes slows down their research progress and delays their graduation.

All of the above funding sources pays not only your stipend, but also your university tuition (since you are using up university resources as a student). Of course, you can self-fund your Ph.D. years by directly paying tuition, but almost nobody does this since it makes no economic sense. The amount of extra money you can earn with a Ph.D. degree will never make up for the ~$150,000 of debt you accumulate by paying your own tuition.

Errands

Throughout your Ph.D. years, you will often need to do tasks that don't contribute towards helping you to graduate. I will refer to all of these tasks collectively as errands. Here are some typical kinds of errands for Ph.D. students:

  • Buying, installing, maintaining, and answering questions about lab equipment necessary for research (e.g., computers, software, laboratory beakers, chemicals, machine tools, measurement devices).

  • Applying for fellowships. If you don't already have a fellowship, then your advisor will encourage you to apply for several each year, hoping that you can win at least one. If you can win a fellowship, then your advisor can use their grant money to fund other students. Fellowship applications can take a month or two to write, and your chances of winning are fairly low. All of that time could have been spent advancing your own research rather than trying to raise money.

  • Helping your advisor write grant proposals. Since grant proposals have an acceptance rate on par with fellowships (3 to 10 percent), you might spend a month or two working on a proposal that will likely get rejected. However, if you are funded by your advisor's grants, then you are expected to help write proposals to continue funding for you and your advisor's other students.

  • Helping your advisor with errands related to existing grants. This usually involves giving demos and presentations of your research to representatives from grant agencies to show that your lab is delivering on the milestones that the grant proposal promised.

  • Helping your advisor review papers. Professors volunteer to review papers as a part of their professional duties. However, since they might have 15 to 20 papers to review per conference, and each paper takes three to five hours to review, they will usually delegate some reviews to their senior students.

  • Helping your advisor organize conferences, professional meetings, lab group outings, or other work-related events.

  • Mentoring new students in your lab, helping them get started on their projects, and counseling them.

  • Helping your fellow Ph.D. students improve their papers and presentations by offering critiques.

  • Department service. You can help plan department-wide events or serve on committees such as the Ph.D. admissions committee.

  • Teaching. It might sound crass to view teaching as an errand, but I define an errand as any activity that does not help you graduate, and that includes teaching.

As a Ph.D. student, you must balance your time between errands and research work. The less time you spend on errands, the more time you leave for meaningful research, and the faster you can hopefully graduate. However, some errands are unavoidable; for instance, it's difficult to turn down a request from your advisor. On the bright side, sometimes errands can be useful for your career development. Teaching, giving presentations to funding agencies, and critiquing your friends' oral presentations can improve your own public speaking skills. Helping your advisor write grants and review papers can improve your technical writing and salesmanship skills. Organizing events can improve your management and logistics skills. Perhaps most importantly, doing favors for your colleagues builds genuine goodwill and makes it more likely that they will do favors for you in the future.

One trend I've noticed is that some students volunteer for more errands than necessary as a way of procrastinating from their research. Progress in research is uncertain, hard-fought, and slow. Thus, it can feel good to complete useful errands that immediately help out one's colleagues. However, doing too many errands can be dangerous, since it can provide a false sense of productivity and take one's mind off of the only kind of work required to get a Ph.D. degree: publishable research.

Intellectual Freedom

I want to end this chapter with a discussion of intellectual freedom. As a Ph.D. student in an ideal world, you would have full freedom to pursue whatever kinds of research topics that you desire, make an innovative creation or discovery, write it up in a dissertation, and then get your Ph.D. degree. However, in the real world, the kinds of research topics that you can pursue are limited by the following:

  • Resources. If your research interests require access to specialized laboratory equipment, personnel, or data sets, then you simply cannot do those sorts of projects without the proper access. Resourceful students are able to, well, gain access to the necessary resources to do the kind of work that they desire.

  • Your advisor's research interests. If your advisor is genuinely interested in a topic, then they will be more enthusiastic about helping you to formulate ideas and to write papers. Since they have far more experience playing the publication game than you do, the more help they are willing to give you on papers, the better chance your papers will have of being accepted for publication.

  • Your advisor's grants. Each grant funds a specific kind of research, so you are limited to those topics if you are funded by that grant. Shrewd students are able to deftly frame their own research topics to fit into grant requirements.

  • Most importantly, what senior researchers in your field deem as acceptable and publishable research. Since these people are reviewing your paper submissions, you ultimately need their approval in order to publish and to earn your Ph.D.

Many new Ph.D. students are shocked at the disconnect between what seems like “cool research” to them, and what is publishable academic research in their particular field at a particular moment in time. Out of the set of all seemingly-innovative and intellectually-stimulating ideas, only a small subset of those count as publishable research, as dictated by paper reviewers. An even smaller subset of those publishable ideas are within reach of what an individual Ph.D. student can feasibly accomplish due to the limitations of resources, advisor interests, and grant funding.

The students who excel at playing the “Ph.D. game” are those who are able to figure out how to pursue research that they personally find interesting but that can also appeal to their advisors, grant agencies, and paper reviewers. I hesitate to give a single blanket definition of “success” for a Ph.D. student, but if I had to give one, I would define success as being able to pursue your own research interests as much as possible while still being able to convince the three to five professors on your thesis committee to let you graduate.

Postscript

In this article, I've purposely not covered why people choose to pursue a Ph.D. degree, what they hope to gain from the experience, or what kinds of jobs they strive to find after graduating. Everyone has their own personal reasons for playing the “Ph.D. game” and their own definitions of what it means to “win.” However, what most students, including myself, share in common is that we knew almost nothing about how this game worked when we started our individual journeys.

Created: 2012-06-20
Last modified: 2012-06-20
Related pages tagged as Ph.D. student life: