Epilogue

The Ph.D. Grind

If you are not going to become a professor, then why even bother pursuing a Ph.D.? This frequently-asked question is important because most Ph.D. graduates aren't able to get the same jobs as their university mentors and role models—tenure-track professors. There simply aren't enough available faculty positions, so most Ph.D. students are directly training for a job that they will never get. (Imagine how disconcerting it would be if medical or law school graduates couldn't get jobs as doctors or lawyers, respectively.)

So why would anyone spend six or more years doing a Ph.D. when they aren't going to become professors? Everyone has different motivations, but one possible answer is that a Ph.D. program provides a safe environment for certain types of people to push themselves far beyond their mental limits and then emerge stronger as a result. For example, my six years of Ph.D. training have made me wiser, savvier, grittier, and more steely, focused, creative, eloquent, perceptive, and professionally effective than I was as a fresh college graduate. (Two obvious caveats: Not every Ph.D. student received these benefits—many grew jaded and burned-out from their struggles. Also, lots of people cultivate these positive traits without going through a Ph.D. program.)

Here is an imperfect analogy: Why would anyone spend years training to excel in a sport such as the Ironman Triathlon—a grueling race consisting of a 2.4-mile swim, 112-mile bike ride, and a 26.2-mile run—when they aren't going to become professional athletes? In short, this experience pushes people far beyond their physical limits and enables them to emerge stronger as a result. In some ways, doing a Ph.D. is the intellectual equivalent of intense athletic training.

Here are twenty of the most memorable lessons that I've learned throughout my Ph.D. years. My purpose in sharing is not to provide unsolicited advice to students, since everyone's Ph.D. experience differs greatly; nor is it to encourage people to pursue a Ph.D., since these lessons can come from many sources. Rather, this section merely serves as a summary of what I gained from working towards my Ph.D.

Results trump intentions: Nobody questions someone's intentions if they produce good results. I didn't have so-called pure intellectual motivations during grad school: I started a Ph.D. because I wasn't satisfied with engineering jobs, pressured myself to invent my own projects out of fear of not graduating on time, and helped out on HCI projects with Scott, Joel, and Jeff to hedge my bets. But I succeeded because I produced results: five prototype tools and a dozen published papers. Throughout this process, I developed strong passions for and pride in my own work. In contrast, I know students with the most idealistic of intentions—dreamy and passionate hopes of revolutionizing their field—who produce few results and then end up disillusioned.
Outputs trump inputs: The only way to earn a Ph.D. is by successfully producing research outputs (e.g., published papers), not merely by consuming inputs from taking classes or reading other people's papers. Of course, it's absolutely necessary to consume before one can produce, but it's all too easy to over-consume. I fell into this trap at the end of my first year when I read hundreds of research papers in a vacuum—a consumption binge—without being able to synthesize anything useful from my undirected readings. In contrast, related work literature searches for my dissertation projects were much more effective because my reading was tightly directed towards clear goals: identifying competitors and adapting good ideas into my own projects.
Find relevant information: My Ph.D. training has taught me how to effectively find the most relevant information for what I need to accomplish at each moment. Unlike traditional classroom learning, when I'm working on research, there are no textbooks, no lecture notes, and no instructors to provide definitive answers. Sometimes what I need for my work is in a research paper, sometimes it's within an ancient piece of computer code, sometimes it's on an obscure website, and sometimes it's inside the mind of someone whom I need to track down and ask for help.
Create lucky opportunities: I got incredibly lucky several times throughout grad school, culminating in getting to work with Margo at Harvard during my final year. But these fortuitous opportunities wouldn't have arisen if I didn't repeatedly put myself and my work on display—giving talks, chatting with colleagues, asking for and offering help, and expressing gratitude. The vast majority of my efforts didn't result in serendipity, but if I didn't keep trying, then I probably wouldn't have gotten lucky.
Play the game: As a Ph.D. student, I was at the bottom of the pecking order and in no position to change the “academic game.” Specifically, although I dreaded getting my papers repeatedly rejected, I had no choice but to keep learning to play the publication game to the best of my abilities. However, I was happy that I played in my own unique and creative way during the second half of grad school by pursuing more unconventional projects while still conforming to the “rules” well enough to publish and graduate.
Lead from below: By understanding the motivations and personalities of older Ph.D. students, professors, and other senior colleagues, I was able to lead my own initiatives even from the bottom of the pecking order. For example, after I learned Margo's research tastes by reading her papers and grant applications, I came up with a project idea (Burrito) that we were both excited about. If I were oblivious to her interests, then it would have been much harder to generate ideas to her liking.
Professors are human: While this might sound obvious, it's all too easy to forget that professors aren't just relentless research-producing machines. They're human beings with their own tastes, biases, interests, motivations, shortcomings, and fears. Even well-respected science-minded intellectuals have subjective and irrational quirks. From a student's perspective, since professors are the gatekeepers to publication, graduation, and future jobs, it's important to empathize with them both as professionals and also as people.
Be well-liked: I was happier and more productive when working with people who liked me. Of course, it's impossible to be well-liked by all colleagues due to inevitable personality differences. In general, I strived to seek out people with whom I naturally clicked well and then took the time to nurture those relationships.
Pay some dues: It's necessary for junior lab members to pay their dues and be “good soldiers” rather than making presumptuous demands from day one. As an undergraduate and master's student at MIT, I paid my dues by working on an advisor-approved, grant-funded project for two and a half years rather than trying to create my own project; I was well-rewarded with admissions into top-ranked Ph.D. programs and two fellowships, which paid for five years of graduate school. However, once I started at Stanford, I paid my dues for a bit too long on the Klee project before quitting. It took me years to recognize when to defer to authority figures and when to selfishly push forward my own agenda.
Reject bad defaults: Defaults aren't usually in the best interests of those on the bottom (e.g., Ph.D. students), so it's important to know when to reject them and to ask for something different. Of course, there's no nefarious conspiracy against students; the defaults are just naturally set up to benefit those in power. For example, famous tenured professors like Dawson are easily able to get multi-year grants to fund students to work on “default” projects like Klee. As long as some papers get published from time to time, then the professor and project are both viewed as successful, regardless of how many students stumbled and failed along the way. Students must judge for themselves whether their default projects are promising, and if not, figure out how to quit gracefully.
Know when to quit: Quitting Klee at the end of my third year was my most pivotal decision of grad school. If I hadn't quit Klee, then there would be no IncPy, no SlopPy, no CDE, no ProWrangler, and no Burrito; there would just be three or more years of painful incremental progress followed by a possible “pity graduation.”
Recover from failures: Failure is inevitable in grad school. Nothing I did during my first three years made it into my dissertation, and many paths I wandered down in my latter three years were also dead-ends. Grad school was a safe environment to practice recovering from failures, since the stakes were low compared to failing in real jobs. In my early Ph.D. years, I would grow anxious, distraught, and paralyzed over research failures. But as I matured, I learned to channel my anger into purposeful action in what I call a productive rage. Every rejection, doubt, and criticism spurred me to work harder to prove the naysayers wrong. Lessons learned from earlier failures led to successes later in grad school. For example, my failure to shadow professional programmers at the beginning of my second year taught me how and who to approach for these sorts of favors, so I later succeeded at shadowing computational researchers to motivate my dissertation work; and my failure to get lots of real users for IncPy taught me how to better design and advertise my software so that I could get 10,000 users for CDE.
Ally with insiders: I had an easy time publishing papers when allied with expert insiders such as Scott and Joel during my second year, Tom during my MSR internship, and Jeff during my fifth year. They knew all the tricks of the trade required to get papers published in their respective subfields; the five papers that I co-wrote with these insiders were all accepted on their first submission attempts. However, struggling as an outsider—with Dawson on empirical software measurement in my second year and then on my solo dissertation projects—was also enriching, albeit more frustrating due to repeated paper rejections.
Give many talks: I gave over two dozen research presentations throughout my Ph.D. years, ranging from informal talks at university lab group meetings to conference presentations in large hotel ballrooms. The informal talks I gave at the beginning of projects such as IncPy were useful for getting design ideas and feedback; those I gave prior to submitting papers were useful for discovering common criticisms that I needed to address in my papers. Also, every talk was great practice for improving my skills in public speaking and in responding to sometimes-hostile questions. Finally, talks sometimes sparked follow-up discussions that led to serendipity: For example, after watching my first talk on IncPy, a fellow grad student emailed me a link to Fernando's blog post about Python in science; that email encouraged me to reach out to Fernando, who would later inspire me to improve IncPy and then to invent CDE. Over a year later, my Google Tech Talk on CDE directly led to my super-chill summer 2011 internship.
Sell, sell, sell: I spent the majority of my grad school days heads-down grinding on implementing research ideas, but I recognized that convincingly selling my work was the key to publication, recognition, and eventual graduation. Due to the ultra-competitive nature of the paper publication game, what often makes the difference between an accept and a reject decision is how well a paper's “marketing pitch” appeals to reviewers' tastes. Thus, thousands of hours of hard grinding would go to waste if I failed to properly pitch the big-picture significance of my research to my target audience: senior academic colleagues. More generally, many people in a field have good ideas, so the better salespeople are more likely to get their ideas accepted by the establishment. As a low-status grad student, one of the most effective ways for me to “sell” my ideas and projects was to get influential people (e.g., famous professors such as Margo) excited enough to promote them on my behalf.
Generously provide help: One of my favorite characteristics of the Ph.D. experience was that I wasn't in competition with my classmates; it wasn't like if they did better, then I would do worse, or vice versa. Therefore, many of us generously helped one another, most notably by giving feedback on ideas and paper drafts before they were subject to the harsher critiques of external reviewers.
Ask for help: Over the past six years, I became good at determining when, who, and how to ask for help. Specifically, whenever I felt stuck, I sought experts who could help me get unstuck. Finding help can be as simple as asking a friend in my department, or it might require getting referrals or even cold-emailing strangers.
Express true gratitude: I learned to express gratitude for the help that others have given me throughout the years. Even though earning a Ph.D. was a mostly-solitary process, I wouldn't have made it without the generosity of dozens of colleagues. People feel good when they find out that their advice or feedback led to concrete benefits, so I strive to acknowledge everyone's specific contributions whenever possible. Even a quick thank-you email goes a long way.
Ideas beget ideas: As I discovered at the end of my first year, it's nearly impossible to come up with substantive ideas in a vacuum. Ideas are always built upon other ideas, so it's important to find a solid starting point. For instance, the motivations for both IncPy and SlopPy came from my frustrations with programming-related inefficiencies I faced during my 2009 MSR internship. A year later, some of my ideas for extending IncPy, mixed with Fernando's insights on reproducible research and Dawson's mention of Linux dependency hell, led to the creation of CDE. Also, ideas can sometimes take years to blossom, usually after several false starts: I started pondering Burrito-like ideas during my second year and then at the end of my fourth, but it wasn't until my sixth year that I was able to solidify those fuzzy thoughts into a real project.
Grind hard and smart: This book is named The Ph.D. Grind because there would be no Ph.D. without ten thousand hours of unglamorous, hard-nosed grinding. This journey has taught me that creative ideas mean nothing without the extreme effort to bring them to fruition: showing up to the office, getting my butt in the seat, grinding hard to make small but consistent progress, taking breaks to reflect and refresh, then repeating day after day for over two thousand consecutive days. However, grinding smart is just as important as grinding hard. It's sad to see students blindly working themselves to death on tasks that won't get favorable results: approaching a research problem from an unwise angle, using the wrong kinds of tools, or doing useless errands. Grinding smart requires perceptiveness, intuition, and a willingness to ask for help.

I'll end by answering a question involving the F-word: Was it fun?

Some aspects of the Ph.D. experience were very fun: Coming up with new ideas was fun; sketching out software designs on the whiteboard was fun; having coffee with colleagues to chat about ideas was fun; hanging out with interesting people at conferences was fun; giving talks and inciting animated discussions was fun; receiving enthusiastic emails from CDE users around the world was fun. But I probably spent only a few hundred hours on those activities throughout the past six years, which was less than five percent of my total work time.

In contrast, I spent about ten thousand hours grinding alone in front of my computer—programming, debugging, running experiments, wrestling with software tools, finding relevant information, and writing, editing, and rewriting research papers. Anyone who has done creative work knows that the day-to-day grind is rarely fun: It requires intense focus, rigorous discipline, keen attention to detail, high pain tolerance, and an obsessive desire to produce great work.

So, Was it fun?

I'll answer using another F-word: It was fun at times, but more importantly, it was fulfilling. Fun is often frivolous, ephemeral, and easy to obtain, but true fulfillment comes only after overcoming significant and meaningful challenges. Pursuing a Ph.D. has been one of the most fulfilling experiences of my life, and I feel extremely lucky to have been given the opportunity to be creative during this time.

Next - Afterword

Download PDF version