2013 CRA-W Graduate Cohort Workshop Finding a Research Topic Carla Brodley Professor and Chair, Department of Computer Science Tufts University (with credits to Lori Pollack and Padma Raghavan)

Academic History Started graduate school, UMASS…………….Fall 1988 Ph.D. awarded………………………………….Aug 1994 Started as Assistant Professor, Purdue….….Nov 1994 Promoted to Assoc. Prof. w/ tenure ………Spring 2000 Started as a Full Professor, Tufts..………..…Fall 2004 Department Chair, Tufts……………………….Fall 2010

The Thesis Equation Topic + Advisor = Dissertation n

What is (CS) Research? the systematic investigation into and study of materials, sources, etc., in order to establish facts and reach new conclusions Oxford dictionary –Experimental scientific research: Observe a problem Formulate a hypothesis Develop a strategy to solve problem based on hypothesis Perform experiments and demonstrate conclusive evidence Interpret results

What is (CS) Research? the systematic investigation into and study of materials, sources, etc., in order to establish facts and reach new conclusions Oxford dictionary Research is not knowing the answer or how to get it –Theoretical scientific research: Identify an open question Formulate a hypothesis Prove hypothesis

What is CS Research? Example from Machine Learning

Classification k-Nearest Neighbor o o o o oo o o o o x x x x xx x x x ?

Classification k-Nearest Neighbor o o o o oo o o o o x x x x xx x x x ?

Classification k-Nearest Neighbor o o o o oo o o o o x x x x xx x x x ? Assign majority class of the k nearest neighbors

What is CS Research? Example from Machine Learning Observe a problem: Performance of k-NN is little better than random guessing for a particular dataset Hypothesis: Classification accuracy will improve if I can find and eliminate irrelevant and noisy features. Strategy: Develop a feature selection algorithm: eliminate features with low correlation with the class label Evaluation/Evidence: Implement and compare accuracy of original k-NN to new feature selection k- NN across a large number of data sets. Interpret results: Feature selection improves performance in M of the N datasets, …next steps?????

So, what isnt PhD research?

How do I choose a topic area for my research? Whose interest do you need to grab? – You – Your advisor – Your research community Love your topic! – Sets the course for your next 2-3 years – Determines, in part, opportunities offered to you upon graduation – May work in same/related area for years

More Things to Consider What are your strengths? weaknesses? – Programming, design, data analysis, proofs – Key insights versus long/detailed verification/simulation What drives you? bores you? – Technology, puzzles, applications, interdisciplinary Do you (i.e., your advisor) have funding for you to work in the area? – Working as a TA – Working as an RA – Having university/college, government, industry, etc… fellowship/scholarship/grant

Which comes first? Advisor or Topic Area? For many people advisor before topic – Meet faculty member with compelling research interests For some people topic before advisor – Need a guide in an area already of great interest to you Want an advisor – Knowledgeable about your topic Interdisciplinary topics may require >1 advisor – With compatible working style (e.g., solo vs team) – With lots of research ideas – With strong interest in working with PhD students

Focusing from Area to Topic Area = subfield – architecture, theory, AI, high performance computing, or interdiscplinary – Is it important? Timely? Jobs in the area? Topic = specific open problems in subfield – Theory: provably better algorithm – AI: Improving a machine learning algorithm – Architecture: multicore cache design – HPC: parallel algorithm, scheduling scheme – Interdisciplinary: computer simulation of tumor growth

Topic Scale and Scope Topic Scale and Scope Scale – Should have more than one open problem, or solving one should lead to another – Should lead to more than one result/finding, some big, some smaller Scope – Too narrow, e.g., just analysis no experiment, many not leave enough room – Too broad, e.g., data mining, for what? why? too open ended

Selecting a Topic Moving from coursework to picking a topic is often a low point – Even for the most successful students Why? – Going from what you know- coursework, to something new- research – It is very important – There is no *one* ideal way, but many good ways

7 Ways to Identify a Good Research Problem

1) Flash of Brillance You wake up one day with a new insight/idea New approach to solve an important open problem Warnings: –This rarely happens if at all –Even if it does, you may not be able to find an advisor who agrees

2) The Apprentice Your advisor has a list of topics Suggests one (or more!) that you can work on Can save you a lot of time/anxiety Warnings: –Dont work on something you find boring, fruitless, badly-motivated,… –Several students may be working on the same/related problem

3) The Extended Course Project You take a project course that gives you a new perspective The project/paper combines your research project with the course project – One (and ½) project does double duty Warnings: –This can distract from your research if you cant find a related project/paper

4) Redo … Reinvent You work on some projects – Re-implement or re-do; Evaluate – Identify an improvement, algorithm, proof You have now discovered a topic Warnings: –You may be without a topic for a long time –It may not be a topic worthy of a doctoral thesis

5) Analyze Data You participate in more senior students evaluation study: – Help with data collection and analysis – Identify open challenges You have now discovered a topic Warnings: –You will have to agree on who works on identified open challenges –It may not be a topic worthy of a doctoral thesis

6) The Stapler You work on a number of small topics that turn into a series of conference papers You figure out somehow how to tie it all together, create a chapter from each paper, and put a BIG staple through it Warning: –May be hard/impossible to find the tie

7) The Synthesis Model You read papers from other subfields in computer science or a related field Look for places to apply insight from another (sub)field to your own – E.g., machine learning to compiler optimizations Warnings: –You can read a lot of papers and not find a connection –Or realize someone has done it already! –Or you have not made a significant impact in either field

Tips and Suggestions Topic + advisor are both important Keep a research ideas journal (wiki) Keep an annotated bibliography (bibtex) Follow your interests and passion – Key driver for success and impact Are you eager to get to work, continue working? If not really interested, adapt – Tedium or actual lack of interest and motivation?

When youre stuck at the start Read/present papers regularly to find open research issues – Practice summarizing, synthesizing & comparing sets of papers – Write your own slides for presentations Work with a senior PhD student on their research Try something…. Get feedback and ideas from others: conferences, research internships, advisors idea

When youre still stuck… Read a PhD thesis in your area – Often contain an open problems or future work section Read your advisors grant proposals Attend PhD oral exams and thesis defenses – Understand how to formulate problems – Understand what constitutes a problem solution Assess your progress, with your advisor – Set goals per semester - Have you ruled out an area? converged on an area? Chosen a topic for an exploratory research project?

When youre stuck again Divide your topic into milestones, and develop a plan to work on them one-by- one – Reward yourself when you finish a milestone – Publications and/or posters as milestones – Vary what you do during the day, but set aside blocks of time for each activity Assess your progress regularly, with your advisor – Have you submitted a workshop paper? A term project with documentation? A poster at a conference? A talk at a regional conf?

When youre really really stuck Change research topics? – May move you out of your advisors comfort zone of expertise – Starting from scratch (e.g., need to learn the related work in a new area) Change research advisor? – May go through shakedown period again – May or may not be better off But change can be invigorating – Whats hard? Need to recognize when things are not working out and take action – Weigh consequences of changing and not changing

The Six Questions…. (from Paul Utgoff) What research issue(s) interest you most? Why? Who else has worked in this vein? What did they accomplish? What can't they do? What kind of progress would you like to see? Why? Do you have an idea for making some such progress? Explain. What do you expect to discover from your investigation? How will your expected result(s) affect the research community?

So how did I find my topic? At ICML1990, I was irritated by –Yet Another Learning Algorithm (YALA) – Strategic selection of UCI benchmark datasets to show YALAs superiority My idea: Given a dataset, select the best algorithm automatically for that dataset My next observation: Why should we assume all parts of the data space have the same bias? My next idea: Recursive automatic bias selection

Identify a research topic and get started! Great relevant article in ACM Crossroads,How to Succeed in Graduate School: A Guide for Students and Advisors, (part I, Dec 1994; part II, Feb 1995), available in ACM Digital Library