A.K.A. 1337SEARCH
Feynman was once asked by a Caltech faculty member to explain why spin 1/2 particles obey Fermi-Dirac statistics. He gauged his audience perfectly and said, "I’ll prepare a freshman lecture on it." But a few days later he returned and said, "You know, I couldn’t do it. I couldn’t reduce it to the freshman level. That means we really don’t understand it."
Trying to explain idea(s) to those less familiar provides an effective litmus test on a researcher's understanding of the subject and the present depth of knowledge.
Research can be made more enjoyable and productive through a little organization. For many, research techniques are a product of experience rather than, unfortunately, formal training. To improve this experience, we've provided several suggestions; it's the hope these suggestions ease some of the confusions faced when beginning a research project.
- Who is Research For?
- Characteristics of the Ideal Research Topic †
- How Do I Find Research Ideas? †
- The "Project"
† These sections take a more academic approach to research; needless to say, it presumes significant dedication in both time and interest.
- How to read an academic paper
- How to write an academic paper
- How to submit a CFP
EVERYONE! No...seriously...everyone can do it
-
Research should facilitate what the researcher(s) already enjoy!
-
Research should be fun! It does not need to be arduous and/or rigorous, rather it should always steer towards interest, ideally with a target goal - whatever that may be.
It can be as simple as dedicated time - time, specifically carved out of our busy lives, that we can dedicate to something we (the researcher) find intellectually stimulating, e.g.,:
- I want to do course X, but it [costs too must | isn't being offered | moves too fast | would be more benificial from self-direction | etc.]
- I see gap Y in field A, I think I can [fill it | contribute to it | learn why it is a gap | learn how to teach others about the gap | etc.]
- I want to pwn this thing, but need/want [dedicated time | a pwnpartner | hardware-software resources | etc.]
Being a good research involves more than "merely" positing brilliant ideas and implementing them. Most researchers spend the majority of their time reading papers, listening to talks, discussing ideas with others, writing and revising papers, staring blankly into space - and, of course, having brilliant ideas and implementing them.
If you have an interest it it, you can research it. The only limits are those self-ascribed. For sake of explaination, we've chosen to divide the topic into applied research and novel research††.
Applied Research: I want to gain/share/discover knowledge
- May rely on cutting edge publications
- Often involves more active work by the researcher in both experimentation and investigation, e.g.:
- git[hub/lab] repo scraping
- targeted web searches, specific to a [sub]domain, timeline, application, and other targeted web searches (use more than one search engine)
- reverse engineering a closed-source technology
Novel Research: I want to create/contribute/posit this new knowledge
- Often rely more on cutting edge journals and publications
- Ideas, initally, tend to be more speculative and difficult to explain
†† Note: Neither form of research is better than the other. They are simply different approaches, with their own brilliant ideas
The characteristics of an ideal topic are, to some extent, incompatible:
-
The subject should be timely. Previous groundwork should leave your research problem ripe for completion, and it should be in an active area with potential for future work and employment.
On the other hand, if a field is too crowded, and the subject too prominent, then you risk being "scooped" by a more experienced researcher who is able to work faster than you. In this case, you may be forced to start over again (rather disastrous) or at least publish jointly (possibly a blessing, but surely an inconvenience).
-
Your work should lead to a well-defined set of results to which you can lay claim. In particular, career prospects will be lessoned if you merely contribute a small piece of a very large project, a piece of software which is closely identified with a project, or is published with a long list of collaborators.
On the other hand, it is impossible to work in a vacuum, and your task can be significantly harder if you don't have a group of people working on closely related problems with whom you can interact and share code.
-
The best topics show a high level of creativity -- and are often somewhat speculative. It is often unclear at first how the ideas will develop.
On the other hand, a multiyear plan of research is a very valuable asset.
-
You should really enjoy the subject, and want to spend the next several weeks/months/years with it!
On the other hand, an ideal project in a foreign subject is of no use without someone(s) who is willing to direct you in it.
Clearly some compromise is necessary here!
It's very important to make the transition from the passive mode of learning that traditional lecture courses encourage to an active, and critical, learning style.
Whenever you read technical material, evaluate a piece of software, or listen to a research talk, ask yourself these canonical questions:
- From where did the author seem to draw the ideas?
- What exactly was accomplished by this piece of work?
- How does it seem to relate to other work in the field?
- What would be the reasonable next step to build upon this work?
- What questions are left unanswered?
- What are the important references cited by this work?
- What ideas from related fields might be brought to bear upon this subject?
- Can the results be generalized?
One technique that some find helpful is to keep a written log of your technical reading and listening. Review it periodically to see if some of the ideas begin to fit together.
Similarly, keep a sheaf of small projects on paper. These are one or two pages of formally presented ideas on a single subject. When ideas are not allowed to get out and wander about they become malformed and misaligned. These writeups keep the pencil sharp and serve to organize spontaneous ideas. Having these available at a moment’s notice pays off when one begins to seriously look for research projects.
A “journal club” provides an excuse for engineers to meet [bi-]weekly to discuss papers. Focus could be on a topic (e.g., “The Month of Garbage Collection”), but allow for digression when a common interest develops.
It is possible to spend almost all of your time in literature review and seminars. It is easy to convince yourself that by doing this you are working hard and accomplishing something. The truth of the matter is that nothing will come of it unless you're an active reader and listener, and unless you assign yourself time to develop your own ideas, too. It is impossible to "finish a literature review and then start research." New literature is always appearing, and as your depth and breadth increases, you will continually see new connections and related areas that must be studied.
Active listening and reading must be viewed as education that will involve you for the rest of your career. Don't fool yourself into thinking it must be finished before you can begin research.
Even after you have decided on your initial focus, it is important to continue a routine of reading new journals, technical reports, and attending seminars/conferences/workshops.
Set aside some time every week/month for trying to generate research ideas or improving on exisiting ideas. Finding and reading related work is the foundation of good research. If you're new to the field, you may only be familiar with buzzwords and/or material presented in textbooks.
Some important initial resources for finding pertinent references are:
- The Association for Computing Machinery (ACM) Digital Library (DL)
- The Computing Research Association (CRA)
- The arXiv Computing Research Repository (CoRR)
- Github/Gitlab, Phrack, PagedOut!
- DEF CON Media Server
- CCC Media Server
Initially, you'll start with a short list of foundation talks and readings. As a researcher, you are responsible for developing a bibliography of related works, and for finding future readings. You'll be surprised how many useful references you'll find that others have missed.
If there's a mailing list, subscribe to it. Trying to pwn something as part of the research? Subscribe to the developer mailing list. All of these sources can contribute to the development of your idea and progression of your research.
At this stage, you can add one question to the canonical list: How can these ideas help me solve my research problem?
Struggling with remaining active? Try one (or more) of these catalysts:
- Make a monthly investigation to read, at least, the recent abstracts from a given journal. Choose an article or two to read in-depth and critique.
- Make a weekly investigation to find technical talks/videos in your field. View selectively and critique.
- Pick a random conference (DEFCON/BH/CCC/etc.) talk or series. Listen and critique.
- Search common code repository sites for keywords relevant to the research. Learn from and/or contribute to any that you find. Critique it's approach.
Add these to your log, and ask the canonical questions. As you review the log six months from now, you may find something that stroke a chord then, but is beyond you now.
One of the most imporants things a researcher should do, for both themselves and their work, is establish themselves as part of the research community:
Join a community: Local IRL or online (slack/discord/zulip/etc) communities provides a researcher, especially a security researcher, an informal and casual settings to learn, teach, and bounce ideas. This is especially import for researchers new to the field.
Attend conferences/workshops: Conferences and workshops are the best place to meet people, discuss your ideas, hear new ideas, and get a sense for the current state of research in a field.
Publish papers: Publishing papers provides a researcher with a source of feedback, establishes themselves member of research community, and forces the researcher to clarify the ideas presented and to fit them into the context of the current state of research.
Deliver talks: Talks are a great way gain visibility, share your ideas, and hear them out-loud. Submit to CFPs at the next conference; lesser-known conferences provide the ideal 'low-pressure' environment for a new speaker. That does not mean DEFCON/CCC/BH/etc. are beyond reach; they are quite achievable!
Networking: Breaking into the research community requires meeting and building relationships with other researchers. Talk about your research interests every chance you get (but make sure to spend time listening, too - this is when you'll learn the most)
Once you have identified a topic that looks feasible, make sure you are aware of all of the literature in the area. Keep reading and listening, and keep distinct in your mind what the goal is of your work.
Don't let other people's frame of mind limit your creativity. Make a list of open problems and possible projects that are of interest to you and discuss them with friends, collegeues, and any other interested party.
Research in computer science often leads to a “project” involving programming. It's important to remember that programming is not computer science research. Instead, for most computer scientists, programming is merely a mechanism for performing an experiement. As with any experiment, it should be carefully planned, ahead of time:
-
Establish goals. Know where you are headed, and approach the solution without distraction. Develop a list of milestones which demonstrate progress, and strive to accomplish them. If you cannot formulate concise goals, you should stop and reconsider the motivations for the project.
Goals should be "justified" regularly. As your knowledge and understanding improve, so may your ideas and motivations. Dynamic goals provide flexibility as you undergo research; don't be afraid to scrap a goal in favour of another.
-
Think simple. Design your projects so that they may be completed within a reasonable period of time. An experienced programmer generates little more than a hundred lines of reliable code per month. A project that demands thousands of lines of code, therefore, will take more than a couple of two-week blocks to implement correctly. Time spent pruning the experiment to a manageable size is time well spent. Large projects do not necessarily yield large results.
-
Build prototypes. Most projects benefit from the construction of a prototype. A well considered prototype validates assumptions, tests the value of abstractions, and motivates reconsideration of weak ideas. While there is little research value associated with polishing off a “product,” many research questions can be answered satisfactorily through mock-ups or partial implementations.
-
Use tools. A programmer’s performance is dramatically improved through the use of a few simple tools. There are, of course, many important and useful tools, but the main point is clear: the correct choice of tools can reduce the total work in a project. Find them, learn from them, and use them.
-
Collaborate. When resources can be coordinated, groups are often more productive than individual efforts in isolation. For many, success comes from collaboration. Try to share and develop ideas in a group atmosphere. Contact others that share your interestes and collaborate! Undoubtedly they will have solved problems you are currently considering, and their solutions will influence how you achieve common goals.
One side effect of collaboration is increased discipline, discipline that is necessary to reduce the amount of energy expended synchronizing efforts. Some tips for collaboration:
- Keep a regular schedule of meetings
- Establish points of synchronization where concentration is refocused on an issue
- Consider critisim carefully; conversely, provide others only with constructive critisim
-
Document results. Finished projects should be documented. At a minimum, a technical overview of the experiment allows others to see what motivates your research. The document should describe the problem, your assumptions, your approach, and an honest evaluation of your results. When documenting software, include illustrative examples, tutorials, and any experience gained from its use. Well written documentation greatly increases the impact of a project.
It's vital to reserve a good portion of time for writing. An hour spent writing is an hour spent considering a problem instead of, say, grappling with a computer. You must spend time away from other distractions to document work and focus your efforts.