吴恩达关于新手论文阅读的建议![强烈推荐]

Reading Research Papers:

How can you learn efficiently and relatively quickly through reading research papers. So, what you should do in case you want to learn from the academic literature whether you want to learn to build a machine learning system/project of interest, or just to stay on top of things, gain more knowledge and evolve as a deep learning person.

Here comes the list:

Compile a list of papers: try to create a list of research papers, medium posts and whatever text or learning resource you have.

Skip around the list: basically, you should read research papers in a parallel fashion; meaning try to tackle more than one paper at a time. Concretely, try to quickly skim and understand each of these paper and do not read it all, maybe you read 10–20% of each one and probably that will be enough to give you a high-level understanding of the paper in hand. After that, you may decide to eliminate some of these papers or just go over one or two them and read them fully.

He mentioned also that if you read:

5–20 papers (in a field of choice, say speech recognition) => it may be probably enough knowledge for you to implement a speech recognition system, but maybe not enough to research or be at the cutting-edge.

50–100 papers => you probably have a very good understanding of the domain application (speech recognition).

阅读论文:

你应该如何通过阅读论文高效且快速地学习。 如果你出于某种原因——为了建立机器学习系统或项目,或者只是想要跟上潮流,获得更多前沿知识并且参与进来——想要学习学术文献,你应该怎么做?

这里是建议:

建立文献的清单: 努力去建立一份关于论文,媒体博客或者任何种类的文献或学习资源的清单。

在清单中省略一部分:基本上,你应该以一种平行的方式去读文献;也就是说你应该同时处理多份文献。 更具体一点,你应该快速略读这些论文的一部分(或许10-20%)来获得对于手上的论文的高层次的理解。 此后,你再决定是否去掉某些论文或者在这些里面只保留一两篇并且完整阅读。

他还提到了,如果你在某个领域(比方说语音识别)阅读了:

5-20篇论文,那么可能你已经有了足够的知识去建立一个语音识别的系统,但是对于前沿科技可能还不够。

50-100篇论文,那么你已经对于这个领域的应用已经有很好的理解了。

How do you read one paper?

Don’t start reading the paper from the first to the last word. Instead, take multiple passes through the paper, here’s how to do it:

Read the Title, the abstract and the figures: by reading the title, abstract, the key network architecture figure, and maybe the experiments section, you will be able to get a general sense of the concepts in the paper. In deep learning, there are a lot of research papers where the entire paper is summarized in one or two figures without the need to go hardly through the text.

Read the introduction + conclusions + figures + skim the rest: the introduction, the conclusions and the abstract are the places where the author(s) try to summarize their work carefully to clarify for the reviewer why their paper should be accepted for publication.
Also, skim the related work section (if possible), this section aims to highlight work done by others that somehow ties in with the author(s) work. Hence, it may be useful to read it but if you’re not familiar with the literature, it is sometimes very hard to understand.

Read the paper but skip the math.

Read the whole thing but skip the parts that don’t make sense: great research means we’re publishing things at the boundaries of our knowledge and understanding.
He also explained that when you read papers (even the most influential ones), you’ll find maybe some parts that is much less used or it doesn’t make sense. Consequently, it’s fine if you read a paper and some of it doesn’t make sense (it’s not unusual), it’s okay to skim it initially. Unless, you’re trying to master it, then spend more time.

如何阅读一篇论文?

不要一字不落得从头读到尾。 相反地,你应该在阅读过程中跳过一些部分,接下来我们将讨论如何做到这件事:

阅读标题,摘要和图:通过读标题、摘要和网络结构的图,或许再加上实验的部分,你就能对这篇论文的概念有一个总的理解。在深度学习中,很多学术论文都可以用一或两张图片来概括,而不需要费力气去读整篇文章。

阅读引言+结论+图+略读剩下的:引言、结论和摘要,是作者概括他们的工作并以此来为审稿人阐述为什么他们的论文应该被收录的重要部分。另外,略读相关工作的部分,这部分是为了标明其他人所做的与论文相关的工作,因此,这部分应该会比较有用,但是如果你对相关文献不熟悉,这部分可能会比较难以理解。
阅读整篇论文,但是跳过数学部分。

阅读整篇论文,但是跳过讲不通的地方:伟大的研究意味着我们在发表处在人类知识以及理解边缘的东西。

他还提到了,当你阅读论文的时候,你可能会遇到一些内容,这些内容很少被使用或者讲不通。 因此,如果你阅读一篇论文并且这篇论文中的部分内容很少见或者讲不通,这是很正常的,你完全可以先跳过这些内容。 除非你想精通这个领域,这样的话你可以花更多时间去搞懂它。

When you read a paper, try to answer the following questions:

What did the author(s) try to accomplish?
What were the key elements of the approach?
What can you use yourself?
What other references do you want to follow?

If you can answer these questions, hopefully, that will reflect that you have a good understanding of the paper.

It turns out as you read more papers, with practice you get faster. Because, a lot of authors use common formats when writing papers.

For example, this is a common format that authors use to describe the network architecture, especially in computer vision:

image.png

Reference: Densely Connected Convolutional Networks.

当你阅读一篇论文时,试着回答下面几个问题:

作者想要实现什么?

实现方法的重要部分是什么?

哪些内容可以为你所用?

作者引用了哪些参考文献你可以去阅读?

如果你能够回答这些问题,这应该能说明你对论文已经有了一个很好的理解。

事实证明,随着你读的论文越来越多,你读得也会越来越快。因为很多作者都会在写论文时用共同的格式。

比方说,这是作者们用来表示网络结构地共同格式,尤其在计算机视觉方向:

image.png

Reference:Densely Connected Convolutional Networks.

How much time it takes to understand a paper?

It’s not unusual for people that are relatively new to machine learning to need maybe an hour to understand a relatively easy paper. But, sometimes you may stumble across papers that takes 3 hours or even longer to really understand it.

Sources of papers:

There are a lot of great resources online. For example, blog posts listing the most important papers in speech recognition would be very useful if you’re new to this domain.

A lot of people try to keep up with the state-of-the-art of deep learning as it evolves rapidly. And so, here’s where you should go:

Twitter: surprisingly, Twitter is becoming an important place for researchers to find out about new things.
ML subreddit.
Important Machine Learning Conferences: NIPS/ICML/ICLR.
Friends: find a community or a group of friends ,interested in the field, that shares interesting research papers.

To understand the math in papers more deeply:

Try to rederive it from scratch. Although, it takes some time but it’s a very good practice.

To practice Coding:

Download open-source code (if you can find it) and run it.
Reimplement from scratch: if you can do this, that’s a strong sign that you have really understood the algorithm in hand.

To keep getting better:

The most important thing to keep on learning and getting better is to learn more steadily rather than having a focus-intensive activity. It’s better to read two papers a week for the next year than cramming everything over a short period of time.

论文的来源:

网络上有很多很好的资源。例如,列举重要论文的博客可能会对作为新人的你很有用。

很多人想要跟随深度学习的前沿发展——毕竟这个领域日新月异——接下来是你应该去的地方:

推特:令人惊讶的是,推特在研究者发现新的事物地过程中正扮演着越来越重要地角色。

ML subreddit

重要的机器学习会议:NIPS/ICML/ICLR.

朋友们:加入一个社区或者一群有同样爱好的朋友,他们会分享有趣的研究论文。

更深地理解论文中的数学:

试着去从头到尾推导出来。 虽然这样很费时间,但是这是很好的实践。

练习代码:

如果能找到的话,下载开源的代码并且运行它。

从头到尾重新实现一遍:如果你能做到,这说明你已经对这个算法理解了。

不断变得更好:

为了保持进步和不断学习,最重要的就是稳定地学习,而不是心血来潮或者临时抱佛脚。 每周读两篇论文,持续做一年,比把所有事情都挤在一段相对短的时间内做完更好。

Advice for navigating a career in machine learning:

Whether your goal is to get a Job (big company, startup and faculty positions) or do more advanced graduate studies (maybe join a PhD program)

Just focus on doing important work and consider your job as a tactic and a chance to do useful work.

What do recruiters look for?

Machine Learning Skills.
Meaningful work: projects which shows that you can do the work.

A very common pattern for successful machine learning engineers ,strong job candidates, is to develop a T-shaped knowledge base. Meaning to have a broad understanding of many different topics in AI and very deep understanding in at least one area.

image.png

Reference: deeplearning.ai.

To build the horizontal piece:

A very efficient way to build foundational skills in these domains is through courses and reading research papers.

To build the vertical piece:

You can build it by doing related projects, open-source contributions, research and internships.

招聘者们在寻求什么?

1.机器学习的技能。

2.有意义的工作:能展现你能胜任这份工作的项目。

在成功的机器学习工程师(有竞争力的工作候选人)当中,很普遍的特征是发展了一个T型的知识基础。也就是说广泛理解AI的多个领域,并且在某一个领域有极深的理解。

image.png

Reference: deeplearning.ai.

如何建立横向部分:

一个很有效的方法就是通过课程和阅读论文。

如何建立纵向部分:

你可以通过做相关的项目,为开源社区做贡献,做研究或者实习来完成这部分。

Selecting a job:

If you want to keep learning new things, here are the things that affect your success:

Whether you are working with great people/projects: being surrounded by hard-working people will influence you.

Focus on knowing about and evaluating the team that you will be working with (the 10–30 people that you will interact with the most) in addition to the manager.

Do not focus on “brand”: the brand of the company is not that much correlated to what your personal experience would be like.

So if you get a job offer, ask about which team you will be working with and don’t accept an offer that says “join us, and we’ll assign to a team later” because you might end up with a team that work on things that don’t interest you; which doesn’t help to evolve efficiently.

On the other hand, if you can track down a good team (even in an unknown company) and join them, you can actually learn a lot.

General Advice:

Learn the most: tend to choose things to work on that allow you to learn the most.

Do important work: work on worthy projects that moves the world forward.

Try to take machine learning to traditional industries: we’ve transformed a lot in the tech industry but I think one of the most exciting work to be done may be in traditional industries (outside the tech industry) because you can create much more value there.

选择一个工作:

如果你想不断学习新的东西,接下来的内容会影响到你的成功:

1.无论你是否在做伟大的项目或者在伟大的团队:和努力的人呆在一起会影响到你。

2.除了了解你的经理,注意了解和评估你接下来会一起工作的团队(比如10-30个你会接触和互动的人)。

3.不要再注重”品牌“:公司的牌子并不完全和你个人的体验相挂钩。

因此如果你拿到了一份工作的offer,询问你的未来团队的情况,不要加入那些说”加入我们,我们会之后把你分到一个团队里面。“之类的公司,因为之后你可能会加入一个你根本不感兴趣的团队,他们做的事情可能意义不大或者很难迭代进步。

另一方面,如果你能够找到某个很好的团队(即使在不知名的企业里)并且加入他们,你实际上能学到很多。

一般建议:

1.多学习:你要倾向于去选择能让你学习到更多知识的事情来做。

2.做重要的工作:做能推动世界进步的有价值的项目。

3.试着把机器学习落地到传统工业中:我们已经转换过很多科技企业,但是我认为最激动人心的就是传统工业(科技之外的),因为在那里你能创造更多的价值。

Key Takeaways:

I tried to summarize the key takeaways of Andrew’s advice in the following list:

Develop a habit of reading research papers: maybe 2 papers a week as a start.

2.Read efficiently: compile a list of papers, read more than one paper at a time and take multiple passes through each one.

When reading a paper: start by reading the title/abstract/figures (especially)/introduction/conclusions.
When trying to understand an algorithm: try to rederive the math and practice coding by reimplementing it.
Try to stay on top of things: by checking papers in ML conferences and other online resources.
Build a T-shaped knowledge base in AI.
Try to join a good team (in a big company or a startup) that will help you grow efficiently.
Work on useful projects that helps you learn the most and moves the world forward.
Try to take machine learning to other industries: healthcare, astronomy, climate change, etc.

重要收获:

我试着将吴恩达的建议总结成了以下几点:

1.养成阅读论文的习惯:或许一开始每周两篇就好了。

2.高效阅读:建立文献清单,同时读多篇文章并且在读每一篇文章时跳过不重要的部分。

3.当你读论文的时候:最先读标题/摘要/图片(重要)/引言/结论。

4.当你试图理解算法时:试着自行推导数学公式,并且用代码实现一遍。

5.试着跟上潮流:通过阅读机器学习会议或者其他网上资源的论文。

6.建立T型的AI知识基础。

7.努力加入一个能帮助你成长的好团队。

8.做能让你学习并且对世界有贡献的有价值的项目。

9.试着将机器学习运用在其他领域:医疗,航天,气候变化等等。