Lisp, Smalltalk, and the Power of Symmetry

In Search of Secrets

Like many hackers, my first real programming language love was Lisp. Paul Graham, who inspired my own explorations of the language, is a particular advocate and has written quite a bit about Lisp and what makes it different from other programming languages. So what does make Lisp different? Why does Lisp continue to be one of the most powerful, flexible, and concise programming languages in existence, despite the fact that it was invented in 1958–making it the second-oldest high-level programming language in the world?

View original post 948 more words

Lisp, Smalltalk, and the Power of Symmetry

Understanding deep learning requires re-thinking generalization

the morning paper

Understanding deep learning requires re-thinking generalization Zhang et al., ICLR’17

This paper has a wonderful combination of properties: the results are easy to understand, somewhat surprising, and then leave you pondering over what it all might mean for a long while afterwards!

The question the authors set out to answer was this:

What is it that distinguishes neural networks that generalize well from those that don’t? A satisfying answer to this question would not only help to make neural networks more interpretable, but it might also lead to more principled and reliable model architecture design.

By “generalize well,” the authors simply mean “what causes a network that performs well on training data to also perform well on the (held out) test data?” (As opposed to transfer learning, which involves applying the trained network to a related but different problem). If you think about that for a moment, the question pretty…

View original post 1,372 more words

Understanding deep learning requires re-thinking generalization

I tried Haskell for 5 years and here’s how it was

Meta Rabbit

One blogpost style which I find almost completely useless is “I tried Programming Language X for 5 days and here’s how it was.” Most of the time, the first impression is superficial discussing syntax and whether you could get Hello World to run.

This blogpost is I tried Haskell for 5 years and here’s how it was.

In the last few years, I have been (with others) developing ngless, a domain specific language and interpreter for next-generation sequencing. For partly accidental reasons, the interpreter is written in Haskell. Even though I kept using other languages (most Python and C++), I have now used Haskell quite extensively for a serious, medium-sized project (11,270 lines of code). Here are some scattered notes on Haskell:

There is a learning curve

Haskell is a different type of language. It takes a while to fully get used to it if you’re coming from…

View original post 984 more words

I tried Haskell for 5 years and here’s how it was