Learning as Stamp
Collecting Versus Diving
into Strangeness
I see two main views of learning. The first is like
stamp collecting. The person wants to collect
more and more knowledge, mostly for the
purposes of showing it off to people they want to
impress. The knowledge here is largely inert and
unimportant for their lives—it’s just a collecting
hobby accruing more facts and ideas.
There’s nothing wrong with stamp collecting.
Knowing facts and ideas, even if they aren’t
particularly useful or central to our lives, isn’t a
bad thing. It’s probably a superior hobby to
many other pursuits, since knowledge can, at
least some of the time, spillover to more
practical consequences.
The other view of learning, however, is centered
around unlearning . This is the view that what we
think we know about the world is a veneer of
sense-making atop a much deeper strangeness.
The things we think we know, we often don’t.
The ideas, philosophies and truths that guide our
lives may be convenient approximations, but
often the more accurate picture is a lot stranger
and more interesting.
Stamp collecting is more popular than diving
into strangeness. For one, it is strictly additive.
Every new trivia fact, book of the month and
water cooler topic gets added to your collection,
which you can whip out in conversations and
impress people who want to talk about them.
Diving into strangeness, in contrast, involves a
cyclical process of first undermining the things
you thought you had learned. Facts, ideas and
theories, are no longer a comforting collection,
but a temporary foothold as you leave them to
try to get to something deeper.
What is Strange?
Almost everything is much, much weirder than it
looks at first. Science is the clearest example of
this. Subatomic particles aren’t billiard balls, but
strange, complex-valued wavefunctions. Bodies
aren’t vital fluids and animating impulses, but
trillions of cells, each more complex than any
machine humans have invented. Minds aren’t
unified loci of consciousness, but the process of
countless synapses firing in incredible patterns.
Science confirms the underlying weirdness, but
for most people, knowing science is another kind
of stamp collecting. Knowing quantum
strangeness doesn’t overlap with most areas of
practical life, so it can be an additional fact or
idea one knows and can bring out in
conversations.
More interesting, for me at least, are all the
skills and knowledge that we depend on and use
everyday that have hidden weirdness beneath
them. When you remember something, did it
actually happen that way? When you give a
reason for your behavior, did reasoning have
anything to do with it? When you think that
achieving something will make you happy, will
it?
Just as science has incredible depths of
strangeness underneath, everyday life also floats
calmly upon a deeper weirdness that first
requires unlearning in order to appreciate.
Unlearning and Local Maxima
Unlearning is unpleasant for most people.
Finding out something you thought you knew was
false, or a misleading simplification, feels bad.
Since strangeness tends to predominate, and we
manage to get by in our lives without worrying
about it most of the time, why bother? Why not
just collect stamps and leave the bedrock of our
intuitions comfortably untouched?
For most people, this aversion to unlearning may
not be so bad. Skillful action exceeds skillful
knowledge, so, for most people we manage to get
by okay even if our articulated theories of the
world are out of sync with a deeper reality.
The main advantage, I see, of trying to get a
deeper picture is that it helps climb out of local
maxima. Theories can, to the extent they are
accurate, shine a light on potential things we
could do, change or experience that are outside
what we’ve experienced directly before.
Theories help us make predictions about
whether those unseen places are good places to
be or not.
A powerful algorithm for machine learning is
gradient descent. It has a complex mathematical
formulation involving vector calculus and partial
derivatives, but the intuitive picture of what it is
doing is quite simple to understand. Imagine
yourself standing at the edge of a valley. Your
goal is to get to the lowest possible spot you can.
However, the terrain is quite complex, and you
aren’t sure exactly what it looks like. What
should you do?
The gradient descent algorithm is simple: go
downhill. If you always walk in the direction of
steepest decline, you’ll eventually reach a spot
where every direction goes uphill again. This
must be a low spot in the terrain.
The problem with gradient descent is that you
can get stuck in little pockets where, to go
further downhill, you must go uphill for awhile
at first.
This is a computer analogy, but I believe that
human learning methods for acquiring many
practical skills through experience work in a
similar way. We are pushed and pulled by our
intuitions to reach a local maxima of “goodness”
in how our lives could be. Although we aren’t
always at this equilibrium, if our lives are
relatively stable, we tend to return to it.
The problem with our lives is the same as with
computers, however. Many people get “stuck” in
local maxima . The person who is addicted to
alcohol is in a local maxima . Drinking less
causes pain, to make things better, they first
have to feel worse.
Procrastination is a local maxima. Starting work
first involves pushing through an unpleasant
feeling about the task at hand. However, as
anyone who procrastinates often knows, the state
of procrastination isn’t particularly good, in an
absolute sense. It feels awful, it’s just that any
immediate action you anticipate makes you feel
a little worse than that, so you stay stuck.
What’s the connection between unlearning and
local maxima? Well one way you can get out of
local maxima is if you have some notion of what
the terrain is shaped like. If you know, for a fact,
that you are sitting in a locally optimal, but
globally awful, position, you can push against
your intuitions and accept transitional badness
in hopes of longer-term goodness.
Knowing what the terrain is shaped like,
however, depends on having an accurate picture
of the very facts and knowledge that are closest
and most fundamental to your life right now. If
those facts are wrong, your ability to make
guesses about what places further from your
immediate vicinity are actually like diminishes
rapidly. Depending on how large the local
maxima is that surrounds you, it may not be
possible to see a better future when one does
exist, or there may appear to be one which is
actually a mirage.
In many ways, unlearning has the same
properties of the local maxima problem for your
overall life situation. To get a more accurate
picture, you have to first sacrifice some certainty
in the things you take for granted. This sacrifice
involves going against your natural local-
optimization inclinations.
Strangeness, Randomness
and Unlearning
So far, I’ve spoken about one method for
overcoming the local maxima problem: having a
better theory of what unvisited places in the vast
space of possible life experiences might be like.
This helps spot genuine opportunities for
improvement and avoids mirages of hope-
inspiring, but ultimately illusory directions to
follow.
Unlearning fits into this because, unlikely with
the stamp collecting of purely additive learning,
we all have pre-existing theories of what the
terrain of nearby life spaces is already like.
Another method, however, for getting out of
local maxima is simply randomness.
Programmers often use some amount of random
motion in their gradient descent algorithms. This
randomness means that their solutions don’t
snag on relatively insignificant dips.
Human beings can use randomness too to avoid
the same problem. Exposing yourself to a larger
variety of experiences can pull you out of
temporary snags. The main disadvantage of this
approach is that randomness can sometimes be
destructive. Trying heroin, cheating on your
spouse or joining a cult may all offer unique
experiences, but their dangers may not be worth
the payoff.
Unlearning, to me, proposes a relatively safer
way of exploring larger swaths of the terrain of
life possibilities. It may create a mental
discomfort and instability, as you contend with
the fact that many of the things you took for
granted before may not be true. However, this is
often a lot less dangerous than undirected
randomness may have on your life.
To be continued...
Reference:
https://www.scotthyoung.com/blog/2018/04/12/the-art-of-unlearning/
into Strangeness
I see two main views of learning. The first is like
stamp collecting. The person wants to collect
more and more knowledge, mostly for the
purposes of showing it off to people they want to
impress. The knowledge here is largely inert and
unimportant for their lives—it’s just a collecting
hobby accruing more facts and ideas.
There’s nothing wrong with stamp collecting.
Knowing facts and ideas, even if they aren’t
particularly useful or central to our lives, isn’t a
bad thing. It’s probably a superior hobby to
many other pursuits, since knowledge can, at
least some of the time, spillover to more
practical consequences.
The other view of learning, however, is centered
around unlearning . This is the view that what we
think we know about the world is a veneer of
sense-making atop a much deeper strangeness.
The things we think we know, we often don’t.
The ideas, philosophies and truths that guide our
lives may be convenient approximations, but
often the more accurate picture is a lot stranger
and more interesting.
Stamp collecting is more popular than diving
into strangeness. For one, it is strictly additive.
Every new trivia fact, book of the month and
water cooler topic gets added to your collection,
which you can whip out in conversations and
impress people who want to talk about them.
Diving into strangeness, in contrast, involves a
cyclical process of first undermining the things
you thought you had learned. Facts, ideas and
theories, are no longer a comforting collection,
but a temporary foothold as you leave them to
try to get to something deeper.
What is Strange?
Almost everything is much, much weirder than it
looks at first. Science is the clearest example of
this. Subatomic particles aren’t billiard balls, but
strange, complex-valued wavefunctions. Bodies
aren’t vital fluids and animating impulses, but
trillions of cells, each more complex than any
machine humans have invented. Minds aren’t
unified loci of consciousness, but the process of
countless synapses firing in incredible patterns.
Science confirms the underlying weirdness, but
for most people, knowing science is another kind
of stamp collecting. Knowing quantum
strangeness doesn’t overlap with most areas of
practical life, so it can be an additional fact or
idea one knows and can bring out in
conversations.
More interesting, for me at least, are all the
skills and knowledge that we depend on and use
everyday that have hidden weirdness beneath
them. When you remember something, did it
actually happen that way? When you give a
reason for your behavior, did reasoning have
anything to do with it? When you think that
achieving something will make you happy, will
it?
Just as science has incredible depths of
strangeness underneath, everyday life also floats
calmly upon a deeper weirdness that first
requires unlearning in order to appreciate.
Unlearning and Local Maxima
Unlearning is unpleasant for most people.
Finding out something you thought you knew was
false, or a misleading simplification, feels bad.
Since strangeness tends to predominate, and we
manage to get by in our lives without worrying
about it most of the time, why bother? Why not
just collect stamps and leave the bedrock of our
intuitions comfortably untouched?
For most people, this aversion to unlearning may
not be so bad. Skillful action exceeds skillful
knowledge, so, for most people we manage to get
by okay even if our articulated theories of the
world are out of sync with a deeper reality.
The main advantage, I see, of trying to get a
deeper picture is that it helps climb out of local
maxima. Theories can, to the extent they are
accurate, shine a light on potential things we
could do, change or experience that are outside
what we’ve experienced directly before.
Theories help us make predictions about
whether those unseen places are good places to
be or not.
A powerful algorithm for machine learning is
gradient descent. It has a complex mathematical
formulation involving vector calculus and partial
derivatives, but the intuitive picture of what it is
doing is quite simple to understand. Imagine
yourself standing at the edge of a valley. Your
goal is to get to the lowest possible spot you can.
However, the terrain is quite complex, and you
aren’t sure exactly what it looks like. What
should you do?
The gradient descent algorithm is simple: go
downhill. If you always walk in the direction of
steepest decline, you’ll eventually reach a spot
where every direction goes uphill again. This
must be a low spot in the terrain.
The problem with gradient descent is that you
can get stuck in little pockets where, to go
further downhill, you must go uphill for awhile
at first.
This is a computer analogy, but I believe that
human learning methods for acquiring many
practical skills through experience work in a
similar way. We are pushed and pulled by our
intuitions to reach a local maxima of “goodness”
in how our lives could be. Although we aren’t
always at this equilibrium, if our lives are
relatively stable, we tend to return to it.
The problem with our lives is the same as with
computers, however. Many people get “stuck” in
local maxima . The person who is addicted to
alcohol is in a local maxima . Drinking less
causes pain, to make things better, they first
have to feel worse.
Procrastination is a local maxima. Starting work
first involves pushing through an unpleasant
feeling about the task at hand. However, as
anyone who procrastinates often knows, the state
of procrastination isn’t particularly good, in an
absolute sense. It feels awful, it’s just that any
immediate action you anticipate makes you feel
a little worse than that, so you stay stuck.
What’s the connection between unlearning and
local maxima? Well one way you can get out of
local maxima is if you have some notion of what
the terrain is shaped like. If you know, for a fact,
that you are sitting in a locally optimal, but
globally awful, position, you can push against
your intuitions and accept transitional badness
in hopes of longer-term goodness.
Knowing what the terrain is shaped like,
however, depends on having an accurate picture
of the very facts and knowledge that are closest
and most fundamental to your life right now. If
those facts are wrong, your ability to make
guesses about what places further from your
immediate vicinity are actually like diminishes
rapidly. Depending on how large the local
maxima is that surrounds you, it may not be
possible to see a better future when one does
exist, or there may appear to be one which is
actually a mirage.
In many ways, unlearning has the same
properties of the local maxima problem for your
overall life situation. To get a more accurate
picture, you have to first sacrifice some certainty
in the things you take for granted. This sacrifice
involves going against your natural local-
optimization inclinations.
Strangeness, Randomness
and Unlearning
So far, I’ve spoken about one method for
overcoming the local maxima problem: having a
better theory of what unvisited places in the vast
space of possible life experiences might be like.
This helps spot genuine opportunities for
improvement and avoids mirages of hope-
inspiring, but ultimately illusory directions to
follow.
Unlearning fits into this because, unlikely with
the stamp collecting of purely additive learning,
we all have pre-existing theories of what the
terrain of nearby life spaces is already like.
Another method, however, for getting out of
local maxima is simply randomness.
Programmers often use some amount of random
motion in their gradient descent algorithms. This
randomness means that their solutions don’t
snag on relatively insignificant dips.
Human beings can use randomness too to avoid
the same problem. Exposing yourself to a larger
variety of experiences can pull you out of
temporary snags. The main disadvantage of this
approach is that randomness can sometimes be
destructive. Trying heroin, cheating on your
spouse or joining a cult may all offer unique
experiences, but their dangers may not be worth
the payoff.
Unlearning, to me, proposes a relatively safer
way of exploring larger swaths of the terrain of
life possibilities. It may create a mental
discomfort and instability, as you contend with
the fact that many of the things you took for
granted before may not be true. However, this is
often a lot less dangerous than undirected
randomness may have on your life.
To be continued...
Reference:
https://www.scotthyoung.com/blog/2018/04/12/the-art-of-unlearning/
No comments:
Post a Comment