The Reason I made the Beauties World Map (Japanese Edition)
The first story, the story of Google Translate, takes place in Mountain View over nine months, and it explains the transformation of machine translation. The second story, the story of Google Brain and its many competitors, takes place in Silicon Valley over five years, and it explains the transformation of that entire community.
The third story, the story of deep learning, takes place in a variety of far-flung laboratories — in Scotland, Switzerland, Japan and most of all Canada — over seven decades, and it might very well contribute to the revision of our self-image as first and foremost beings who think. All three are stories about artificial intelligence. The seven-decade story is about what we might conceivably expect or want from it. The five-year story is about what it might do in the near future. The nine-month story is about what it can do right this minute.
These three stories are themselves just proof of concept. All of this is only the beginning. Jeff Dean, though his title is senior fellow, is the de facto head of Google Brain.
Dean is a sinewy, energy-efficient man with a long, narrow face, deep-set eyes and an earnest, soapbox-derby sort of enthusiasm. The son of a medical anthropologist and a public-health epidemiologist, Dean grew up all over the world — Minnesota, Hawaii, Boston, Arkansas, Geneva, Uganda, Somalia, Atlanta — and, while in high school and college, wrote software used by the World Health Organization.
He has been with Google since , as employee 25ish, and has had a hand in the core software systems beneath nearly every significant undertaking since then. Ng told him about Project Marvin, an internal effort named after the celebrated A. Now, over the previous five years, the number of academics working on neural networks had begun to grow again, from a handful to a few dozen.
Pretty soon, he suggested to Ng that they bring in another colleague with a neuroscience background, Greg Corrado. In graduate school, Corrado was taught briefly about the technology, but strictly as a historical curiosity. By then, a number of the Google engineers had taken to referring to Project Marvin by another name: Google Brain. If you wanted to translate from English to Japanese, for example, you would program into the computer all of the grammatical rules of English, and then the entirety of definitions contained in the Oxford English Dictionary, and then all of the grammatical rules of Japanese, as well as all of the words in the Japanese dictionary, and only after all of that feed it a sentence in a source language and ask it to tabulate a corresponding sentence in the target language.
You would give the machine a language map that was, as Borges would have had it, the size of the territory. There are two main problems with the old-fashioned approach. The second is that it only really works in domains where rules and definitions are very clear: in mathematics, for example, or chess. Translation, however, is an example of a field where this approach fails horribly, because words cannot be reduced to their dictionary definitions, and because languages tend to have as many exceptions as they have rules. There were, however, limits to what this system could do.
In the s, a robotics researcher at Carnegie Mellon pointed out that it was easy to get computers to do adult things but nearly impossible to get them to do things a 1-year-old could do, like hold a ball or identify a cat. There has always been another vision for A. This notion dates to the early s, when it occurred to researchers that the best model for flexible automated intelligence was the brain itself.
This structure, in its simplicity, has afforded the brain a wealth of adaptive advantages. The brain can operate in circumstances in which information is poor or missing; it can withstand significant damage without total loss of control; it can store a huge amount of knowledge in a very efficient way; it can isolate distinct patterns but retain the messiness necessary to handle ambiguity. They could also, at least in theory, learn the way we do.
An artificial neural network could do something similar, by gradually altering, on a guided trial-and-error basis, the numerical relationships among artificial neurons. It would, instead, rewire itself to reflect patterns in the data it absorbed. This attitude toward artificial intelligence was evolutionary rather than creationist.
If you wanted a flexible mechanism, you wanted one that could adapt to its environment. You wanted to begin with very basic abilities — sensory perception and motor control — in the hope that advanced skills would emerge organically. Google Brain was the first major commercial institution to invest in the possibilities embodied by this way of thinking about A. Dean, Corrado and Ng began their work as a part-time, collaborative experiment, but they made immediate progress. We were sitting, as usual, in a whiteboarded meeting room, on which he had drawn a crowded, snaking timeline of Google Brain and its relation to inflection points in the recent history of neural networks.
We can build them around the capabilities that now exist to understand photos. Robots will be drastically transformed. Its speech-recognition team swapped out part of their old system for a neural network and encountered, in pretty much one fell swoop, the best quality improvements anyone had seen in 20 years. It was because Google had finally devoted the resources — in computers and, increasingly, personnel — to fill in outlines that had been around for a long time.
A great preponderance of these extant and neglected notions had been proposed or refined by a peripatetic English polymath named Geoffrey Hinton. Ng now leads the 1,person A. Hinton wanted to leave his post at the University of Toronto for only three months, so for arcane contractual reasons he had to be hired as an intern. I took your course! What are you doing here?
Heritage of Japan | Discovering the Historical Context and Culture of the People of Japan
A few months later, Hinton and two of his students demonstrated truly astonishing gains in a big image-recognition contest, run by an open-source collective called ImageNet, that asks computers not only to identify a monkey but also to distinguish between spider monkeys and howler monkeys, and among God knows how many different breeds of cat. Google soon approached Hinton and his students with an offer. They accepted. Hinton comes from one of those old British families emblazoned like the Darwins at eccentric angles across the intellectual landscape, where regardless of titular preoccupation a person is expected to make sideline contributions to minor problems in astronomy or fluid dynamics.
He trained at Cambridge and Edinburgh, then taught at Carnegie Mellon before he ended up at Toronto, where he still spends half his time. His work has long been supported by the largess of the Canadian government. I visited him in his office at Google there. He has tousled yellowed-pewter hair combed forward in a mature Noel Gallagher style and wore a baggy striped dress shirt that persisted in coming untucked, and oval eyeglasses that slid down to the tip of a prominent nose.
Hinton had been working on neural networks since his undergraduate days at Cambridge in the late s, and he is seen as the intellectual primogenitor of the contemporary field. For most of that time, whenever he spoke about machine learning, people looked at him as though he were talking about the Ptolemaic spheres or bloodletting by leeches. Neural networks were taken as a disproven folly, largely on the basis of one overhyped project: the Perceptron, an artificial neural network that Frank Rosenblatt, a Cornell psychologist, developed in the late s.
He was also competing for Defense Department funding. Along with an M.
2-Minute Table Top: $456 per Map
But Hinton already knew at the time that complex tasks could be carried out if you had recourse to multiple layers. With one layer, you could find only simple patterns; with more than one, you could look for patterns of patterns. Each successive layer of the network looks for a pattern in the previous layer. A pattern of edges might be a circle or a rectangle. A pattern of circles or rectangles might be a face. And so on. This more or less parallels the way information is put together in increasingly abstract ways as it travels from the photoreceptors in the retina back and up through the visual cortex.
How do you begin to correct the child? You cannot just repeat your initial instructions, because the child does not know at which point he went wrong. Hinton and a few others went on to invent a solution or rather, reinvent an older one to this layered-error problem, over the halting course of the late s and s, and interest among computer scientists in neural networks was briefly revived. It was true within artificial intelligence. An average brain has something on the order of billion neurons.
Each neuron is connected to up to 10, other neurons, which means that the number of synapses is between trillion and 1, trillion. For a simple artificial neural network of the sort proposed in the s, the attempt to even try to replicate this was unimaginable. To understand why scale is so important, however, you have to start to understand some of the more technical details of what, exactly, machine intelligences are doing with the data they consume.
A lot of our ambient fears about A. If that brief explanation seems sufficiently reassuring, the reassured nontechnical reader is invited to skip forward to the next section, which is about cats. If not, then read on. This section is also, luckily, about cats. Imagine you want to program a cat-recognizer on the old symbolic-A. All this information is stored in a special place in memory called Cat. Now you show it a picture. First, the machine has to separate out the various distinct elements of the image.
Then it has to take these elements and apply the rules stored in its memory. But what if you showed this cat-recognizer a Scottish Fold, a heart-rending breed with a prized genetic defect that leads to droopy doubled-over ears? Our symbolic A. On one side of the blob, you present the inputs the pictures ; on the other side, you present the corresponding outputs the labels.
- Prey of the Goat?
- To Act and Let go;
- How many of these 39 natural wonders of the world have you been to?;
Then you just tell it to work out for itself, via the individual calibration of all of these interconnected switches, whatever path the data should take so that the inputs are mapped to the correct outputs. The training is the process by which a labyrinthine series of elaborate tunnels are excavated through the blob, tunnels that connect any given input to its proper output. The more training data you have, the greater the number and intricacy of the tunnels that can be dug. Once the training is complete, the middle of the blob has enough tunnels that it can make reliable predictions about how to handle data it has never seen before.
The reason that the network requires so many neurons and so much data is that it functions, in a way, like a sort of giant machine democracy. Imagine you want to train a computer to differentiate among five different items. If you have only Joe, Frank and Mary, you can maybe use them only to differentiate among a cat, a dog and a defibrillator. If you have millions of different voters that can associate in billions of different ways, you can learn to classify data with incredible granularity.
Your trained voter assembly will be able to look at an unlabeled picture and identify it more or less accurately. It just knows them when it sees them.
- Sous la lune (French Edition).
- Belle Jingle A Christmas Miracle.
- Reggie the Veggie;
- An odessey to the south!
- Georgia Aquarium Visitor Information.
- How to Hang a Door (Doc Handys Home Repair & Improvement Series Book 1).
- Top 10 Natural Wonders in Japan | Places To See In Your Lifetime;
This wooliness, however, is the point. You just need lots and lots of the voters — in order to make sure that some part of your network picks up on even very weak regularities, on Scottish Folds with droopy ears, for example — and enough labeled data to make sure your network has seen the widest possible variance in phenomena. Supervised learning is a trial-and-error process based on labeled data. The machines might be doing the learning, but there remains a strong human element in the initial categorization of the inputs. Labeled data is thus fallible the way that human labelers are fallible.
If a machine was asked to identify creditworthy candidates for loans, it might use data like felony convictions, but if felony convictions were unfair in the first place — if they were based on, say, discriminatory drug laws — then the loan recommendations would perforce also be fallible.
- Trash Divorce.
- Mount Koya.
- Communist Architecture of Skopje, Macedonia – A Brutal, Modern, Cosmic, Era.
- Piece of Cake Paleo - Bread and Slow Cooker Recipes?
Image-recognition networks like our cat-identifier are only one of many varieties of deep learning, but they are disproportionately invoked as teaching examples because each layer does something at least vaguely recognizable to humans — picking out edges first, then circles, then faces. They threw into the training mix some photos of solo barbells. The problem was solved. Not everything is so easy. They still had fewer than 10 people and only a vague sense for what might ultimately come of it all. But even then they were thinking ahead to what ought to happen next. First a human mind learns to recognize a ball and rests easily with the accomplishment for a moment, but sooner or later, it wants to ask for the ball.
And then it wades into language. The first step in that direction was the cat paper , which made Brain famous. The machine had not been programmed with the foreknowledge of a cat; it reached directly into the world and seized the idea for itself. The researchers discovered this with the neural-network equivalent of something like an M. Most machine learning to that point had been limited by the quantities of labeled data.
The cat paper showed that machines could also deal with raw unlabeled data, perhaps even data of which humans had no established foreknowledge. This seemed like a major advance not only in cat-recognition studies but also in overall artificial intelligence. The lead author on the cat paper was Quoc Le. Le is short and willowy and soft-spoken, with a quick, enigmatic smile and shiny black penny loafers. He grew up outside Hue, Vietnam. His parents were rice farmers, and he did not have electricity at home. His mathematical abilities were obvious from an early age, and he was sent to study at a magnet school for science.
In the late s, while still in school, he tried to build a chatbot to talk to. He thought, How hard could this be? He left the rice paddies on a scholarship to a university in Canberra, Australia, where he worked on A. The dominant method of the time, which involved feeding the machine definitions for things like edges, felt to him like cheating.
In a reading group there, he encountered two new papers by Geoffrey Hinton. People who entered the discipline during the long diaspora all have conversion stories, and when Le read those papers, he felt the scales fall away from his eyes. There was a successful paradigm, but to be honest I was just curious about the new paradigm. In , there was very little activity. What happened, soon afterward, was that Le went to Brain as its first intern, where he carried on with his dissertation work — an extension of which ultimately became the cat paper.
On a simple level, Le wanted to see if the computer could be trained to identify on its own the information that was absolutely essential to a given image. He fed the neural network a still he had taken from YouTube. The machine threw away some of the information, initially at random. Now recreate the initial image you were shown based only on the information you retained. Its reaction would be akin to that of a distant ancestor whose takeaway from his brief exposure to saber-tooth tigers was that they made a restful swooshing sound when they moved.
A neural network, however, was a black box. The same network that hit on our concept of cat also became enthusiastic about a pattern that looked like some sort of furniture-animal compound, like a cross between an ottoman and a goat. After the cat paper, he realized that if you could ask a network to summarize a photo, you could perhaps also ask it to summarize a sentence.
This problem preoccupied Le, along with a Brain colleague named Tomas Mikolov, for the next two years. In that time, the Brain team outgrew several offices around him. For a while they were on a floor they shared with executives. It unsettled incoming V. Le had never seemed so solemn. They spent this period trying to come up with neural-network architectures that could accommodate not only simple photo classifications, which were static, but also complex structures that unfolded over time, like language or music. Many of these were first proposed in the s , and Le and his colleagues went back to those long-ignored contributions to see what they could glean.
They knew that once you established a facility with basic linguistic prediction, you could then go on to do all sorts of other intelligent things — like predict a suitable reply to an email, for example, or predict the flow of a sensible conversation. You could sidle up to the sort of prowess that would, from the outside at least, look a lot like thinking. The hundred or so current members of Brain — it often feels less like a department within a colossal corporate hierarchy than it does a club or a scholastic society or an intergalactic cantina — came in the intervening years to count among the freest and most widely admired employees in the entire Google organization.
Their microkitchen has a foosball table I never saw used; a Rock Band setup I never saw used; and a Go kit I saw used on a few occasions. I did once see a young Brain research associate introducing his colleagues to ripe jackfruit, carving up the enormous spiky orb like a turkey. When I first visited, parking was not an issue. The closest spaces were those reserved for expectant mothers or Teslas, but there was ample space in the rest of the lot. By October, if I showed up later than , I had to find a spot across the street. At a certain point he did some back-of-the-envelope calculations, which he presented to the executives one day in a two-slide presentation.
There was, however, another option: just design, mass-produce and install in dispersed data centers a new kind of chip to make everything faster. These chips would be called T. Rather than compute It usually works to speed up one thing. But because of the generality of neural networks, you can leverage this special-purpose hardware for a lot of other things.
Just as the chip-design process was nearly complete, Le and two colleagues finally demonstrated that neural networks might be configured to handle the structure of language. When you summarize images, you can divine a picture of what each stage of the summary looks like — an edge, a circle, etc. When you summarize language in a similar way, you essentially produce multidimensional maps of the distances, based on common usage, between one word and every single other word in the language. Instead, it is shifting and twisting and warping the words around in the map.
In two dimensions, you cannot make this map useful. It can be related to all these other words simultaneously only if it is related to each of them in a different dimension. Le gave me a good-natured hard time for my continual requests for a mental picture of these maps. Still, certain dimensions in the space, it turned out, did seem to represent legible human categories, like gender or relative size.
You just had to give it millions and millions of English sentences as inputs on one side and their desired French outputs on the other, and over time it would recognize the relevant patterns in words the way that an image classifier recognized the relevant patterns in pixels. You could then give it a sentence in English and ask it to predict the best French analogue. The major difference between words and pixels, however, is that all of the pixels in an image are there at once, whereas words appear in a progression over time.
In a period of about a week, in September , three papers came out — one by Le and two others by academics in Canada and Germany — that at last provided all the theoretical tools necessary to do this sort of thing. It also cleared the way toward an instrumental task like machine translation. Hinton told me he thought at the time that this follow-up work would take at least five more years. Small for Google, that is — it was actually the biggest public data set in the world. A decade of the old Translate had gathered production data that was between a hundred and a thousand times bigger.
Mike Schuster, who then was a staff research scientist at Brain, picked up the baton. The project took him the next two years. Schuster is a taut, focused, ageless being with a tanned, piston-shaped head, narrow shoulders, long camo cargo shorts tied below the knee and neon-green Nike Flyknits. In the s, he ran experiments with a neural-networking machine as big as a conference room; it cost millions of dollars and had to be trained for weeks to do something you could now do on your desktop in less than an hour.
He published a paper in that was barely cited for a decade and a half; this year it has been cited around times. He is not humorless, but he does often wear an expression of some asperity, which I took as his signature combination of German restraint and Japanese restraint. The issues Schuster had to deal with were tangled. Hughes was eating alone, and the two Brain members took positions at either side. They told Hughes that seemed like a good time to consider an overhaul of Google Translate — the code of hundreds of engineers over 10 years — with a neural network.
The old system worked the way all machine translation has worked for about 30 years: It sequestered each successive sentence fragment, looked up those words in a large statistically derived vocabulary table, then applied a battery of post-processing rules to affix proper endings and rearrange it all to make sense. It would capture context — and something akin to meaning. The stakes may have seemed low: Translate generates minimal revenue, and it probably always will. But there was a case to be made that human-quality machine translation is not only a short-term necessity but also a development very likely, in the long term, to prove transformational.
If Google was going to compete in China — where a majority of market share in search-engine traffic belonged to its competitor Baidu — or India, decent machine translation would be an indispensable part of the infrastructure. Baidu itself had published a pathbreaking paper about the possibility of neural machine translation in July And in the more distant, speculative future, machine translation was perhaps the first step toward a general computational facility with human language. This would represent a major inflection point — perhaps the major inflection point — in the development of something that felt like true artificial intelligence.
Most people in Silicon Valley were aware of machine learning as a fast-approaching horizon, so Hughes had seen this ambush coming. Raccoons tear open garbage and knock over compost bins. They damage buildings and gardens. They carry diseases, including rabies and distemper. They even, on rare occasions, attack people and their pets.
So raccoons are cute, but they can also be troublesome neighbors. After all, raccoons were around long before human beings. Well, yes… in North America. But elsewhere? Raccoons are indeed found in a few other places, namely Europe especially Germany , the Caucasus region, and you guessed it!
This term denotes an organism that, due to human activity, inhabits a region to which it is not indigenous. Sometimes we even bring them along on purpose, as crops, farm animals, or pets. A few, however, manage to hang on and consistently reproduce. Some even find themselves at a sudden advantage, often due to lack of natural predators. An infamous example is the brown rat, originating in China and carried across the world, primarily by seafaring Europeans.
The effect was often devastating, notably on bird populations in regions where land predators were hitherto absent, such as remote islands. Indeed, remote islands seem to have experienced the brunt of invasive-species carnage, given their sheer evolutionary isolation. The most famous victim of species invasion may be Australia, whose vegetation has been ravaged by the introduction of rats and rabbits, while native animals have been gobbled up by wildcats and foxes the latter having been introduced to control the rabbits.
Cane toads, which were introduced to control crop-destroying beetles, ended up slurping down crop-beneficial insects, as well as killing off indigenous predators that are poisoned by their toxic secretions when they eat one. Pretty much every part of the world has its invasive species issues, though. Many native species of fish in the African Great Lakes have been supplanted by introduced fish. The North American Great Lakes struggle with invasive mussels and eels.
The storks and small mammals of Florida are being snapped up by released pythons, which are even out-competing alligators for food. So what about Japan? Meanwhile, native fish struggle to dodge the voracious mouths of introduced bass and bluegill. Someone protect the emperor!
Men's Standards Of Beauty Around The World
Japan also has issues with animals originally imported as pets. Through a combination of escape and deliberate release, several foreign species have come to form wild populations. Hedgehogs, red-eared turtles, and ferrets are three prominent examples. Another is the raccoon. This is one of those rare instances where a major environmental change can be traced back to one individual.
And, even rarer, a fictional individual. The series, set in early twentieth-century rural Wisconsin, follows the adventures of a young boy who rescues a baby raccoon orphaned by a hunter. Rascal, as the boy names his new friend, proves a loving companion. Rascal becomes a crucial source of comfort when, not long after his arrival, the boy loses his mother. When Rascal is caught snacking on the crops of neighboring farmers, his outdoor ramblings are suddenly reduced to the confines of a pen. Ultimately, the boy faces a very difficult, emotional choice…where does Rascal truly belong?
With the massive success of this anime, suddenly everyone wanted a raccoon for a pet. Over the following years, upwards of two thousand raccoons were imported to Japan annually. It was a pretty sweet deal for the raccoons, too, given that in Japan they lack any significant natural predators. Over the ensuing decades, the raccoon population soared, and Rascal established himself in nearly every part of the country. What was good for the raccoons was bad for a range of native species, notably birds whose eggs, as noted earlier, are a staple of the raccoon diet.
In human-inhabited areas, raccoons have been scavenging garbage, damaging crops notably corn and melon fields , and even attacking pets. Which brings us to perhaps the most infamous raccoon issue in Japan: temple damage. Architectural wounds are inflicted as the raccoons climb around, leaving gashes in pillars and walls, and punching holes in roofs and ceilings. As they find themselves snug little corners to sleep in, they tear and pry at anything that stands in their way, be it wood, tile, insulation, pipes, or wires.
And they naturally leave lots of little raccoon droppings around, which are inappropriate for most buildings, but especially temples. Traps were set, and metal fencing was laid over potential points of entry. The first approach is the most immediate. In Japan, as in other raccoony parts of the world, laborious attempts are made at sealing buildings from entry particularly in the case of historic properties , while garbage is locked away from those dexterous little hands.