#i know neural nets run on computers technically

[cugzarui]

start new trend of calling things what they are:

LLM, machine learning, perceptrons, linear algebra, statistical model

automaton, automation, NPC, machine, computer.

[mugasofer]

It seems like many, perhaps most, people historically believed in some immanent apocalypse.

Many philosophies claim that the world is passing into a degenerate age of chaos (Ages of Man, Kali Yuga, life-cycle of civilisation), or divine conflict will shortly spill over & destroy the Earth (Ragnorok, Revelations, Zoroastrian Frashokereti), or that the natural forces sustaining us must be transient.

Yet few panic or do anything. What anyone does "do about it" is often symbolic & self-admittedly unlikely to do much.

Maybe humans evolved not to care, to avoid being manipulated?

Many cults make similar claims, and do uproot their lives around them. Even very rarely committing mass suicide or terror attacks etc on occasion. But cults exist that don't make such claims, so it may not be the mechanism they use to control, or at most a minor one. "This is about the fate of the whole world, nothing can be more important than that, so shut up" may work as as a thought terminating cliche, but it doesn't seem to work that strongly, and there are many at least equally effective ones.

Some large scale orgs do exist that seem to take their eschatology "seriously". The Aztecs committed atrocities trying to hold off apocalypse, ISIS trying to cause it. Arguably some Communist or even fascist groups count, depending on your definition of apocalypse.

But even then, one can argue their actions are not radically different from non-apocalypse-motivated ones - e.g. the Aztecs mass-executed less per capita than the UK did at times & some historians view them as more about displaying authority.

I'm thinking about this because of two secular eschatologies - climate apocalypse and the Singularity.

My view on climate change, which as far as I can tell is the scientific consensus, is that it is real and bad but by no means apocalyptic. We're talking incremental increases in storms, droughts, floods etc, all of which are terrible, but none of which remotely threaten human civilisation. E.g. according to the first Google result, the sea is set to rise by 1 decimeter by 2100 in a "high emissions scenario", not to rise by tens or hundreds of meters and consume all coastal nations as I was taught as a child. Some more drastic projections suggest that the sea might rise by as much as two or three meters in the worst case scenario.

It really creeps me out when I hear people who confess to believe that human civilisation, the human species, or even all life on Earth is most likely going to be destroyed soon by climate change. The most recent example, which prompted this post, was the Call of Cthulhu podcast I was listening to casually suggesting that it might be a good idea to summon an Elder God of ice and snow to combat climate change as the "lesser existential risk", perhaps by sacrificing "climate skeptics" to it. It's incredibly jarring for me to realise that the guys I've been listening to casually chatting about RPGs think they live in a world that will shortly be ended by the greed of it's rulers. But this idea is everywhere. Discussions of existential risks from e.g. pandemics inevitably attract people arguing that the real existential risk is climate change. A major anti-global-warming protest movement, Extinction Rebellion, is literally named after the idea that they're fighting against their own extinction. Viral Tumblr posts talk about how the fear of knowing that the world is probably going to be destroyed soon by climate change and fascism is crippling their mental health, and they have no idea how to deal with it because it's all so real.

But it's not. It's not real.

Well, I can't claim that political science is accurate enough for me to definitively say that fascism isn't going to take over, but I can say that climate science is fairly accurate and it predicts that the world is definitely not about to end in fire or in flood.

(There are valid arguments that climate change or other environmental issues might precipitate wars, which could turn apocalyptic due to nuclear weapons; or that we might potentially encounter a black swan event due to our poor understanding of the ecosystem and climate-feedback systems. But these are very different, as they're self-admittedly "just" small risks to the world.)

And I get the impression that a lot of people with more realistic views about climate change deliberately pander to this, deliberately encouraging people to believe that they're going to die because it puts them on the "right side of the issue". The MCU's Loki, for instance, recently casually brought up a "climate apocalypse" in 2050, which many viewers took as meaning the world ending. Technically, the show uses a broad definition of "apocalypse" - Pompeii is given as another example - and it kind of seems like maybe all they meant was natural disasters encouraged by climate change, totally defensible. But I still felt kinda mad about it, that they're deliberately pandering to an idea which they hopefully know is false and which is causing incredible anxiety in people. I remember when Greta Thurnberg was a big deal, I read through her speeches to Extinction Rebellion, and if you parsed them closely it seemed like she actually did have a somewhat realistic understanding of what climate change is. But she would never come out and say it, it was all vague implications of doom, which she was happily giving to a rally called "Extinction Rebellion" filled with speakers who were explicitly stating, not just coyly implying, that this was a fight for humanity's survival against all the great powers of the world.

But maybe there's nothing wrong with that. I despise lying, but as I've been rambling about, this is a very common lie that most people somehow seem unaffected by. Maybe the viral tumblr posts are wrong about the source of their anxiety; maybe it's internal/neurochemical and they world just have picked some other topic to project their anxieties on if this particular apocalypse wasn't available. Maybe this isn't a particularly harmful lie, and it's hypocritical of me to be shocked by those who believe it.

Incidentally, I believe the world is probably going to end within the next fifty years.

Intellectually, I find the arguments that superhuman AI will destroy the world pretty undeniable. Sure, forecasting the path of future technology is inherently unreliable. But the existence of human brains, some of which are quite smart, proves pretty conclusively it's possible to get lumps of matter to think - and human brains are designed to run on the tiny amounts of energy they can get by scavenging plants and the occasional scraps of meat in the wilderness as fuel, with chemical signals that propagate at around the speed of sound (much slower than electronic ones), with only the data they can get from input devices they carry around with them, and which break down irrevocably after a few decades. And while we cannot necessarily extrapolate from the history of progress in both computer hardware and AI, that progress is incredibly impressive, and there's no particular reason to believe it will fortuitously stop right before we manufacture enough rope to hang ourselves.

Right now, at time of writing, we have neural nets that can write basic code, appear to scale linearly in effectiveness with the available hardware with no signs that we're reaching their limit, and have not yet been applied at the current limits of available hardware let alone what will be available in a few years. They absorb information like a sponge at a vastly superhuman speed and scale, allowing them to be trained in days or hours rather than the years or decades humans require. They are already human-level or massively superhuman at many tasks, and are capable of many things I would have confidently told you a few years ago were probably impossible without human-level intelligence, like the crazy shit AI dungeon is capable of. People are actively working on scaling them up so that they can work on and improve the sort of code they are made from. And we have no ability to tell what they're thinking or control them without a ton of trial and error.

If you follow this blog, you're probably familiar with all the above arguments for why we're probably very close to getting clobbered by superhuman AI, and many more, as well as all the standard counter-arguments and the counter-arguments to those counter arguments.

(Note: I do take some comfort in God, but even if my faith were so rock solid that I would cheerfully bet the world on it - which it's not - there's no real reason why our purpose in God's plan couldn't be to destroy ourselves or be destroyed as an object lesson to some other, more important civilization. There's ample precedent.)

Here's the thing: I'm not doing anything about it, unless you count occasionally, casually talking about it with people online. I'm not even donating to help any of the terrifyingly-few people who are trying to do something about it. Part of why I'm not contributing is, frankly, I don't have a clue what to do, nor do I have much confidence in any of the stuff people are currently doing (although I bloody well hope some of it works.)

And yet I don't actually feel that scared.

I feel more of a visceral chill reading about the nuclear close calls that almost destroyed the world in the recent past than thinking about the stuff that has a serious chance of doing so in a few decades. I'm a neurotic mess, and yet what is objectively the most terrifying thing on my radar does not actually seem to contribute to my neurosis.

[aiweirdness]

Escape rooms

Now that so many of us are spending so much time in our own homes, the thought of being stuck in a room is very much on our minds. If you’ve ever done an escape room, you know that you can pay to be stuck in a room - except that you get to choose your own form of peril and there’s generally less time spent baking and watching shows. In case we run out of ways to be trapped in a room, Jesse Morris, an actual escape room designer, sent me the names of about 1100 existing escape rooms so I could train a neural net to generate more.

I gave all 1100 escape room names to the 124M size of GPT-2 and trained it for literally just a few seconds (if I gave it more training time it would probably have memorized all the examples, since that’s technically a solution to “generate names like these”).

The neural net did pretty well at escape rooms! It could draw not only on the names themselves, but also on other things it had seen during its general internet training. (And sometimes it seemed like it thought it was doing movie titles or computer game levels). I had to check to make sure it didn’t copy these from the training data:

The Forgotten Castle Scarlet Room The Silent Chamber

It did its best to make spooky rooms, although there’s something off about these maybe:

Maw of the Ice Throne Malvo's Death Star Cryopod Void Bomb The Floor is Haunted The Tortoise and the Crypt Nuclear Zombies Nursery

And these escape rooms just sound weird:

The Cat Abounds The Mail Dragon I've found Grandpa's 2.0 Bison Countdown to the Pizza Shop Chocolate Truck The Elevator Belongs to Me Three Dogs and a Bison Forgotten Pharaohs Pet Shop The Room with a Chance of Being in it

These sound commonplace in a frankly rather unsettling manner.

Cat House Cake Shop Sausage Experience Cabinet of Sainsbury Adventure Cutthroat Tea House The Body Shop Escape from Wonderful World of Gumball

I’m super curious about these celebrity/pop culture escape rooms:

Miss Piggy's Curse The Haunted: The Lorax Mr. T's Clocks The X-Files: Tee Time Miss A.I.'s Deco Room

For the images above, I used GPT-2-simple to prompt a default instance of GPT-2-778M with a couple of example escape room descriptions, plus the name of the escape room I wanted it to generate a description for. It figured out how to copy the format, which I found super impressive. Then to generate creepy images to go with the descriptions, I used artbreeder to combine different scenes. “Scarlet Room”, for example, is “vault” plus “theater curtain” plus “prison” plus “butcher shop”.

Subscribers get bonus content: I generated more escape rooms and descriptions than would fit in this blog post.

My book on AI is out, and, you can now get it any of these several ways! Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s - Boulder Bookstore

[unheardradical]

How an AI Took Over an Adult Knitting...

SkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... SkyKnit: How an AI Took Over an Adult Knitting Community Ribald knitters teamed up with a neural-network creator to generate new types of tentacled, cozy shapes. A SkyKnit design as interpreted by michaela112358, a Ravelry user Ravelry / michaela112358 ALEXIS C. MADRIGAL | MAR 6, 2018 | TECHNOLOGY Like The Atlantic? Subscribe to The Atlantic Daily, our free weekday email newsletter. Email SIGN UP Janelle Shane is a humorist who creates and mines her material from 1 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... neural networks, the form of machine learning that has come to dominate the field of artificial intelligence over the last half-decade. Perhaps you’ve seen the candy-heart slogans she generated for Valentine’s Day: DEAR ME, MY MY, LOVE BOT, CUTE KISS, MY BEAR, and LOVE BUN. Or her new paint-color names: Parp Green, Shy Bather, Farty Red, and Bull Cream. Or her neural-net-generated Halloween costumes: Punk Tree, Disco Monster, Spartan Gandalf, Starfleet Shark, and A Masked Box. Her latest project, still ongoing, pushes the joke into a new, physical realm. Prodded by a knitter on the knitting forum Ravelry, Shane trained a type of neural network on a series of over 500 sets of knitting instructions. Then, she generated new instructions, which members of the Ravelry community have actually attempted to knit. “The knitting project has been a particularly fun one so far just because it ended up being a dialogue between this computer program and these knitters that went over my head in a lot of ways,” Shane told me. “The computer would spit out a whole bunch of instructions that I couldn’t read and the knitters would say, this is the funniest thing I’ve ever read.” The human-machine collaboration created configurations of yarn that you probably wouldn’t give to your in-laws for Christmas, but they were interesting. The user citikas was the first to post a try at one of the earliest patterns, “reverss shawl.” It was strange, but it did have some charisma. 2 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... Shane nicknamed the whole effort “Project Hilarious Disaster.” The community called it SkyKnit. The first yarn product of SkyKnit, by the Ravelry user citikas (Ravelry / citikas) The idea of using neural networks to do computer things has been around for decades. But it took until the last 10 years or so for the right mix of techniques, data sets, chips, and computing power to transform neural networks into deployable technical tools. There are many different kinds suited to different sorts of tasks. Some translate between different languages for Google. Others automatically label pictures. Still others are part of what powers Facebook’s News Feed software. In the tech world, they are now everywhere. The different networks all attempt to model the data they’ve been fed 3 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... by tuning a vast, funky flowchart. After you’ve created a statistical model that describes your real data, you can also roll the dice and generate new, never-before-seen data of the same kind. How this works—like, the math behind it—is very hard to visualize because values inside the model can have hundreds of dimensions and we are humble three-dimensional creatures moving through time. But as the neural-network enthusiast Robin Sloan puts it, “So what? It turns out imaginary spaces are useful even if you can’t, in fact, imagine them.” Out of that ferment, a new kind of art has emerged. Its practitioners use neural networks not to attain practical results, but to see what’s lurking in the these vast, opaque systems. What did the machines learn about the world as they attempted to understand the data they’d been fed? Famously, Google released DeepDream, which produced trippy visualizations that also demonstrated how that type of neural network processed the textures and objects in its source imagery. Google’s David Ha has been working with drawings. Sloan is working with sentences. Allison Parrish makes poetry. Ross Goodwin has tried several writerly forms. But all these experiments are happening inside the symbolic space of the computer. In that world, a letter is just a defined character. It is not dark ink on white paper in a certain shape. A picture is an arrangement of pixels, not oil on canvas. And that’s what makes the knitting project so fascinating. The outputs of the software had to be rendered in yarn. * * * 4 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... Knitting instructions are a bit like code. There are standard maneuvers, repetitive components, and lots of calculation involved. “My husband says knitting is just maths. It’s maths done with string and sticks. You have this many stitches,” said the Ravelry user Woolbeast in the thread about the project. “You do these things in these places that many times, and you have a design, or a shape.” In practice, knitting patterns contain a lot of abbreviations like k and p, for knit and purl (the two standard types of stitches), st for stitches, yo for yarn over, or sl1 for “slip one stitch purl-wise.” The patterns tend to take a form like this: row 1: sl1, k�, k1 (4 sts) o row 2: sl1, k�, k to end of row (5 sts) The neural network knows nothing of how these letters correspond to words like knit or the actual real-world action of knitting. It is just taking the literal text of patterns, and using them as strings of characters in its model of the data. Then, it’s spitting out new strings of characters, which are the patterns people tried to knit. The project began on December 13 of last year, when a Ravelry user, JohannaB, suggested to Shane that her neural net could be taught to write knitting patterns. The community responded enthusiastically, like the user agadbois, who proclaimed, “I will absolutely teach a computer to knit!!! Or at least help one design a scarf (or whatever godforsaken mangled bit of fabric will come out of this).” 5 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... Over the next couple of weeks, they crept toward a data set they could use to build the model. First, they were able to access a fairly standardized set of patterns from Stitch-maps.com, a service run by the knitter J. C. Briar. Then, Shane began to add submissions crowdsourced from Ravelry’s users. The latter data was messy and filled with oddities and even some NSFW knitted objects. When I expressed surprise at the ribaldry evident in the thread (Knitters! Who knew?), one Ravelry user wanted it noted that the particular forum on which the discussion took place (LSG) has a special role on the site. “LSG (lazy, stupid, and godless) is an 18+ group designed to be swearing-friendly,” the user LTHook told me. “The main forums are family-friendly, and the database tags mature patterns so people can tailor their browsing.” Thus, the neural network was being fed all kinds of things from this particular LSG community. “A few notable new additions: Opus the Octopus, Dice Bag of Doom, Doctor Who TARDIS Dishcloth, and something merely called ‘The Impaler,’” Shane wrote on the forum. “The number of patterns with tentacles is now alarmingly high,” she said in another post. When they hit 500 entries, Shane began training the neural network, and slowly feeding some of the new patterns back to the group. The instructions contained some text and some descriptions of rows that looked like actual patterns. For example, here’s the first 4 rows from one set of instructions that the neural net generated and named “fishcock.” 6 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... fishcock row 1 (rs): *k3, k2tog, [yo] twice, ssk, repeat from * to last st, k1. row 2: p1, *p2tog, yo, p2, repeat from * to last st, k1. row 3: *[p1, k1] twice, repeat from * to last st, p1. row 4: *p2, k1, p3, k1, repeat from * to last 2 sts, p2. The network was able to deduce the concept of numbered rows, solely from the texts basically being composed of rows. The system was able to produce patterns that were just on the edge of knittability. But they required substantial “debugging,” as Shane put it. One user, bevbh, described some of the errors as like “code that won’t compile.” For example, bevbh gave this scenario: “If you are knitting along and have 30 stitches in the row and the next row only gives you instructions for 25 stitches, you have to improvise what to do with your remaining five stitches.” But many of the instructions that were generated were flawed in complicated ways. They required the test knitters to apply a lot of human skill and intelligence. For example, here is the user BellaG, narrating her interpretation of the fishcock instructions, which I would say is just on the edge of understandability, if you’re not a knitter: 7 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... “There’s not a number of stitches that will work for all rows, so I started with 15 (the repeat done twice, plus the end stitch). Rows two, four, five, and seven didn’t have enough stitches, so I just worked the pattern until I got to the end stitch and worked that as written,” she posted to the forum. “Double yarn-overs can’t be just knit or just purled on the recovery rows; you have to knit one and purl the other, so I did that when I got to the double yarn-overs on rows two and six. The SkyKnit design “fishcock” as interpreted by the Ravelry user BellaG (Ravelry / BellaG) This kind of “fixing” of the pattern is not unique to the neural-network- generated designs. It is merely an extreme version of a process that knitters have to follow for many kinds of patterns. “My wrestling with the [SkyKnit-generated] ‘tiny baby whale Soto’ pattern was different from other patterns not so much in what needed to be done, as the 8 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... degree to which I needed to interpret and ‘read between the lines’ to fit it together,” the user GloriaHanlon told me. An attempt to knit the pattern “tiny baby whale Soto” by the user GloriaHanlon (Ravelry / gloriahanlon) Historically, knitting patterns have varied in the degree of detail they provided. New patterns are a little more foolproof. Old patterns do not suffer fools. “I agree that an analogy with 19th-century knitting patterns is quite fitting,” the user bevbh said. “Those patterns were often cryptic by our standards. Interpretation was expected.” But a core problem in knitting the neural-network designs is that there was no actual intent behind the instructions. And that intent is a major part of how knitters come to understand a given pattern. “When you start a knitting pattern, you know what it is that you’re trying to make (sock, sweater, blanket square) and the pattern often comes with a picture of the finished object, which allows you to see the details. You go into it knowing what the designer’s intention is,” BellaG 9 of 10 3/10/18, 5:37 AMSkyKnit: How an AI Took Over an Adult Knitting... https://www.theatlantic.com/technology/archive/... explained to me. “With the neural-network patterns, there’s no picture, and it doesn’t know what the finished object is supposed to be, which means you don’t know what you’re going to get until you start knitting it. And that affects how you adjust to the pattern ‘mistakes’: The neural network knows the stitch names, but it doesn’t understand what the stitches do. It doesn’t know that a k2tog is knitting two stitches together (a decrease) and a yo is a yarn-over (a lacy increase), so it doesn’t know to keep the stitch counts consistent, or to deliberately change them to make a particular shape.” Of course, that is what makes neural-network-inspired creativity so beguiling. The computers don’t understand the limitations of our fields, so they often create or ask the impossible. And in so doing, they might just reveal some new way of making or thinking, acting as a bridge into the future of these art forms. “I like to imagine that some of the techniques and stitch patterns used today [were] invented via a similar process of trying to decipher instructions written by knitters long since passed, on the back of an envelope, in faded ink, assuming all sorts of cultural knowledge that may or may not be available,” the user GloriaHanlon concluded. The creations of SkyKnit are fully cyborg artifacts, mixing human whimsy and intelligence with machine processing and ignorance. And the misapprehensions are, to a large extent, the point. ABOUT THE AUTHOR is a staff writer at The Atlantic. He's the author of Powering the Dream: The History and Promise of Green Technology. ALEXIS C. MADRIGAL  Twitter  Email 10 of 10 3/10/18, 5:37 AM

[oathkeeper-of-tarth]

How would you, in simple terms, explain what a neural network is?

Ooooh, cool ask! Hm, okay, trying to keep it simple and in non-technical terms (please let me know if I’ve failed).

You can think of artificial neural networks as a way of making a computer program. They’re basically systems of mathematical equations structured and interconnected in such a way (usually heavily involving layers - the output of one is the input of the one right after it, if that helps you picture it) that they, when working together, mimic the basic behaviour of neurons/axons in human brains. And I mean the very basic, core idea -  that an electrical impulse travels along pathways between a bunch of interconnected neurons and may or may not be propagated further depending on some threshold, and that the more certain pathways are used, the “stronger” they are as connections - i.e. the higher the chance that an impulse will go down that path. In the NN itself you don’t really use operations more advanced than sums, some multiplication, and some thresholding/comparison - the trick is, again, how they work together.

The equations that make up the model of a NN have a bunch of parameters that can be tuned, adjusted bit by bit - factors by which a certain multiplication is done, usually - the weight given to a certain “connection”. So this idea of training a neural net for some problem is basically feeding a lot of data into its input, letting it run this data through all the equations it has in its layers, then comparing whatever response the NN has calculated and spit out to either the original data (if you’re just going for some kind of imitation/replication/fitting) or the result you want to get (i.e. the result you know is the correct “answer” for the inputs you gave).

After each bit of data you’ve run through the net you “score” the result - penalise large differences between the data you want the net to give back and the data it actually gave back, for example - and change up the parameters in the equations that describe your NN, with the ultimate goal of minimising the differences (minimisation/mathematical optimisation procedures are a whole field of their own, so I won’t go into the technicalities of how exactly you know how to tweak each parameter of what could be a huge number of them). Basically, the end effect of this is that connections that give results closer to your goal get gradually strengthened, and those that cause the result to move away from your goal get weakened - and mathematically this would be represented simply by increasing or decreasing the number (or “weight”) by which a certain signal value is multiplied at certain points during the calculations.

You repeat this cycle a ton of times, and your net will keep trying to improve (bless it, it tries so hard), while being adjusted at every step via the parameters you (or, rather - and hopefully - some automated algorithm, because this can take a lot of tries) are tweaking, gradually inching closer (up to a certain point, of course - there are always limitations). When the differences are small enough to satisfy whatever conditions you originally set, the training process is over.

Another thing you should do is make sure you aren’t making a net that’s only good at giving the correct result for this one specific problem/set you’ve trained it on (imagine it as the net, in its eagerness to please, having memorised by rote the solution to a problem, instead of learning how to solve it and problems like it in a more general sense). So what you do is you use only a portion of the data you have available on training your net - the rest you use as validation data, meaning that after the training is done and you’ve gotten results you deem acceptable, you test your net on this remaining, unused and unseen data, and see if it performs well on it even though it hasn’t been trained on it. This helps with avoiding something called overfitting.

So if you’re still with me, it makes sense that NNs would be good at pattern recognition and similar problems. See, for example, DeepFace, Facebook’s face recognition thing - they certainly don’t hurt for data to use for training their nets.

There’s a ton of variations - there are nets with far more advanced structures, with long or short term memory, or nets whose structures dynamically change over time, and a ton of ways to actually go about the training - but what I described above should give you the basic idea. I hope it made sense!

[diet-lemonade]

Allegro.AI nabs $11M for a platform that helps businesses build computer vision-based services

Artificial intelligence and the application of it across nearly every aspect of our lives is shaping up to be one of the major step changes of our modern society. Today, a startup that wants to help other companies capitalise on AI’s advances is announcing funding and emerging from stealth mode.

Allegro.AI, which has built a deep learning platform that companies can use to build and train computer-vision-based technologies — from self-driving car systems through to security, medical and any other services that require a system to read and parse visual data — is today announcing that it has raised $11 million in funding, as it prepares for a full-scale launch of its commercial services later this year after running pilots and working with early users in a closed beta.

The round may not be huge by today’s startup standards, but the presence of strategic investors speaks to the interest that the startup has sparked and the gap in the market for what it is offering. It includes MizMaa Ventures — a Chinese fund that is focused on investing in Israeli startups, along with participation from Robert Bosch Venture Capital GmbH (RBVC), Samsung Catalyst Fund and Israeli fund Dynamic Loop Capital. Other investors (the $11 million actually covers more than one round) are not being disclosed.

Nir Bar-Lev, the CEO and cofounder (Moses Guttmann, another cofounder, is the company’s CTO), started Allegro.AI first as Seematics in 2016 after he left Google, where he had worked in various senior roles for over 10 years. It was partly that experience that led him to the idea that with the rise of AI, there would be an opportunity for companies that could build a platform to help other less AI-savvy companies build AI-based products.

“We’re addressing a gap in the industry,” he said in an interview. Although there are a number of services, for example Rekognition from Amazon’s AWS, which allow a developer to ping a database by way of an API to provide analytics and some identification of a video or image, these are relatively basic and couldn’t be used to build and “teach” full-scale navigation systems, for example.

“An ecosystem doesn’t exist for anything deep-learning based.” Every company that wants to build something would have to invest 80-90 percent of their total R&D resources on infrastructure, before getting to the many other apsects of building a product, he said, which might also include the hardware and applications themselves. “We’re providing this so that the companies don’t need to build it.”

Instead, the research scientists that will buy in the Allegro.AI platform — it’s not intended for non-technical users (not now at least) — can concentrate on overseeing projects and considering strategic applications and other aspects of the projects. He says that currently, its direct target customers are tech companies and others that rely heavily on tech, “but are not the Googles and Amazons of the world.”

Indeed, companies like Google, AWS, Microsoft, Apple and Facebook have all made major inroads into AI, and in one way or another each has a strong interest in enterprise services and may already be hosting a lot of data in their clouds. But Bar-Lev believes that companies ultimately will be wary to work with them on large-scale AI projects:

“A lot of the data that’s already on their cloud is data from before the AI revolution, before companies realized that the asset today is data,” he said. “If it’s there, it’s there and a lot of it is transactional and relational data.

“But what’s not there is all the signal-based data, all of the data coming from computer vision. That is not on these clouds. We haven’t spoken to a single automotive who is sharing that with these cloud providers. They are not even sharing it with their OEMs. I’ve worked at Google, and I know how companies are afraid of them. These companies are terrified of tech companies like Amazon and so on eating them up, so if they can now stop and control their assets they will do that.”

Customers have the option of working with Allegro either as a cloud or on-premise product, or a combination of the two, and this brings up the third reason that Allegro believes it has a strong opportunity. The quantity of data that is collected for image-based neural networks is massive, and in some regards it’s not practical to rely on cloud systems to process that. Allegro’s emphasis is on building computing at the edge to work with the data more efficiently, which is one of the reasons investors were also interested.

“AI and machine learning will transform the way we interact with all the devices in our lives, by enabling them to process what they’re seeing in real time,” said David Goldschmidt, VP and MD at Samsung Catalyst Fund, in a statement. “By advancing deep learning at the edge, Allegro.AI will help companies in a diverse range of fields—from robotics to mobility—develop devices that are more intelligent, robust, and responsive to their environment. We’re particularly excited about this investment because, like Samsung, Allegro.AI is committed not just to developing this foundational technology, but also to building the open, collaborative ecosystem that is necessary to bring it to consumers in a meaningful way.”

Allegro.AI is not the first company with hopes of providing AI and deep learning as a service to the enterprise world: Element.AI out of Canada is another startup that is being built on the premise that most companies know they will need to consider how to use AI in their businesses, but lack the in-house expertise or budget (or both) to do that. Until the wider field matures and AI know-how becomes something anyone can buy off-the-shelf, it’s going to present an interesting opportunity for the likes of Allegro and others to step in.

      Read more: feedproxy.google.com

from Reviews247.net https://ift.tt/2raytPQ

[hub-pub-bub]

The MIT Media Lab is located in a classically modern building. At night, its glossy white walls, floors, and ceilings shine through its glass perimeter to cast a futuristic glow onto the surrounding streets.

But the part of the building that best reflects the Media Lab’s forward-thinking spirit is actually a picture of the past. Dominating the hallway of the Amherst Street entrance is a towering, three-part portrait showing the eccentric Brookline living room of legendary MIT staffer Marvin Minsky.

Marvin Minsky's living room

Many people consider Minsky, who passed away last year, the father of artificial intelligence. His influence spans from his 1951 invention of the first neural net machine at Harvard to the pioneering work currently being done at MIT’s Computer Science and Artificial Intelligence Laboratory (which Minsky co-founded).

For most of Minsky’s career, however, progress in AI was limited to research papers and experimental, one-off prototypes. Today, the hype surrounding AI’s commercial applications has many people insisting it will be the driver of the next big tech wave, creating systems that aid (or replace) workers in every sector of the economy.

“I think machine intelligence represents a once in a lifetime opportunity for entrepreneurs,” said Vivjan Myrto, Co-Founder of AI-focused VC firm Hyperplane Venture Capital. “It also represents a massive opportunity to invest in technologies that are solving really large-scale problems.”

Boston has long been known for its hard tech contributions, but that hasn’t always translated into the creation of market-leading tech giants. Even more discouragingly, in several cases, some of the city’s most promising startups have moved to California as they’ve scaled.

Rob May, Founder of AI-driven virtual assistant company Talla, believes Boston tech companies have had a more conservative approach to the market in the past, a mindset that has sometimes taken its toll. But May thinks AI’s emergence will play out differently than previous tech waves, and he’s not alone.

In fact, people within the Boston tech community are almost uniformly optimistic that Boston is as well-positioned as anywhere to produce the next great companies built around AI solutions.

A History of Intelligence

The colorful portrait of Minsky’s room serves as a vibrant reminder that the city of Boston is the birthplace of much of humanity’s knowledge of AI.

As progress in the field has quickened, many local and state governments have pledged support for AI research and implementation. But in Boston, AI isn’t the latest fad, it’s a long-standing tech frontier we’ve been involved with since its inception.

“We’ve been doing this since way before it was the next big thing in software, since way before it was cool,” said Catherine Havasi, a longtime member of MIT’s Media Lab who founded text analytics spinoff Luminoso in 2010.

Boston’s experience with AI gives local companies several advantages when developing AI solutions that can actually add value in a business environment, which is more difficult than many people think.

Machine learning techniques, for example, use data to train systems to complete a range of tasks including object recognition and detecting credit card fraud. Although these systems don’t need to be painstakingly programmed for each task, they’re often only as effective as the datasets going into them, thus requiring companies to maintain collections of clean, well-labeled data to optimize a system’s performance.

Deep learning, a subset of machine learning that has exploded in popularity over the last five years, typically requires even more expertise to work effectively. Deep learning systems rely on layers of connected processing units to extract insights from data. Getting a successful output from a deep learning system requires users to adjust the weights between units in a technique that’s often more of an art than a science.

That means trying to use deep learning systems in a new space can be next to impossible if you don’t have a mathematical understanding of how they work.

“Whenever there are algorithms or datasets being used, the people who created those things are going to know how to use them the best because they understand things at a deeper level,” Havasi said. “If you run into a problem, you have to go under the hood and make changes, and the people who truly understand these things are always going to have an advantage.”

That advantage is a big reason why Myrto describes Hyperplane as a very “founder-centric” VC firm.

“In AI, you have to be very team-focused and vision-focused,” Myrto says. “The technical teams are absolutely crucial. You need to have a team that understands the technology in a way that others don’t. At the end of the day, that’s what we’re investing in, really outstanding engineers.”

If building successful AI companies requires intellectual capital, then Boston should feel pretty good about its prospects. In their 2015 pitch to bring General Electric to the area, Boston officials described the city as “the world’s most sustainable source of exceptional talent.’’

There are more than 50 colleges and universities in the greater Boston area. According to the state’s 2017 Budget and Policy Center report, Massachusetts has the highest percentage of workers holding bachelor’s degrees of any state in the country.

When it comes to AI, MIT is unquestionably one of the leading research entities in the world, but other schools in the area such as Harvard, Northeastern, Boston University, Tufts and UMass Amherst also have AI-related programs and research labs that have produced a host of intriguing spinoffs.

“Boston really has an incredible heritage of engineers that are focused on these heavy-lifting technologies,” Myrto said. “Boston has always been on the forefront of solving the biggest problems in the world with frontier technologies.”

An Improving Tech Ecosystem

All that brainpower provides only fleeting benefits to Boston, however, if entrepreneurs feel the need to relocate before applying their research and ideas to the private sector. Strong tech ecosystems also require ample support structures for entrepreneurs, and it’s recent improvements in that area in particular that have people bullish on Boston’s future.

Habib Haddad, the president and managing director of MIT’s new investment group the E14 Fund, is one of those people. Haddad said as recently as five or ten years ago, there were several factors that made places like New York and San Francisco more appealing to entrepreneurs than Boston, and they had nothing to do with the demoralizing effects of the snow.

“Great companies like Facebook and Dropbox moved quickly to the other coast because some key elements just weren’t here,” said Haddad. “Now city officials, the startup community, investors, and universities are all saying, ‘We’re not going to miss out on the AI revolution the way we missed out on the consumer revolution.’”

There’s an old-fashioned mindset that academic research should focus on long-term, fundamental breakthroughs at the expense of more commercially applicable advances. In the past, that kind of thinking was certainly more pervasive in Cambridge than in Palo Alto.

Now the proliferation of university-based incubators in Boston is sending a clear message that schools support entrepreneurship. The emergence of university-linked venture capital funds such as UMass Amherst’s Maroon Venture Partners Fund, Boston College’s SSP Venture Partners, and MIT’s E14 Fund further blur the lines between the education and business sectors.

The number of accelerators in the city has also grown over the last ten years, led by groups like Techstars, which has helped local companies raise more than $750 million since it came to Boston, and MassChallenge, which has supported more than 1,200 companies since it launched with money from local officials in 2010.

And many research labs in the area now have corporate partnerships that allow researchers to consider real-world problems instead of the high-level work encouraged by more traditional funding sources like the National Science Foundation. Those partnerships offer a huge advantage in overcoming two of the biggest hurdles of starting an AI company: Determining product-market fit and securing access to large amounts of data.

“With AI, you see people building really cool technologies without identifying a problem, so they’re always trying to find a beachhead,” Myrto said. “But the research labs are interacting way more with investors, and that shift has happened in the last three to five years. They’re interested in building companies from this research. So we’ve seen a shift in mindset, and it’s accelerating big time now.”

Tech Giants Take Notice

One way of looking at this shift is that Boston universities are finally opening their doors to the private sector. The other way to interpret it is that the private sector finally beat their doors down.

GE’s decision to move its world headquarters to the Seaport District is just the latest example of an industrial giant establishing a connection to the city. All around Boston, companies are competing to gain access to the city’s cutting-edge research and talent pool, often elbowing out space for themselves in the process.

This summer Amazon announced a new office along Fort Port Channel, literally next to the space GE has claimed for its flashy new headquarters. Google and IBM have also expanded their local offices in the last three years. Other tech behemoths such as Facebook, Microsoft, and Twitter have made Kendall Square one of the most densely packed tech hubs in the country.

Many of these companies’ local branches are focused on AI. Amazon’s Cambridge office has been deeply involved with the company’s intelligent voice assistant Alexa. IBM has a local lab which seems to focus exclusively on AI, and last month the company announced a 10-year, $240 million investment in the new MIT-IBM Watson AI Lab.

“When you talk about industrial technologies and industrial AI, the race is ours to lose for sure,” Myrto said. “Soon that’ll spread to every industry. GE, Siemens already know this, that’s why they’ve been here. Boston’s background in industrial knowledge is really deep.”

These big companies are also competing for attention, forming partnerships with local tech groups, hosting events and creating their own events in an effort to position themselves as thought leaders. Such events give Boston’s growing tech community a way to keep its small-town feel and provide newcomers with a way to connect with peers over free drinks.

“We’re seeing enormous amounts of growth if you look at the number of events that are happening in Boston, especially around AI,” May said. “The support we’re getting is great.”

But partnerships and free drinks, of course, aren’t all it takes to be successful.

Follow the Money

CB Insights has tracked a rise in Boston VC funding over the last five years (and early 2017 results follow that trend), addressing a weakness that had major implications for area startups in the past. If advances in AI methodologies and computational resources had aligned fifteen years ago, Boston entrepreneurs looking to start companies would’ve had a much more difficult time than today.

Havasi, for instance, said Luminoso had some trouble securing seed funding in Boston in 2010.

“There were certainly funding gaps,” Havasi said. “There wasn’t as much early-stage venture capital or AI venture capital in Boston when we started. That has changed tremendously both across the country and in Boston. Now there are a lot of funds that are very savvy about AI, and that’s fantastic.”

Indeed, when the folks at NextView Ventures sat down to update their excellent Hitchhiker's Guide to Boston Tech last year, they had a lot of additions to make to the investor section. AI companies seeking their first round of funding have been helped by a number of angel groups that have recently institutionalized, including Converge Venture Partners and Half Court Ventures, which May started with Todd Earwood last year.

Myrto said the founders of Hyperplane saw a gap in early-stage AI investing in the area when they launched their VC firm two years ago.

“We saw an opportunity with big data and machine learning in Boston seed investing,” Myrto said. “The thesis of Hyperplane from the very beginning was about machine intelligence and systems intelligence, and we believe Boston is one of the best places in the world to invest in enterprise systems intelligence.”

New firms in the area such as Pillar and Underscore.vc have invested in companies offering AI-driven solutions, with others such as Glasswing Ventures (founded in 2016) and Hyperplane (founded in 2015), focusing almost exclusively on AI.

The growing number of Boston VC firms comes as every firm scrambles to adopt an AI investing strategy. More established VC firms in the area including NextView Ventures, Boston Seed Capital, and Flybridge Capital Partners have also counted AI-driven local companies among their recent investments.

And, perhaps most importantly, we’re seeing investment strategies increasingly veer from the conservative reputation Boston earned in the past. May described west coast investors as more aggressive, helping companies raise large rounds in order to achieve heavy market share, then using their balance sheets as strategic weapons.

“There are pros and cons to that strategy, but it also leads to really big companies at the end of the day,” May said. “I think Boston VCs think less that way typically, and our culture always needs more people thinking big. But you’re seeing some companies raise a lot of money now, and there are some investors that are very west coast-minded.”

Boston’s shortcomings in this area have been talked about a lot. Often they’re referenced as a mistake not to be repeated. Boston investors heading new firms such as Pillar'sJaime Goldstein and Jeff Fagnan from Accomplice (which split from Atlas Venture a few years ago) are among those who have talked about the importance of building large, sustainable companies in the area.

Myrto, who describes Hyperplane as very “west coast-minded,” said he’s seen a change in investor mindsets as well.

“The new generation of venture capital is certainly more inclusive and risk-taking, and those two ingredients are important to having a sustainable ecosystem here,” Myrto said. “Being a little more aggressive in the way we look at technologies and a little more futuristic in the way we see the world is a key ingredient.”

Riding the AI Wave

The race to produce great AI companies has, of course, already begun. Haddad guesses we’re in the “second or third inning” of AI perforating every industry.

Boston has already seen many AI startups gain traction, in some cases helped by recent eye-popping funding rounds. In a three week stretch of March, for instance, DataRobot's predictive analytics platform helped it raise a $54 million Series C and Kensho's financial analysis software earned the company a $50 million Series B.

Big funding rounds are becoming increasingly common in the area. Boston startups are working to overcome some of the largest technical barriers holding AI back, and they’re attracting attention across a wide variety of industries in the process.

Examples of startups working to increase AI’s potential impact include Lightmatter and Forge.ai. Lightmatter is focused on using light, rather than electricity, to improve computational speed and efficiency for AI operations. Forge.ai helps businesses use unstructured data in machine learning systems.

Havasi’s Luminoso, meanwhile, uses AI to analyze customer feedback in 13 languages to give companies insights on product reviews, social media posts and more. May’s Talla integrates AI into office chat programs like Slack and Gchat to help employees automate repetitive processes within a company.

And a number of Boston AI startups are competing with the same tech giants that have recently set up shop in the city. Netra CEO Richard Lee says the company’s visual intelligence software is more accurate than Google in image recognition tasks in head-to-head tests. Newton-based Semantic Machines has raised more than $12 million to make a conversational AI that its website says will “enable people to communicate naturally with computers for the first time.” The Semantic team shares that goal with a number of companies building the next generation of smartphones and smart speakers.

Other Boston AI startups are just plain cool. Cogito uses AI and behavioral science to read people’s emotions in real time. Neurala's product, the Neurala Brain, uses neural network software that’s been designed to closely mimic the functioning of the brain. It was first used to increase the intelligence of NASA’s Mars rover.

Other Boston tech companies such as Localytics, dataxu and HubSpot also now leverage machine learning for core product offerings.

“With AI and machine learning, we’re still exploring where it’s going, but it’ll be everywhere,” Myrto said. “And it’s Boston’s race to lose because you can see all the ingredients are here, from the labs to the talent to even the government being more innovation focused. We’re very young and very hungry to make a big impact on the world. This could be a huge long-term benefit for Boston in general.”

Boston taking a leading role in AI’s implementation could also be good for humanity. Silicon Valley’s “move fast and break things” mindset poses little real danger in consumer tech, but the implications of further AI advancements require a much more thoughtful approach.

“You want to really change the world with AI,” Haddad explained. “Boston has been thinking about AI for a long time. If all you’re doing is optimizing for the short-term opportunity to make money, you’re looking too close to you. And if you’re stepping back and looking at the horizon, that’s also not good because the world needs faster change. Boston has really converged those two views. We’re looking at AI in terms of its impact on society, and those conversations don’t happen as much elsewhere.”

No one can predict exactly how far-reaching the AI wave will be or what companies will come to define it.  All we can do is consider how prepared the city is to support the next generation of entrepreneurs seeking to make their mark on the world.

In that sense, Boston seems to be in good shape to welcome the next Minsky to town.

Zach Winn is a contributor to VentureFizz. Follow Zach on Twitter: @ZachinBoston.

Images courtesy of Henry Han, GE, Gensler, Harvard i-Lab, and CB Insights.

[aiweirdness]

A.I.nktober: A neural net creates drawing prompts

There’s a game called Inktober where people post one drawing for every day in October. To help inspire people, the people behind Inktober post an official list of daily prompts, a word or phrase like Thunder, Fierce, Tired, or Friend. There’s no requirement to use the official lists, though, so people make their own. The other day, blog reader Kail Antonio posed the following question to me:

What would a neural network’s Inktober prompts be like?

Training a neural net on Inktober prompts is tricky, since there’s only been 4 years’ worth of prompts so far. A text-generating neural net’s job is to predict what letter comes next in its training data, and if it can memorize its entire training dataset, that’s technically a perfect solution to the problem. Sure enough, when I trained the neural net GPT-2 345-M on the existing examples, it memorized them in just a few seconds. In fact, it was rather like melting an M&M with a flamethrower.

My strategy for getting around this was to increase the sampling temperature, which means that I forced the neural net to go not with its best prediction (which would just be something plagiarized from the existing list), but something it thought was a bit less likely.

Temperature 1.0

At a temperature setting of 1.0 (already relatively high), the algorithm tends to cycle through the same few copied words from the dataset, or else it fills the screen with dots, or with the repeated words like “dig”. Occasionally it generates what looked like tables of D&D stats, or a political article with lots of extra line breaks. Once it generated a sequence of other prompts, as if it had somehow made the connection to the overall concept of prompts.

The theme is: horror. Please submit a Horror graphic This can either be either a hit or a miss monster. Please spread horror where it counts. Let the horror begin... Please write a well described monster. Please submit a monster with unique or special qualities. Please submit a tall or thin punctuated or soft monster. Please stay the same height or look like a tall or thin Flying monster. Please submit a lynx she runs

This is strange behavior, but training a huge neural net on a tiny dataset does weird stuff to its performance apparently.

Where did these new words come from? GPT-2 is pretrained on a huge amount of text from the internet, so it’s drawing on words and letter combinations that are still somewhere in its neural connections, and which seem to match the Inktober prompts.

In this manner I eventually collected a list of newly-generated prompts, but It took a LONG time to sample these because I kept having to check which were copies and which were the neural net’s additions.

Temperature 1.2

So, I tried an even higher sampling temperature, to try to nudge the neural net farther away from copying its training data. One unintended effect of this was that the phrases it generated started becoming longer, as the high temperature setting made it veer away from the frequent line breaks it had seen in the training data.

Temperature 1.4

At an even higher sampling temperature the neural net would tend to skip the line breaks altogether, churning out run-on chains of words rather than a list of names:

easily lowered very faint smeared pots anatomically modern proposed braided robe dust fleeting caveless few flee furious blasts competing angrily throws unauthorized age forming Light dwelling adventurous stubborn monster

It helped when I prompted it with the beginning of a list:

Computer Weirdness Thing

but still, I had to search through long stretches of AI garble for lines that weren’t ridiculously long.

So, now I know what you get when you give a ridiculously powerful neural net a ridiculously small training dataset. This is why I often rely on prompting a general purpose neural net rather than attempting to retrain one when I’ve got a dataset size of less than a few thousand items - it’s tough to thread that line between memorization and glitchy irrelevance.

One of these days I’m hoping for a neural net that can participate in Inktober itself. AttnGAN doesn’t quiiite have the vocabulary range.

Subscribers get bonus content: An extra list of 31 prompts sampled at temperature 1.4. I’m also including the full lists of prompts in list/text-only format so you can copy/print them more easily (and for those using screen readers).

And if you end up using these prompts for Inktober, please please let me know! I hereby give you permission to mix and match from the lists.

Update: My US and UK publishers are letting me give away some copies of my book to people who draw the AInktober prompts - tag your drawings with AInktober and every week I'll choose a few people based on *handwaves* criteria to get an advance copy of my book. (US, UK, and Canada only, sorry)

In the meantime, you can order my book You Look Like a Thing and I Love You! It’s out November 5 2019.

Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s