Top Podcasts
Fetch.ai With Emma Smith
NFT revolutionized how auction is being done nowadays, which brings the action to the internet realm. But Fetch.ai takes it up a notch by putting up artwork created with the help of machine learning. Joining Jeff Kelley, Eathan Janney, and Josh Kriger is the company’s Senior Engineer, Emma Smith. She talks about (and presents a short demo) using artificial intelligence to develop the most amazing digital art. Emma explains how this innovative technology works through random sources, different artist patterns, and the user’s sheer creative skills. She also dives into the potential use of machine learning for other purposes, from healthcare, transportation, to smart contracts.
—
Listen to the podcast here
Fetch.ai With Emma Smith
This episode features Emma Smith who leads the Collective Learning team at Fetch.ai, which is building an open-access, tokenized and decentralized machine learning network to enable smart infrastructure built around a decentralized digital economy. Collective learning is the framework upon which the NFT platform by Fetch.ai was built. Emma has an MSc in Physics from the University of Cambridge and is an experienced software engineer with a demonstrated history of working in the research and crypto industry. She is adept at coding using Python and C++, and is skilled in data science and machine learning. Emma, it’s great to have you here. That’s an impressive background. It sounds like it’s going to be a riveting conversation.
Thanks, Eathan. I’m excited to be here as well. We’ve had a great engagement with our CoLearn pAInt platform so far. I’m excited to be able to share it with you all.
Machine Learning: Collective learning came in using tools from the decentralized economy. This enables people to benefit from the data to collaborate, train these algorithms, and get the benefits at the end.
We’re excited, Emma. Our show is all about the convergence of technology and culture. What you’re doing at Fetch with this project that you’re leading sounds amazing. It says a lot about you. They gave you this massive responsibility to break new ground.
I hope I explained it well. It’s exciting. That interface between people and technology is at the core of what’s the CoLearn pAInt is about. The central part of it is that we’ve got a machine learning model that produces artwork. We’ve also got input from people who shaped the artwork in the way they want to go. The end result is these artworks that were meant to use NFT are both AI and human beings.
To take a step back, Fetch is a leader in the industry. Maybe not everyone knows the origin story. How did this idea come together for Fetch? How did you get involved?
Fetch does a lot of things around the decentralized economy. One of them is collective learning. There’s all to this machine learning stuff and sometimes it gets called AI. It’s doing incredibly cool things. You can get models that detect cancer from slides, drive cars, or detect fraud. We used to think that all of these things were something only a human could do like detect faces in a picture. Now, we’ve got machine learning models that can do all of these. That’s exciting.
There are a lot of drawbacks still, things that aren’t great. These models tend to be proprietary. They’re owned by one big company. The data that you need to train them, either you need to be like Facebook or Google, those people with a huge lake of data to train this stuff on. There’s the data that you need but it can’t be shared. It’s got to be kept private. Ordinary people can’t benefit from these models. They’re often trained on their data. That’s where the idea of collective learning came in.
Using tools from the decentralized economy like smart contracts, we can enable people to benefit from their data, collaborate, train these algorithms, and then get the benefits from them at the end. That’s a little bit abstract. The idea between CoLearn pAInt is, let’s do this in a way that everybody can see and understand. People can see artworks. They understand what they like and what they don’t like. On the platform, there’s a model at the center that’s producing these artworks, and then there are people controlling it, shaping it, and getting a benefit from the end because they get the proceeds from the NFTs.
How did you get involved? Were you drafted from the outside or were you already working at Fetch? Did you suggest the project or did you join the team?
My background is in machine learning. I worked in a research group for a while. I’m always interested in what’s the next cool thing you can do with the machine learning model. I joined Fetch because I thought that the combination of machine learning and a decentralized economy had a huge number of things you could do with it.
This project was inspired by lots of these algorithmic art ones. Something that’s cool in NFTs is an art that’s made by an algorithm. It hasn’t been seen before like Art Blocks. Somebody must have seen CryptoPunks and had gone, “Can we build one of those?” There’s a huge amount of creativity in this whole space. We thought, “Let’s combine that with the collective learning that we’re already building.”
**It makes sense. Everything is building on top of everything, and getting mashed up in interesting ways. What Art Blocks is doing is cool. What you guys are doing establishes your own point of view on what the future can bring in terms of converging technology. **
I hope so.
**It’s fun to hear about these CoLearn pAInt projects. I’ve got a creative background. I play the piano, I like to draw and things like that. Our show art is something that I created. I’m curious how this came together. Was it just one day this idea fell in your lap to do a generative art type project or a collaborative art type project? How did this originate? How did it develop? **
There have been some interesting papers in the field about art generation. There was a big breakthrough in architecture that someone designed for a certain model. This produces art that is way cooler looking than all the previous art. I’m inspired by that. That was a key thing. It’s interesting that you mentioned music because I still haven’t seen the same thing for music. It’s very hard to get a machine to write music. You listen to it and you think, “This a little bit makes sense.” You then listen to the thing as a whole and it has no structure to it. It never comes back to the original chords. It wanders off in some weird direction.
It’s one of those things that Elon Musk mentions a lot when he’s talking about Neuralink in his approach to handling AI. It’s the creativity part of it and trying to reflect what humans do from a creative perspective. It’s been difficult. It seems like on the roadmap for AI, that’s a pretty big challenge versus all the left brain-type stuff. Do you agree on that point? It sounds like it is a big challenge as you mentioned already. Do you think that’s something that can be overcome with AI machine learning?
The field keeps coming on leaps and bounds. Each time I see a new model, I’m like, “I can make better art,” and that type of thing. It doesn’t fail in the same way a human would do, which often makes it a bit difficult to understand. If you told a human to draw, maybe they would have faces that looked a bit weird and lumpy, whilst your machine learning model most of the time makes faces. Occasionally, it makes some completely random mess and you’ve got no idea why. That’s a bit of a roadblock. In something like generating art. It’s fine because you can take the cool-looking ones. In something like self-driving cars, I’m sure 2021 was meant to have self-driving cars in it. Getting the model so that they don’t fail in weird ways. How we can understand why they’re failing is still a big problem in the field.
It’s a great concept, the collaboration with human intelligence and artificial intelligence to create art and music. I’m pretty impressed with my simple GarageBand software on my Macintosh. It’s got these “drummers.” You can give them different tempos to play. You can decide which drums they’re going to use in a specific segment, style or whatever. You can speed up or slow down the tempo, change the mood, and change the intensity. It does a pretty good job. It’s your own private robot drummer. There’s a lot of space here for the stuff that you’re working on in music and art. It’s like, “Let’s hold hands with robots and make art.”
It’s a cool idea. I’d love to see machine learning-generated music as an NFT. It would be unique.
With CoLearn pAInt, we’d love to learn more for our readers about how it works. What does the user experience look like at the intersection of the AI and machine learning aspects of the platform?
The way the platform works is everybody who’s taking part sees this generator with some uniquely generated randomness. These generators need a source of randomness to turn the art. If you imagine, it’s just an algorithm. If you didn’t have a random source, it would produce the same thing each time. There needs to be some random input for it, which people generate by drawing a little picture. The art doesn’t come out looking at the pictures. It’s used to see this generator. That’s the first step.
The machine learning model tries its best to make cool-looking artworks. We’ve trained the model on a data set that we’ve collected to open license images from the internet. What people see is they see what the model thinks is art. That’s a stage in its training. Sometimes it’s quite good at picking out patterns and things. It often gets eyes. In those eyes, an eye is a nice repeating pattern. It’ll stick eyes randomly on things. It has a bit of a sense of interesting colors and patterns and things like that. That’s the first stage of the voting.
Everybody votes. Everybody who’s staked their tokens to take part in the competition gets to vote for the winners. We take those winners through to round two. They get fed into the training of this algorithm, so we trade a bit more. We say that these are the cool ones and they produce more like this. You can see as a user from stage 1 to stage 2. It’s using some of the patterns or colors that were selected from stage one and trying to use them more because it’s learned that that’s what’s good and what people like to see. There’s another around and then the winning images from the third round, we mint them as NFTs.
I love that rhythm of the process and finding ways to collaborate between an AI and it came in feedback. We don’t see that frequently. Maybe it happens a lot behind the scenes. In our circle, we don’t hear about that much or see that in action. It sounds like it’s going to elevate awareness of this concept. A lot of people might take that and be inspired by it to develop other projects or other concepts around similar ideas. It’s cool.
**It reminds me of improv, something that the three of us have done. You’re in this troupe and sometimes there’s a call out from the audience of something random that maybe has nothing to do with anything that you had in mind but you roll with it. You get feedback based on what the audience likes and what they think is funny, and you go in that direction. As the troupe does more and more projects together, they get more comfortable with each other. Anything that they get thrown at them, they can make it awesome. **
You’ve never had anything that stumped you?
Totally.
That’s why improv is entertaining. Part of the entertainment of improv is the awkwardness that you get to watch performers having to figure out what to do. The more difficult, the better.
Let’s take a step back here and talk about the bigger picture of collective learning and its range of used cases. Where can this all go? What are some of the potential areas where this could be applied? How do we optimize this type of thing? What’s on your mind and keeping you up at night in terms of what’s next?
It’s useful in a huge number of areas wherever machine models can be used. One of the key ones we have looked at is healthcare. In healthcare, there are a lot of health data that have to be kept private. Nobody wants to share their private health data. You don’t want to upload that to some central server. There’s still a huge use of that data that could be made to diagnose things from scans, classify slides, and those things. This is something we’ve been focusing on. There are a bunch of techniques that you can use. Everybody should take some model train and bid on their own data and then pass it on. That way, you don’t have to share the private training data. You just get the benefit of this model in the center. That’s a used case that we’re excited about.
We’ve been in talks with some transport ones. Making cars now like supercomputers on wheels. There’s an awful lot of data there, but it’s the privacy concerns. Also, people say, “Why would I bother sharing my data? Why do I bother training this machine learning model on my car where it’s consuming my power?” The other thing is by using smart contracts, you can incentivize people to share their data to do these tasks. You can say, “Your update to this model is very useful. This model predicts when electric cars need to go to charging points. In reward for that, you can get some cryptocurrency.” Because it’s all a lot more seamless, we can incentivize things that otherwise wouldn’t have been economical to do.
It reminds me of my former life in the consulting world. I worked on a predictive analytics project trying to look at trends around homelessness for veterans. There were 60 different systems around the country and different data collection teams. It sounds like something like this could have helped them predict patterns around preventing homelessness among veterans.
That sounds like an important use case. Certainly, our machine learning models can often be complicated. It’s not as simple as this means this relationship. It’s a huge number of different factors. Machine learning models can find those patterns that a human wouldn’t be able to find. With collective learning, you don’t need to have one person that has more data. People with whole data can come in together and train the model together.
From the optimization side, is it repetition and practice or is there more to it?
In terms of training the model, do you mean how it gets better at its task?
Yes. Is it a matter of practice makes perfect?
Essentially, they are learning from their mistakes. The errors that they make get sent backward through the network. If you’ve got this network, the inputs come in at one end and they get transformed through layers of this network until you get outputs that classify. It’s like a scan, having this disease or not having this disease. When it makes a mistake, you can propagate the error backward through this network and work out which of the weights were wrong, and which of them ought to be changed. You do loads of steps like that. Gradually, your training progresses and the model gets better at capturing those patterns.
Machine Learning 101.
It’s a tricky topic to explain.
It sounds a little bit like The Lean Startup methodology.
Make something terrible and then iterate through the mistakes.
This is fascinating stuff. We’re excited about this unboxing coming up. I got one more question for you before that. We know you’re super sharp and you’re keeping track of the coolest stuff in the space with tech and art. What NFT projects and platforms, either around now or maybe that you foresee, stand out as potential game-changers here? We’ll call it long-term. I know 5 or 10 years is hard to project. What do you think?
It’s tricky, especially when the space is moving quickly. Lots of the generative art stuff looks beautiful. The algorithms you use to make it are interesting. Those Fidenzas from Art Blocks look nice. The pattern that they use to make them is amazing. There are beautiful things that can come out. Some of the ones that have already been around for a couple of years will still be around.
I don’t think CryptoPunks is going to go to zero anytime soon. A lot of them are fun where your NFT gives you access to something. I spoke to Josh about ZED RUN Horse Racing. I had to look at that after our conversation and I was like, “This is fun.” You buy your horse. You get to train your horse. We’ve seen a huge amount of personal engagement with people and things like Bored Ape Yacht Club where it’s not just a collectible thing, it also gives you access to something. It’s something you get to interact with.
**Speaking of beautiful, there’s now the Mutant Apes that have come out. They are disfigured in a beautiful way. For those folks that don’t know, they also dropped M1, M2 and M3 mutant viruses. You can turn your Bored Ape into a mutant. The M1, the rarest ones, are selling for $1 million. Eight lucky people got airdropped $1 million. The M2s are going for $75,000. The M3s are going for $25,000. Either way, it’s not a bad airdrop, to say the least. I apologize if any Ape holders out there read this and I’ve transposed the information. **
To your point, generative art is amazing and there’s an unlimited possibility. There are the dances now. It’s going to keep going and going. It’s about community and having fun. There’s so much about NFTs. The real game-changing stuff is functional and practical and something that maybe all of us won’t realize or even NFTs in the future. The community and the fun part of it are driving so much of it forward. It’s great to be part of that. What we’re excited about is to get a look at a demo of your project, show some of our readers what it’s all about, and learn a little bit more.
I’m looking forward to showing you all. I’m going to show you all the demo. I’ve got it running locally on my computer. We are running our live platform. We’ve had a great amount of engagement with it. People have staked 1.6 million FET tokens. That’s about $800,000.
Machine Learning: Machine learning requires some sort of randomness for it to produce art. Otherwise, it would just produce the same output over and over again.
**That’s an impressive base of supporters already. **
There is a bit of an application process to be part of the team.
There’s this idea of having a machine learning model and it’s something valuable and that you get to control it. To take part in this, you can stake some of your FET tokens. That’s the token for Fetch.ai, which is the company that Collective Learning is part of. You stake these tokens to take part. We organize that through a Dutch auction. It’s a Dutch auction organized through a smart contract.
People can be certain that they’ll get their FET back because they can verify the contract themselves. It’s a Dutch auction so it’s backward. You start with the high price and you gradually move it lower. People are going like, “Will I pay that much for a slot? Maybe I’ll wait until it gets lower. Maybe it will sell out.” I’ve made my bid with some FETs. I got some predictive slots. This is the first stage. This is how we select people. There’s a fixed number of slots available. In our event, we’ve got 200. That determines the proceeds people get from the final sales. At the end of this process, we’ll take our top three best-looking artworks and mint them as NFTs. We auction them on OpenSea. All those people who staked their FETs get their FET tokens back. They also get their cut of the proceeds from this. It’s getting people to use this idea that you take part in a machine learning model, you invest into it, and then you get rewards out from that process.
I wonder how many participants are going to be excited that they’re going to buy the art.
That would be cool. Also, when you interact with something, you then start to feel a sense of ownership of it, which is good for this thing. They’ll think, “I made that. I want to own it.” I’m going to switch around to the build phase. This is the first one where people are submitting their entropy to a seed generator. It needs a source of randomness. We’re generating this from all the users by getting them to draw a picture. They draw a picture and we use this as the entropy that’s used to seed the generator. It’s a fun way of coming up with some unique random input. It’s been amazing what creativity people had, the things you can draw in a little box. There’s been a lot of rockets, moons and attempts at drawing dogs and things.
Here’s the rocket and there’s the moon.
**It’s almost like a bear. **
It’s very animaloid but it’s also abstract.
Bright colors, shapes, and things.
I could imagine that’s an eye.
When in doubt, add eyes.
**Is there a marine element to it almost? **
I’m excited. This can turn me into an artist.
These are random seeds for our brand machines.
**KC has some of our colors in it. Keep going. **
It’s like a ‘90s style.
Eathan does love the color pink though.
It does, doesn’t it?
Yeah.
**I pick that one. That’s mine. **
**We’re going to get that. **
**I’m in. I’m going to go with my gut. **
**We might gift that to the Frogman guys. **
**I got gruesome but in a very cute way. **
Yes.
Jeff is up.
We’ve got stars.
Flash master Jeff.
**I see crows in there. It’s cool to me. **
**We got to pick one now for the readers. **
I’ll pick on for the readers. Creepy Eyes?
**He’s a sweet boy. **
I see rainbows.
**That’s called Bloody, by the way. **
**Steadfast is also pretty dope. **
I like Fast.
**Fast is cool too. Edge of NFT and Fast. **
It’s like the beak of a bird.
I began thinking of Edge of NFT colors.
**Let’s look at colorful. **
It’s tasty.
I like the colors. It matches our colors.
It’s about a boy and his dog.
Do you want Tasty or Bloody, Jeff?
**That was not easy. **
**Thank you. That was a fun process. **
Sure. Let’s go for it.
Let’s go for it.
**Your background is looking very feng shui. **
You just can’t see the floor.
I’ll quite possibly crack open a beer.