The Joys and the Struggles of AR Lens Production
Well hello everyone!
And let’s talk about the joys and the struggles of the Snapchat lens production. So the idea arose to record our small internal interview. Participants are interviewer Andrey Sazonov, founder of the Lensman studio; Andrey Kolesnikov, co-founder of the studio and lead project manager and our lead developer Evgeniy Palamarchuk. So, openly, honestly and with humor about this whole ordeal.
And let’s talk about the joys and the struggles of the Snapchat lens production. So the idea arose to record our small internal interview. Participants are interviewer Andrey Sazonov, founder of the Lensman studio; Andrey Kolesnikov, co-founder of the studio and lead project manager and our lead developer Evgeniy Palamarchuk. So, openly, honestly and with humor about this whole ordeal.
oct 31 · 5 min read
Andrey Sazonov
Founder
Hello guys! Let’s start with a general question. How do you see the prospects for social AR in general? From your perspective, where is this whole Snapchat lens/mask industry heading?
Evgeniy Palamarchuk
Lead Developer
Three years ago I didn’t know what AR lenses were and where it would lead us (laughs). The first lens was very simple, but nevertheless it took quite a lot of struggle to make! As somebody who has come from Unity development, I had to delve into the specifics of lens making and yes, it was a normal switching process, but with challenges. Now we are dealing with a wide variety of lens projects: games, product presentations, that raise brand awareness, etc.
That is, the directions are very different. Snapchat and their Lens Studio don't stay in one place, that's great. They develop, add new features that we are already implementing into new lenses.
And, looking back, I can say that we are no longer sitting on technologies we used 2−3 years ago. Lens Studio updates on average once a month. So we have to learn Lens Studio innovations and apply them. Plus, the creative agencies we work with also actively follow the updates and ask us to use them to the maximum in new projects. For example, ground segmentation (a feature in Lens Studio) demanded from us both effort and time for its implementation. Despite the fact that the development process took place not in calm measured conditions of R’n’D, but "in the field" - we immediately implemented everything into a commercial project.
Andrey Sazonov
Founder
Is there such a thing that only one technical feature is actively promoted and used, for example, only high-quality face / head tracking, and everything else does not develop, or exists in very raw and buggy condition?
Evgeniy Palamarchuk
Lead Developer
No, no. Snapchat developers actively diversify Lens Studio. Full body tracking and new shaders, as an example. I would compare the vector of development of Lens studio with the way development takes place in Unity: everything is actively evolving in different directions. And let’s not forget the big plus of Lens Studio — cross platform, there is no need to separately develop lenses for iOS and Android.
But! But in fact, there is one big sore — a bunch of unfixed old problems remain! And yes, it’s great that everything is growing and developing, but the unresolved problems of the software have been hanging for years now. That’s why I compare it with Unity, where there are such trends of developments and the accumulation of old unfixed bugs.
Andrey Sazonov
Founder
Okay, and how do you feel about this? Well, should we take it for granted, live and create with a bunch of old bugs in the software? Or should we endlessly bother the support team, accumulate total dissatisfaction in the community and demand to fix something old and broken?
Evgeniy Palamarchuk
Lead Developer
Hmm… How should we take it? I would say that depends on your status. If you are just an ordinary creator of lenses, of which there are many, the chance of being heard is very small. Or if you are the official partner of Snapchat, your bug reports will be heard much earlier.
Andrey Kolesnikov
Project Manager
Yes, yes. I want to add that the voice of one creator will not affect anything. But if hundreds of community members point to some kind of problem, then yes, the Snapchat guys will pay attention and fix it.
Andrey Sazonov
Founder
Okay. Let’s dream up. Let’s imagine that you have in your hands the steering wheel of the further development of Snapchat lenses. Where would you go next for development? Would you treat old bugs, or maybe you would go deeper into the development of new missing features? Or maybe would you cut off some features and focus on the necessary, throwing away the unnecessary?
Evgeniy Palamarchuk
Lead Developer
Wow! This is a big question! Well, let me see. Machine learning receives rapid development and plays a huge role in the recognition of facial expressions, objects, poses.
But you need to know Python to use Machine learning. In addition, you need to feed the neural network a bunch of images to train it. So, the simplification of this process in the Lens Studio would be cool.
Another area for improvement: reducing CPU usage. Now the image processing algorithms in Snapchat often lead to overheating of devices, which, of course, affects performance — the lens starts to drop frames. I would like to see optimization of how the image is processed inside Snapchat, which ultimately will lead to less load and overheating of the processor, and the smartphone battery will not drain so quickly when the lens is played.
Andrey Sazonov
Founder
Oh, here’s another point with the lens size limitation. That it should be no more than 4 MB. True, lately this threshold has been increased to 8 MB. Where did this very limitation in lens size come from? Is this due to the will of Snapchat to ensure their lenses will work without slowdowns on low-end smartphones?
Evgeniy Palamarchuk
Lead Developer
Yes it is. After all, if you do not limit the appetites of lens creators by limits on the size of the lens, then you can inflate the lens to an unbearable size. And not even a single top device will be able to play it. And besides, the disk space on the Snapchat servers does not have an infinite capacity.
Of course, the more a megabyte is given for the lens, the more detailed textures and models with a large polygon count we can use in the lens. Which means both the realism and visual quality increase, but we also need to follow the realities of smartphone performance — so that everything we have done can run on a very wide variety of devices and this very 8MB limit almost guarantees it.
Of course, the more a megabyte is given for the lens, the more detailed textures and models with a large polygon count we can use in the lens. Which means both the realism and visual quality increase, but we also need to follow the realities of smartphone performance — so that everything we have done can run on a very wide variety of devices and this very 8MB limit almost guarantees it.
Andrey Kolesnikov
Project Manager
Yes. We can also see that more optimized lightweight lenses appear in the first places in the search. Thus, Snapchat encourages making it lightweight and pleasant for the eyes.
Andrey Sazonov
Founder
Snapchat team is great, they have a cool marketplace, and now they are creating a digital playground. What’s going on with gamification right now? How many customers are ready to incorporate game elements into their lenses?
Andrey Kolesnikov
Project Manager
The main thing to understand here is that for customers, lenses are primarily advertising, and if the introduction of game elements increases audience engagement, then it comes in handy.
A regular advertising lens will not immediately bring an increase in sales. It only contributes to brand awareness. If you want more: more sales, more customer involvement, then the introduction of game elements in the lenses just helps.
That is, you first warm up the audience with an entertaining advertising lens, and then you work to create gamified content that is directly related to a specific product or service. For example, collecting physical tokens or cards with AR elements.
Andrey Sazonov
Founder
Is it possible to say that the gaming potential is underused?
Andrey Kolesnikov
Project Manager
Yes, you could say that. But Snapchat is working in this direction. For example, there are already turn-based co-op games that are being developed more and more.
There are many genres that can potentially be realized. You can implement multiplayer or quest passing which the player receives, for example, a discount on goods.
Andrey Sazonov
Founder
By the way, gamified lenses are very suitable for sports products, since the elements of the game are inherent in sports activities.
Andrey Kolesnikov
Project Manager
Right! I would like to add that elements of gamification for retailers immediately generate a large growth of customers. Because such projects are very popular with children. Children adore augmented reality and lenses: they play these games with pleasure, collect characters from cards that they received in the store in exchange for points, and ask their parents to collect more points. Thus, the number of sales and the average receipt in supermarkets increases.
Andrey Sazonov
Founder
And it is a great way to learn through the game. We all were kids and are good at learning something new through play.
Evgeniy Palamarchuk
Lead Developer
Yes, that’s what my first programming lessons as a child looked like! I was intrigued by how you can automate the drawing of five squares, and when I programmed this very drawing, I felt like a real creator!
Andrey Sazonov
Founder
Well then, gentlemen, let’s stop there. Thank you very much for your insights and vision for future prospects! We wish us new creative breakthroughs and growth! See you in lens-space ;)
|
By clicking “Send” you agree to our Terms of Use, Privacy and Cookies Policy.
More articles:
© 2021 LENSMAN.
ALL RIGHTS RESERVED.
ALL RIGHTS RESERVED.
GET IN TOUCH