#applied hardware

[thursdayisbetterthanfriday]

Applied AI - Integrating AI With a Roomba

AKA. What have I been doing for the past month and a half

Everyone loves Roombas. Cats. People. Cat-people. There have been a number of Roomba hacks posted online over the years, but an often overlooked point is how very easy it is to use Roombas for cheap applied robotics projects.

Continuing on from a project done for academic purposes, today's showcase is a work in progress for a real-world application of Speech-to-text, actionable, transformer based AI models. MARVINA (Multimodal Artificial Robotics Verification Intelligence Network Application) is being applied, in this case, to this Roomba, modified with a Raspberry Pi 3B, a 1080p camera, and a combined mic and speaker system.

The hardware specifics have been a fun challenge over the past couple of months, especially relating to the construction of the 3D mounts for the camera and audio input/output system.

Roomba models are particularly well suited to tinkering - the serial connector allows the interface of external hardware - with iRobot (the provider company) having a full manual for commands that can be sent to the Roomba itself. It can even play entire songs! (Highly recommend)

Scope:

Current:

The aim of this project is to, initially, replicate the verbal command system which powers the current virtual environment based system.

This has been achieved with the custom MARVINA AI system, which is interfaced with both the Pocket Sphinx Speech-To-Text (SpeechRecognition · PyPI) and Piper-TTS Text-To-Speech (GitHub - rhasspy/piper: A fast, local neural text to speech system) AI systems. This gives the AI the ability to do one of 8 commands, give verbal output, and use a limited-training version of the emotional-empathy system.

This has mostly been achieved. Now that I know it's functional I can now justify spending money on a better microphone/speaker system so I don't have to shout at the poor thing!

The latency time for the Raspberry PI 3B for each output is a very spritely 75ms! This allows for plenty of time between the current AI input "framerate" of 500ms.

Future - Software:

Subsequent testing will imbue the Roomba with a greater sense of abstracted "emotion" - the AI having a ground set of emotional state variables which decide how it, and the interacting person, are "feeling" at any given point in time.

This, ideally, is to give the AI system a sense of motivation. The AI is essentially being given separate drives for social connection, curiosity and other emotional states. The programming will be designed to optimise for those, while the emotional model will regulate this on a seperate, biologically based, system of under and over stimulation.

In other words, a motivational system that incentivises only up to a point.

The current system does have a system implemented, but this only has very limited testing data. One of the key parts of this project's success will be to generatively create a training data set which will allow for high-quality interactions.

The future of MARVINA-R will be relating to expanding the abstracted equivalent of "Theory-of-Mind". - In other words, having MARVINA-R "imagine" a future which could exist in order to consider it's choices, and what actions it wishes to take.

This system is based, in part, upon the Dyna-lang model created by Lin et al. 2023 at UC Berkley ([2308.01399] Learning to Model the World with Language (arxiv.org)) but with a key difference - MARVINA-R will be running with two neural networks - one based on short-term memory and the second based on long-term memory. Decisions will be made based on which is most appropriate, and on how similar the current input data is to the generated world-model of each model.

Once at rest, MARVINA-R will effectively "sleep", essentially keeping the most important memories, and consolidating them into the long-term network if they lead to better outcomes.

This will allow the system to be tailored beyond its current limitations - where it can be designed to be motivated by multiple emotional "pulls" for its attention.

This does, however, also increase the number of AI outputs required per action (by a magnitude of about 10 to 100) so this will need to be carefully considered in terms of the software and hardware requirements.

Results So Far:

Here is the current prototyping setup for MARVINA-R. As of a couple of weeks ago, I was able to run the entire RaspberryPi and applied hardware setup and successfully interface with the robot with the components disconnected.

I'll upload a video of the final stage of initial testing in the near future - it's great fun!

The main issues really do come down to hardware limitations. The microphone is a cheap ~$6 thing from Amazon and requires you to shout at the poor robot to get it to do anything! The second limitation currently comes from outputting the text-to-speech, which does have a time lag from speaking to output of around 4 seconds. Not terrible, but also can be improved.

To my mind, the proof of concept has been created - this is possible. Now I can justify further time, and investment, for better parts and for more software engineering!

[tricitymonsters]

Does Kazu hand draw plans or does he use a computer program to draw them (It would be funny if he was super bad with tech)

He's actually not bad with tech but it depends on some factors. He typically hand draws plans, though he's used CAD for really complicated shit and can 3D model stuff for printing but he's kind of an idiot savant. He picked up most of his engineering skills (including maths and physics) from first hand projects and working alongside more experienced builders/fabricators. So he's not totally isolated from tech but like where he knows how to use Blender or something, he gets mad at setting up a local network/when his internet runs slower than it should and the ISP is like lolidk. He has a pile of destroyed printers that he leaves out where it serves as a warning to the next New Printer.

[the-mewrderus-duck]

im hawkining commissions to help pay for my car

https://ko-fi.com/itsdeadjim

I'm in the middle of a job hunt that's taking longer than I thought it would. So if you have a birthday coming up or want to see one of your little guys doing something consider getting some art from me. Many tiers available. If you wouldn't mind sharing this around that would be great. 💕💕

It takes me about two hours to finish most digital art pieces.

[magebird]

Sometimes I feel like the discourse about AI art misses the actual point of why it’s not a good tool to use.

“AI art isn’t ‘real’ art.” —> opinion-based, echoes the same false commentary about digital art in general, just ends up in a ‘if you can’t make your own store-bought is fine’ conversation, implies that if art isn’t done a certain way it lacks some moral/ethical value, relies on the emotional component of what art is considered “real” or not which is wildly subjective

“AI art steals from existing artists without credit.” —> fact-based, highlights the actual damage of the tool, isn’t relying on an emotional plea, can actually lead to legally stopping overuse of AI tools and/or the development of AI tools that don’t have this problem, doesn’t get bogged down in the ‘but what if they caaaaan’t make art some other way’ argument

Like I get that people who don’t give a shit about plagiarism aren’t going to be swayed, but they weren’t going to be swayed by the first argument either. And the argument of “oh well AI art can’t do hands/isn’t as good/can’t do this thing I have decided indicates True Human Creativity” will eventually erode since… the AI tools are getting better and will be able to emulate that in time. It just gets me annoyed when the argument is trying to base itself on “oh this isn’t GOOD art” when AI does produce interesting and appealing images and the argument worth having is much more about the intrinsic value of artists than the perceived value of the works that are produced.

[titancores]

Me, writing fiction about techfolk: using random-ass tech terminology from a mix of real-life machines because hey, it's all pretend and I'm having fun.

A robot writing fiction about humans: "The human clicked on her tapetum lucidum to switch to night vision, then everted her lungs to change the gill filters. She climbed the wall, constantly rotating her radius and ulna bones around each other. Her hyoid bone rattled a warning to other humans in the area, and her ceacum growled in sympathy. Her loops of henle would need draining soon."

[sisterrmorphine]

I think I'm gonna stick it out two weeks and if I hate it every day I'm gonna put my two weeks in and try to get a new job for the remaining five -_- idt anyone will hire me for five damn weeks so my plan will be to quit again after those. And then move. Maybe if I can start looking for a new job next week I can just work this one for the next three <- coping cuz. I won't quit and get a new job

[juh-wasnt-deactivated19691231]

[waugh-bao]

*

[darkandlightsides]

Happy about having a deskjob and realizing I'll never have to work retail again and thinking god might be real

[attila-werther]

photos taken five minutes before disaster

[foxpunk]

"no brand is actually good--" true. "--or better." now hold on a minute.

[mothbaaalls]

so funnily enough in my town there are two music stores basically right next to each other and as someone who's never owned an instrument before i've never had cause to go into one (always been a little afraid to go in because I Didn't Belong) but i've always been BURNINGLY CURIOUS. do they have beef?? the owners must know each other right??? I WANNA KNOW SO BADDD

and now that i have a guitar i actually have a reason to go into them because i desperately need new strings (mine were already really rusted because the guitar was secondhand from somebody who played it maybe a handful of times and then left it in its gig bag for years) AND I WANNA KNOW SO BADDDDDD

[tofupixel]

⭐ So you want to learn pixel art? ⭐

🔹 Part 1 of ??? - The Basics!

Edit: Now available in Google Doc format if you don't have a Tumblr account 🥰

Hello, my name is Tofu and I'm a professional pixel artist. I have been supporting myself with freelance pixel art since 2020, when I was let go from my job during the pandemic.

My progress, from 2017 to 2024. IMO the only thing that really matters is time and effort, not some kind of natural talent for art.

This guide will not be comprehensive, as nobody should be expected to read allat. Instead I will lean heavily on my own experience, and share what worked for me, so take everything with a grain of salt. This is a guide, not a tutorial. Cheers!

🔹 Do I need money?

NO!!! Pixel art is one of the most accessible mediums out there.

I still use a mouse because I prefer it to a tablet! You won't be at any disadvantage here if you can't afford the best hardware or software.

Because our canvases are typically very small, you don't need a good PC to run a good brush engine or anything like that.

✨Did you know? One of the most skilled and beloved pixel artists uses MS PAINT! Wow!!

🔹 What software should I use?

Here are some of the most popular programs I see my friends and peers using. Stars show how much I recommend the software for beginners! ⭐

💰 Paid options:

⭐⭐⭐ Aseprite (for PC) - $19.99

This is what I and many other pixel artists use. You may find when applying to jobs that they require some knowledge of Aseprite. Since it has become so popular, companies like that you can swap raw files between artists.

Aseprite is amazingly customizable, with custom skins, scripts and extensions on Itch.io, both free and paid.

If you have ever used any art software before, it has most of the same features and should feel fairly familiar to use. It features a robust animation suite and a tilemap feature, which have saved me thousands of hours of labour in my work. The software is also being updated all the time, and the developers listen to the users. I really recommend Aseprite!

⭐ Photoshop (for PC) - Monthly $$

A decent option for those who already are used to the PS interface. Requires some setup to get it ready for pixel-perfect art, but there are plenty of tutorials for doing so.

Animation is also much more tedious on PS which you may want to consider before investing time!

⭐⭐ ProMotion NG (for PC) - $19.00

An advanced and powerful software which has many features Aseprite does not, including Colour Cycling and animated tiles.

⭐⭐⭐ Pixquare (for iOS) - $7.99 - $19.99 (30% off with code 'tofu'!!)

Probably the best app available for iPad users, in active development, with new features added all the time.

Look! My buddy Jon recommends it highly, and uses it often.

One cool thing about Pixquare is that it takes Aseprite raw files! Many of my friends use it to work on the same project, both in their office and on the go.

⭐ Procreate (for iOS) - $12.99

If you have access to Procreate already, it's a decent option to get used to doing pixel art. It does however require some setup. Artist Pixebo is famously using Procreate, and they have tutorials of their own if you want to learn.

⭐⭐ ReSprite iOS and Android. (free trial, but:) $19.99 premium or $$ monthly

ReSprite is VERY similar in terms of UI to Aseprite, so I can recommend it. They just launched their Android release!

🆓 Free options:

⭐⭐⭐ Libresprite (for PC)

Libresprite is an alternative to Aseprite. It is very, very similar, to the point where documentation for Aseprite will be helpful to Libresprite users.

⭐⭐ Pixilart (for PC and mobile)

A free in-browser app, and also a mobile app! It is tied to the website Pixilart, where artists upload and share their work. A good option for those also looking to get involved in a community.

⭐⭐ Dotpict (for mobile)

Dotpict is similar to Pixilart, with a mobile app tied to a website, but it's a Japanese service. Did you know that in Japanese, pixel art is called 'Dot Art'? Dotpict can be a great way to connect with a different community of pixel artists! They also have prompts and challenges often.

🔹 So I got my software, now what?

◽Nice! Now it's time for the basics of pixel art.

❗ WAIT ❗ Before this section, I want to add a little disclaimer. All of these rules/guidelines can be broken at will, and some 'no-nos' can look amazing when done intentionally.

The pixel-art fundamentals can be exceedingly helpful to new artists, who may feel lost or overwhelmed by choice. But if you feel they restrict you too harshly, don't force yourself! At the end of the day it's your art, and you shouldn't try to contort yourself into what people think a pixel artist 'should be'. What matters is your own artistic expression. 💕👍

◽Phew! With that out of the way...

🔸"The Rules"

There are few hard 'rules' of pixel art, mostly about scaling and exporting. Some of these things will frequently trip up newbies if they aren't aware, and are easy to overlook.

🔹Scaling method

There are a couple ways of scaling your art. The default in most art programs, and the entire internet, is Bi-linear scaling, which usually works out fine for most purposes. But as pixel artists, we need a different method.

Both are scaled up x10. See the difference?

On the left is scaled using Bilinear, and on the right is using Nearest-Neighbor. We love seeing those pixels stay crisp and clean, so we use nearest-neighbor. 

(Most pixel-art programs have nearest-neighbor enabled by default! So this may not apply to you, but it's important to know.)

🔹Mixels

Mixels are when there are different (mixed) pixel sizes in the same image.

Here I have scaled up my art- the left is 200%, and the right is 150%. Yuck!

As we can see, the "pixel" sizes end up different. We generally try to scale our work by multiples of 100 - 200%, 300% etc. rather than 150%. At larger scales however, the minute differences in pixel sizes are hardly noticeable!

Mixels are also sometimes seen when an artist scales up their work, then continues drawing on it with a 1 pixel brush.

Many would say that this is not great looking! This type of pixels can be indicative of a beginner artist. But there are plenty of creative pixel artists out there who mixels intentionally, making something modern and cool.

🔹Saving Your Files

We usually save our still images as .PNGs as they don’t create any JPEG artifacts or loss of quality. It's a little hard to see here, but there are some artifacts, and it looks a little blurry. It also makes the art very hard to work with if we are importing a JPEG.

For animations .GIF is good, but be careful of the 256 colour limit. Try to avoid using too many blending mode layers or gradients when working with animations. If you aren’t careful, your animation could flash afterwards, as the .GIF tries to reduce colours wherever it can. It doesn’t look great!

Here's an old piece from 2021 where I experienced .GIF lossiness, because I used gradients and transparency, resulting in way too many colours.

🔹Pixel Art Fundamentals - Techniques and Jargon

❗❗Confused about Jaggies? Anti-Aliasing? Banding? Dithering? THIS THREAD is for you❗❗ << it's a link, click it!!

As far as I'm concerned, this is THE tutorial of all time for understanding pixel art. These are techniques created and named by the community of people who actually put the list together, some of the best pixel artists alive currently. Please read it!!

🔸How To Learn

Okay, so you have your software, and you're all ready to start. But maybe you need some more guidance? Try these tutorials and resources! It can be helpful to work along with a tutorial until you build your confidence up.

⭐⭐ Pixel Logic (A Digital Book) - $10 A very comprehensive visual guide book by a very skilled and established artist in the industry. I own a copy myself.

⭐⭐⭐ StudioMiniBoss - free A collection of visual tutorials, by the artist that worked on Celeste! When starting out, if I got stuck, I would go and scour his tutorials and see how he did it.

⭐ Lospec Tutorials - free A very large collection of various tutorials from all over the internet. There is a lot to sift through here if you have the time.

⭐⭐⭐ Cyangmou's Tutorials - free (tipping optional) Cyangmou is one of the most respected and accomplished modern pixel artists, and he has amassed a HUGE collection of free and incredibly well-educated visual tutorials. He also hosts an educational stream every week on Twitch called 'pixelart for beginners'.

⭐⭐⭐ Youtube Tutorials - free There are hundreds, if not thousands of tutorials on YouTube, but it can be tricky to find the good ones. My personal recommendations are MortMort, Brandon, and AdamCYounis- these guys really know what they're talking about!

🔸 How to choose a canvas size

When looking at pixel art turorials, we may see people suggest things like 16x16, 32x32 and 64x64. These are standard sizes for pixel art games with tiles. However, if you're just making a drawing, you don't necessarily need to use a standard canvas size like that.

What I like to think about when choosing a canvas size for my illustrations is 'what features do I think it is important to represent?' And make my canvas as small as possible, while still leaving room for my most important elements.

Imagine I have characters in a scene like this:

I made my canvas as small as possible (232 x 314), but just big enough to represent the features and have them be recognizable (it's Good Omens fanart 😤)!! If I had made it any bigger, I would be working on it for ever, due to how much more foliage I would have to render.

If you want to do an illustration and you're not sure, just start at somewhere around 100x100 - 200x200 and go from there.

It's perfectly okay to crop your canvas, or scale it up, or crunch your art down at any point if you think you need a different size. I do it all the time! It only takes a bit of cleanup to get you back to where you were.

🔸Where To Post

Outside of just regular socials, Twitter, Tumblr, Deviantart, Instagram etc, there are a few places that lean more towards pixel art that you might not have heard of.

⭐ Lospec Lospec is a low-res focused art website. Some pieces get given a 'monthly masterpiece' award. Not incredibly active, but I believe there are more features being added often.

⭐⭐ Pixilart Pixilart is a very popular pixel art community, with an app tied to it. The community tends to lean on the young side, so this is a low-pressure place to post with an relaxed vibe.

⭐⭐ Pixeljoint Pixeljoint is one of the big, old-school pixel art websites. You can only upload your art unscaled (1x) because there is a built-in zoom viewer. It has a bit of a reputation for being elitist (back in the 00s it was), but in my experience it's not like that any more. This is a fine place for a pixel artist to post if they are really interested in learning, and the history. The Hall of Fame has some of the most famous / impressive pixel art pieces that paved the way for the work we are doing today.

⭐⭐⭐ Cafe Dot Cafe Dot is my art server so I'm a little biased here. 🍵 It was created during the recent social media turbulence. We wanted a place to post art with no algorithms, and no NFT or AI chuds. We have a heavy no-self-promotion rule, and are more interested in community than skill or exclusivity. The other thing is that we have some kind of verification system- you must apply to be a Creator before you can post in the Art feed, or use voice. This helps combat the people who just want to self-promo and dip, or cause trouble, as well as weed out AI/NFT people. Until then, you are still welcome to post in any of the threads or channels. There is a lot to do in Cafe Dot. I host events weekly, so check the threads!

⭐⭐/r/pixelart The pixel art subreddit is pretty active! I've also heard some of my friends found work through posting here, so it's worth a try if you're looking. However, it is still Reddit- so if you're sensitive to rude people, or criticism you didn't ask for, you may want to avoid this one. Lol

🔸 Where To Find Work

You need money? I got you! As someone who mostly gets scouted on social media, I can share a few tips with you:

Put your email / portfolio in your bio Recruiters don't have all that much time to find artists, make it as easy as possible for someone to find your important information!

Clean up your profile If your profile feed is all full of memes, most people will just tab out rather than sift through. Doesn't apply as much to Tumblr if you have an art tag people can look at.

Post regularly, and repost Activity beats everything in the social media game. It's like rolling the dice, and the more you post the more chances you have. You have to have no shame, it's all business baby

Outside of just posting regularly and hoping people reach out to you, it can be hard to know where to look. Here are a few places you can sign up to and post around on.

/r/INAT INAT (I Need A Team) is a subreddit for finding a team to work with. You can post your portfolio here, or browse for people who need artists.

/r/GameDevClassifieds Same as above, but specifically for game-related projects.

Remote Game Jobs / Work With Indies Like Indeed but for game jobs. Browse them often, or get email notifications.

VGen VGen is a website specifically for commissions. You need a code from another verified artist before you can upgrade your account and sell, so ask around on social media or ask your friends. Once your account is upgraded, you can make a 'menu' of services people can purchase, and they send you an offer which you are able to accept, decline, or counter.

The evil websites of doom: Fiverr and Upwork I don't recommend them!! They take a big cut of your profit, and the sites are teeming with NFT and AI people hoping to make a quick buck. The site is also extremely oversaturated and competitive, resulting in a race to the bottom (the cheapest, the fastest, doing the most for the least). Imagine the kind of clients who go to these websites, looking for the cheapest option. But if you're really desperate...

🔸 Community

I do really recommend getting involved in a community. Finding like-minded friends can help you stay motivated to keep drawing. One day, those friends you met when you were just starting out may become your peers in the industry. Making friends is a game changer!

Discord servers Nowadays, the forums of old are mostly abandoned, and people split off into many different servers. Cafe Dot, Pixel Art Discord (PAD), and if you can stomach scrolling past all the AI slop, you can browse Discord servers here.

Twitch Streams Twitch has kind of a bad reputation for being home to some of the more edgy gamers online, but the pixel art community is extremely welcoming and inclusive. Some of the people I met on Twitch are my friends to this day, and we've even worked together on different projects! Browse pixel art streams here, or follow some I recommend: NickWoz, JDZombi, CupOhJoe, GrayLure, LumpyTouch, FrankiePixelShow, MortMort, Sodor, NateyCakes, NyuraKim, ShinySeabass, I could go on for ever really... There are a lot of good eggs on Pixel Art Twitch.

🔸 Other Helpful Websites

Palettes Lospec has a huge collection of user-made palettes, for any artist who has trouble choosing their colours, or just wants to try something fun. Rejected Palettes is full of palettes that didn't quite make it onto Lospec, ran by people who believe there are no bad colours.

The Spriters Resource TSR is an incredible website where users can upload spritesheets and tilesets from games. You can browse for your favourite childhood game, and see how they made it! This website has helped me so much in understanding how game assets come together in a scene.

VGMaps Similar to the above, except there are entire maps laid out how they would be played. This is incredible if you have to do level design, or for mocking up a scene for fun.

Game UI Database Not pixel-art specific, but UI is a very challenging part of graphics, so this site can be a game-changer for finding good references!

Retronator A digital newspaper for pixel-art lovers! New game releases, tutorials, and artworks!

Itch.io A website where people can upload, games, assets, tools... An amazing hub for game devs and game fans alike. A few of my favourite tools: Tiled, PICO-8, Pixel Composer, Juice FX, Magic Pencil for Aseprite

🔸 The End?

This is just part 1 for now, so please drop me a follow to see any more guides I release in the future. I plan on doing some writeups on how I choose colours, how to practise, and more!

I'm not an expert by any means, but everything I did to get to where I am is outlined in this guide. Pixel art is my passion, my job and my hobby! I want pixel art to be recognized everywhere as an art-form, a medium of its own outside of game-art or computer graphics!

This guide took me a long time, and took a lot of research and experience. Consider following me or supporting me if you are feeling generous.

And good luck to all the fledgling pixel artists, I hope you'll continue and have fun. I hope my guide helped you, and don't hesitate to send me an ask if you have any questions! 💕

My other tutorials (so far): How to draw Simple Grass for a game Hue Shifting

[gay--dog]

one thing about caring even a little bit about tech is you immediately learn to put your guard way up upon hearing the words "streamlined" and "effortless" or anything along those lines

[anasabdin]

🎨 You don't need cutting-edge tools or a high-end machine to create meaningful art. This pixel art landscape was created entirely in MS Paint, the same simple program that comes pre-installed on Windows. I use it for most of my background artworks because it helps me focus purely on composition, color, and detail without overcomplicating the process. Even my PC is a humble, older machine. Yet despite the limitations, my work has been featured in major galleries and on screens around the world. For animation, I use other tools (mainly because MS Paint doesn’t support animation), but I still apply old-school onion skinning techniques to keep the creative process simple and hands-on. The takeaway? You don’t need fancy software or powerful hardware to make something beautiful. Creativity finds a way.

[mgcem]

Experience the Simple and Beautiful White Walls with India's No.1 Premium Powder-Based Cement Primer MG PLUS.💫

For @mgcem / MG PLUS Dealership / Distributorship,

Call/Whatsapp 080049 82291

India Ka Number One Premium White Cement Wash MG CEM aka an Advance Replacement to the Best White Cement in India & Premium Powder Based Cement Primer MG PLUS Ki Details Ke Liye, Visit 🌐 www.mgcem.com

#MG #Cem #Cement #Plus #Primer #Premium #PowderBased #Paint #Base #Coat #WhiteWash #LimeWash #First #Maharaja #Eshan #White #Ujala #Coverage #Whiteness #Wall #Putty #Smart #Lime #Gold #Balaji #Shree #Plaster #WhiteCement #MastermindGroup