#code.tryperse

[perseverantdt]

Thanks for the tag, @a-fox-studies! I don’t really know that many codeblrs, so I’m leaving this as an open tag for others.

At what point in your life did you decide that the world of computers was the one for you?

I wanted to make games long before I knew what it takes to make one. All I knew is that I enjoyed games, and that I wanted to make something enjoyable too. So I guess, from the start, I was going to get hooked to computers one way or another. Knowing past me, though, I would not have gone this path if I found programming too hard, so the incremental steps I was given was a factor.

Back in 8th grade, we had to take C++ in our computer subject. It was just a general computer subject but it was nothing like I had taken in earlier years. We didn’t glance at any of the STL aside from strings and iostream (for stdin/stdout), so it was really enjoyable for me. Plus, we were spared from pointers because we didn’t have enough time for the school year to discuss any of those.

The next year, I was invited by our teacher to join the robotics team. We didn’t do well, but I liked that time of my studies. We went to numerous seminars, all of which I enjoyed (especially that time where we controlled a robot with our phones; that was a blast making it roam all around the room). I tried to make a piezo electric speaker sing, but I didn’t like how the piezo sounded, so I stopped after a few notes. We tried to make an RFID reader work for a competition. It worked in practice but, without changing anything in our setup, failed during presentation. Thankfully, it wasn’t an on-stage presentation, so only the judges (and the teams beside us) saw us trying to figure out why our working code wasn’t working.

I tried C# on my own time. I asked our computer teacher about the programming language we use for the Arduino and he said “C#”. In hindsight, he probably didn’t know as well, but thought that it looked similar to C#. It challenged me, which was my main motivation for learning. However, it wasn’t too hard that I dropped it. My previous experience with C (C++ implies STL) meant that I knew the syntax. My experience with Arduino meant that strange problems weren’t that strange to me anymore. Plus, using Visual Studio made it easy for me to transfer my idea to code.

From then on, I knew that I wanted to pursue this path. I continued learning C#, explored other languages, and made some of my own packages, modules, and applications. I have yet to create a game of my own, but that will come when I can finally force myself to make sprite assets.

[perseverantdt]

I'm all for AI being used to help people in what they do. Asking ChatGPT to write your essays is not having AI help you with something. That is delegating a task to an AI. That will fix your short-term problem of having to make an essay. However, you will miss out on what essay writing is meant to do for you. Essay writing isn't about the end product (though, that's still a big factor). It's about the process itself. It's about how you approach the topic and how you can convert an idea into writing. Generative AI can help you with that, but not by writing the entire essay for you.

Also, I don't like AI detection tools being used as an argument against AI. Their effectiveness depends on how many students actually submit AI generated work for their studies. Assuming a 100% detection rate for actual AI generated submissions, a 10% false positive rate means that at least 9% of submissions must be AI generated for the true positives to outnumber the false positives. I don't know what percent of submissions are AI generated, but 9% of all submissions is an extremely large percentage. Solution for this claim is in the image below.

I'd like to have great AI detection tools. But having an aggressive detection mechanism does not help anyone. It makes positive results questionable at best and dismissible at worst. At its current state, AI detection tools aren't as great as people make them out to be.

I feel like the only person not tempted to use ChatGPT like it doesn’t even occur to me as an option

[perseverantdt]

I keep seeing all these posts about ChatGPT and stories and I'm just sitting here with this post I'm still working on because holy fuck this thing is so deep. Just a quick note to those who wish to know what OpenAI used to train ChatGPT.

The Common Crawl dataset that OpenAI used to pre-train GPT-3 (and ChatGPT by extension) is named CC-MAIN-2021-39. This is the dataset that Common Crawl released in September 2021, which matches with the knowledge cut-off date on ChatGPT. CC-MAIN-2021-39 contains 97 versions of Ao3's homepage all from different times.

Since CC datasets are notorious for being extremely low quality when it comes to training data, OpenAI filtered CC-MAIN-2021-39 through a classifier based on WebText, a dataset that OpenAI used to train GPT-2. This dataset was made from a web scrape that OpenAI developed. To scrape the web for WebText, they used Reddit as a starting point, scraping all outbound links from it which had garnered at least 3 karma. After that, they only scraped web pages which have been "curated/filtered by humans".

I urge everyone to have a deeper look at how ChatGPT works. It doesn't matter if you're for or against AI. Look at how it works because there's a whole lot of misinformation about this thing.

[perseverantdt]

I have managed to install enough emulators on my phone to make it run LOVE2D.

Frame rate sucks, because of course it will, but it should work as a development platform.

[perseverantdt]

An Intro Post

So much to do... so little time...

About Me

Howdy! The name's Perseverant Determination, but that's exteremely long. Feel free to call me anything, though Perse will do just fine. You might also see me in other places under other names, especially EF159. Feel free to use that as well.

I do a lot of things, which means that this main blog will have an assortment of topics. I create beatmaps for VSRGs, write stories, and code personal programs. I'll tag my posts as shown below, so you can filter my posts.

#the-void's-determination: For all stuff related to writing

#code.tryperse: For all stuff related to programming

#perse-the-beat: For all stuff related to rhythm games and beatmapping

If you have questions about a certain topic, feel free to ask me. While I'm more likely to answer programming-related questions, I'll try to do my best to give you satisfactory answers.

The Void's Determination

I like writing sci-fi, though lately, I find myself preferring fantasy more. I've been writing for a few years and have a few short stories on r/WritingPrompts, but those are years old now and they're not that good, in my opinion. If you can find them somehow, good for you. I'm not linking to them.

Projects

Days of Moonlight Riddles

I play Sky, that game of Light. Riddles, I see with great delight. I might not solve them all the time. I might be wrong but still, I'll try.

Along with Duet's last two weeks, The lanterns cast, for all to see. In each of them, a riddle held. Answer three — three tickets left.

And thus, to bring my prose to test, A riddle a day; I bring my best. If you so seek to answer those, Look below and have your go.

Riddles: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]

Ethereal Dreams (currently being reworked from the ground up)

Code.TryPerse()

I have a side blog for this now!

I consider myself experienced in C#, though I've also practiced a bit of C/C++, Java, Javascript/Typescript, Python, and Lua. I don't do much with that knowledge though. I just code some stuff that can help me with random trivial problems. I want to be a game developer but that's going to take time, what with me doing all this writing and beatmapping.

[perseverantdt]

Thanks for the insights, too. I'm planning to make a post about what happens to prompts given to ChatGPT. I'm not done with writing the thing, but these help me figure out what exactly I should be giving focus on that post.

I'm pretty sure that people who were developing AI art generators continued scraping the web even after they released working models to the public. That whole thing about the art obfuscator software which made art unreadable for AI art generators kinda proved that point. If people weren't continuously scraping the web for new art, they wouldn't be so outraged that artists started using a software that turned their art into absolute trash when viewed by an AI tool. (Real amazing software, that one.) After all, you wouldn't get data poisoning if you no longer added data to your training sets.

If they allowed users to feed their datasets with images, then they also run into the same data poisoning issues. After all, no one's stopping a large group of users from feeding it with trash. And quite frankly, with how they vaguely focused on the fact that people can generate art that mimicked the style of artists, that kinda destroyed the possibility of artists helping to train those AI art generators. They inadvertently marketed those generators as a replacement for artists despite saying that those should just be tools for inspiration.

This is one of the main reasons I wasn't convinced by the idea that a prompt to ChatGPT gets used for its training data. There's an entire year of data missing on ChatGPT when they released it. It doesn't make sense for them to suddenly add prompts to its training data. Plus, it would be very easy for people to feed the AI with garbage.

After looking deeper into it, though, it's actually very nuanced. I read through every single one of OpenAI's policies, and I'm still not sure on how it works. Here's a short answer based on what I know so far. This might change depending on what I find, but this seems to be the most plausible answer.

Disclaimer: I do not claim to be an expert in large language models, nor in machine learning in general.

No. I don't think ChatGPT will start making responses that mimic an author's style just by giving it prompts. As long as the prompts have nothing on them that can relate a given piece of writing to an author, I don't see how it would make the inference that that piece of writing was written by that author. However, assuming that OpenAI continues using RLHF with Proximal Policy Optimizations, then the prompt-response pair will be used to train the reward model which gets used to train the text completion model on what it should output. So yes, prompting ChatGPT with people's writings means that it will get trained on those writings. However, I think it wouldn't be used as a basis for output, but rather as a basis for input. I wouldn't hold my hopes that it does work like that, though.

Howdy!

I recently saw a reply you made in a post about feeding unfinished fic to large language models like ChatGPT. I'm interested in the idea that giving an LLM a prompt allows it to use that prompt as a datapoint for future iterations. I've heard of this idea before but haven't figured out where it came from. Can you tell me more about the origin of that?

I'm not an expert and don't feel like digging too deep into it, but take these two quotes from makeuseof.com (accessed 2023-06-09):

Considering OpenAI's privacy policies, you can rest assured that your data will remain safe. ChatGPT only uses conversations for data training. Its developers study the collected insights to improve output accuracy and reliability, not steal personal data.

ChatGPT uses supervised learning techniques. Although the platform remembers all inputs, it doesn't learn from them in real-time. OpenAI trainers collect and analyze them first. Doing so ensures that ChatGPT never absorbs the harmful, damaging information it receives.

That, coupled with my meagre understanding, says yes; feeding ChatGPT one of my fics would mean it could be used to train the AI. As I said in that other post, I think I should have the moral (maybe not legal, since, you know, fanfic) right to say no to that.

(Note that this is a seperate issue from the webscraping used to collect training data in the first place. So, yes, sure, maybe my writing has already been fed into the algorithm, but that just means my consent has been violated before. So there's that.)

Oh, also, I vaguelly remember a campaign to flood google images with pictures of read text saying "No to AI art" which started making art AI's go wonky. That wouldn't work if those AI weren't either still scraping google images or if users weren't feeding them pics.

But maybe someone is more of an expert and can fill in.

Thanks for the ask:)

[perseverantdt]

Would have been nice to see this mistake much, much earlier.

[perseverantdt]

It's weird seeing Minecraft scoreboard documentation written on the same site where I look at C# documentation, but here we are. And it actually seems useful.

[perseverantdt]

Indexing 160 million entries turns out to take a long time. Who would have guessed?

[perseverantdt]

With April 1st coming up in two weeks, it might be the time to start figuring out how to make a websocket client.

Just in case.

[perseverantdt]

That post I made about Visual Studio just made me remember why I initially chose Visual Studio over Rider.

I might try Visual Studio with Resharper to get Rider's analysis capacity with Visual Studio's clean UI.

[perseverantdt]

That can’t be good.

[perseverantdt]

Thanks for the insights. I've been looking into where the idea that prompts you give to an LLM get used as a basis for responses came from. While I'm still not fully sure about it, I have a clear possibility for how that idea came to be. Once I'm done looking up on it, I might make a post about that.

Generative AI technology is at the state where it's powerful enough to be convincing at first glance, but not powerful enough to be completely undetectable. Because of this, and the lack of countermeasures (for the lack of a better term) against abusive use of these tools, people and corporations use these tools with wanton disregard for the effects of using such tools.

As much as I want to say that people shouldn't claim AI generated art/writing as their own, it's pretty hard to make a compelling case towards adversarial parties when the companies making these AI tools "hereby assigns to you all its right, title and interest in and to Output." Hoping that people won't do that is wishful thinking. There will always be someone who will claim the work of AI tools as their own. Putting a disclaimer that "any content regarding medical, legal, financial, or other professional advice is provided for informational purposes only and is not a substitute for advice from a qualified professional" will not prevent people from using them as a substitute for advice from a qualified professional.

I'm all for the development of AI. I use ChatGPT to help me in worldbuilding. I use Codeium whenever I'm coding my personal projects. But until this issue about AI generated items gets resolved, until people and corporations have come into an agreement over what's alright and what's not, I can't fully support their development. I can't look someone in the eye and say that AI is good for society. It's not. Not yet.

Howdy!

I recently saw a reply you made in a post about feeding unfinished fic to large language models like ChatGPT. In there, you said that free-to-use AIs are "free to use because the company who makes them is actively profiting and taking aspects like phrasing, style, etc." I've heard of variants of this idea but could never figure out where it came from. Can you tell me more about the origin of that?

Well I'll start off by saying I'm not an expert in the field, and that AI technology is actually relatively "new" in terms of very recent rapid developments and it's in this stage where the future is generally uncertain. However, as early as now, people are already using AI for commercial works. There was a movie announced where all backgrounds were made with an AI, and a manga that was made near entirely with AI. The rhythm game Cytus has recently drawn ire from fans and even its own creators and designers for using AI generated art instead of the very stylistic art it used to use. The moral and economical issue here would lie in that they still charge the same amount for the game and it's already a game that is criticized by some for its high price point compared to other rhythm games. So not only are they earning more, in a way are putting less effort into the game. Those who purchase it would essentially be paying more for less and the money would not be going to the artists because their styles will have been copied but they themselves wouldn't be credited or paid.

Now for a more direct example, I've also heard reports that publishing houses are being spammed with hundreds of purely AI generated work. This of course directly negatively affects writers trying to get published if only by uhhh what's the word,,,, adding a bunch to the pool (sorry English is my third language).

I myself do writing for companies overseas by editing and writing the info on their websites. As someone from a third world country, this contributes to a good chunk of my income as the local economy is not great. Due to the recent popularity of bots like chat GPT, the market had dropped by a LOT and while at first it was my primary source of income, its not really sustainable anymore.

Now these examples don't all use language based examples but they do show that the creation of AI art forms so far is mostly just harming artists and from the examples such as Cytus and just,, life in general I suppose, as "young" as commercial AI is, we can likely assume that corporations don't intend on making their AI for purely wholistic reasons. The art AI midjourney was even found using stolen art after saying that they don't. AI writing, I believe, is going to be more finicky than AI art and as such, I expect theyd want to do a lot more fine tuning before using it to write whole textbooks while art bots are already being used on book covers and movies, only because you don't need to fact check art.

Sure people can see something is wrong, but with the sheer amount if it used, it's rapidly adapting to fix the anatomical issues in its art. Written work I think would need to be double checked a lot and by a human before they release one making bold claims such as "You can't buy food but you can buy paint thinner from home depot instead" (an actual AI result generated by Quora when I was trying to find a place to eat that Google tool as the top rated answer on the website and proudly presented to me). But that doesn't mean it won't eventually go as far as to take someone's job entirely. It's already starting to take mine.

For a clear cut example, sorry to say I can't name one myself, but you can look at the way AI is already being used this early on and how it's already being used to substitute and replace some artists and writers and how apparently even fanfiction writers who do their work out of love, and look just a few years in the future based on the patterns that have been happening and the way corporations will always value profits over the heart of what they make, and for most the picture of what will happen is a very grim one for art.

The "Origin" of it differs from person to person. Some artists have seen their art put into AIs and their styles mimicked (art which will be very difficult to claim the person who generated it shouldn't be allowed to use for commercial purposes). Some writers who write more boring industry stuff that is very easy to mimic are getting their jobs taken away from them. Others without firsthand experience can only look at examples or patterns and infer a probable and large scale outcome similar to that of Cytus. All in all, to me the backlash and opinion that AI is copying peoples works is more of a social movement with no clear cut origin but a lot of evidence that points towards AI generated writing and imagery being a bastardization of the work of hundreds even if it's just a lot harder to see when it comes to a non visual form of art like writing.

Hmm I think if you want a clearer answer or example, the best personal one I can give you is an article I edited which was so poorly written I sent it back and they had a different writer do it. When it got back to me it was better, but extremely familiar. It repeated phrases from the OG article and had the same problems I had noted (strange wording, odd vocabulary, etc) so I asked them if they had wrote it. Apparently they just put it into chat GPT and told the bot to rewrite it without changing too much, so the bot mimicked phrases and words but changed the flow by adding conjunctions or paraphrasing, but to me, who has read the first persons work several hundred times, I still recognized the style, if I can call it that. The person profiting wasn't Chat GPT, but if the state of AI art is anything to go by, in a few years it could very well be.

(sorry about the long reply and if anything is messy or hard to understand. I am not an organized thinker.)

[perseverantdt]

For context, this puzzle was marked as an easy puzzle but, for some reason, I decided to solve the thing in C.

[perseverantdt]

Who would have known that porting a mod would be this difficult?

[perseverantdt]

Here goes nothing.