#stackexchange moderation

[gudguy1a]

Stupid, Crappy StackExchange Moderation!!!

Please be advised, there is quite a bit more FREE SPEECH expression being used here – so continue at your peril (or damnation)… I SWEAR TO SATAN HIMSELF (or Her) (and I’m not religious) – this damn stackexchange site, yeah…. The one that has been declining for some years now, due to their overbearing, overreaching moderation style. Just do a quick search regarding the turmoil and some chaos that…

[tanadrin]

I've been using a spreadsheet to do sanity checks on Pr's astrophysics--it doesn't have to be exact (I'm not going to try to solve equations of state for neutron stars or anything like that), but I want to be at least in the right ballpark. And that means not fudging the intensity of a magnetar's magnetic field by 10 orders of magnitude, which is so far out of the ballpark we've started playing a different game altogether

If the magnetic field around Ast isn't produced by the star itself, or is only partially produced by the star itself--if it is produced in, say, an accretion disk close to the star (in this case extending out to about 150,000 km or more)--then the magnetic field doesn't need to be nearly as intense; it could actually be quite moderate. You'd need a hell of a lot of current moving through the accretion disk, but only the disk as a whole, and the exact mechanics of this would depend on the magnetohydrodynamics of an accretion disk around a neutron star.

Which I am comfortable saying that--at least in the configuration Ast presents--is not likely to be a natural phenomenon. But Pr is, of course, artificial; my concern isn't whether such a system could form in the first place, but whether it could remain stable for ten million years or so, and I think it could.

Accretion disk lifetime depends on the luminosity of the disk and its mass; the approximation I'm using here is from a physics stackexchange answer about accretion disks around black holes, and I'm assuming that they hold roughly for accretion disks around neutron stars; the neutron star's magnetic field would distort the inner part of the disk as it fell onto the star's surface; the real question is whether the gravity of the neutron star would be enough to keep the disk from being evaporated by its radiation, and the answer here seems clearly to be yes: after all, planets are known around neutron stars, and they take quite some time to form.

Ast also has to remain quite hot throughout its lifetime, but this is actually not something I'm too worried about--for Plot reasons, it's actually getting hotter over time, which, yes, neutron stars generally shouldn't be able to do.

[sp-epari-digitalmedia]

Critical Analysis

Nardone, V. et al. (2023) 'Video game Bad Smells: What they are and How developers perceive them,' ACM Transactions on Software Engineering and Methodology, 32(4), pp. 9-12. https://doi.org/10.1145/3563214.

Introduction

The abstract presents an in-depth analysis of video game's "bad smells," categorizing issues into five distinct groups that include design, game logic, multiplayer, animation, physics, and rendering. It presents a methodical examination of every category, going over particular bad smells and offering instances from pertinent conversations. Using information from survey participants, the paper provides workable ways to reduce these bad smells. To ensure traceability, the study connects every "bad smell" and its remedy directly to discussion boards or poll results, enhancing the legitimacy of the suggested fixes.

Bad Smells

According to the authors, this dual approach in the study, strengthens the paper's overall reliability, making it a valuable resource for both game developers and researchers. One noteworthy finding is the fact that respondents disagreed on how important some issues are. A few perspectives present the idea that the impact of a particular challenge can change depending on the project's size and how much moderation is used. This highlights how game development challenges are subjective and that the importance of a problem varies depending on the context.

Weak temporization strategies are a major focus of the analysis, with 80% of respondents rating them as critical or extremely critical. In particular, when it comes to frame-based updates, the highlighted issue involves incorrect assumptions about the amount of time that has passed between game object updates. To provide a consistent gaming experience, it is crucial to address this issue as it may lead to differences in animation speed among devices.

Respondents offer workable solutions to address these inadequate temporization strategies and lessen the problems noted. Typical errors such as frame-based updates affecting animation speed, are identified and workable fixes are proposed. Time-based updates, like the FixedUpdate method in Unity, should be implemented. Additionally, movement should be proportionate to the intervals between frames. A few respondents' emphasis on these solutions lends credibility to their efficacy in addressing common game development pitfalls. The analysis also emphasizes how crucial early intervention is in resolving temporization issues. one of the respondents makes it clear that the longer these issues persist, the more difficult it is to address them. This insight highlights how proactive developers should be in spotting and fixing important problems early on in the process to avoid problems later.

A real-world example from the Unreal Engine forum provides tangible evidence of developers grappling with temporization issues in a racing multiplayer game. The discussion highlights the practical application of "delta time scaling" as a solution to address input responsiveness, illustrating how these challenges manifest in real projects.

Focusing on the lack of separation of concerns and the issue of bloated assets. While these challenges may not be unique to the gaming industry, they are pertinent in the context of game development due to the specific characteristics of game engines.

One sign of poor design that can appear in a variety of software programs, including video games, is the inability to separate concerns. The observation that game engines may naturally lead developers to write code that exhibits this problem led to the decision to keep this "smell" in the catalog. In the example given, source code for object animations is mixed with code handling controller inputs; this practice was deemed critical or extremely critical by 77% of respondents.

The significance of this design challenge is further highlighted by the discussion on StackExchange's Game Development section. Logic and data in the same object or class, according to platform developers, is bad practice that can result in hacking that creates more problems than it fixes. It is recommended that game logic and game data be kept apart, and that a modular approach be used, with different classes handling different concerns like movement, firing, and defense. Notably, Unity and other contemporary game engines are taking proactive measures to address this problem by including features like an updated input system. Bloated assets, or reusable assets like complicated game objects that bring along different elements like textures and predefined animations, are the subject of the second design challenge that was discussed. Sixty-three percent of respondents rated this "smell" as critical or extremely critical. Real-world examples are given in the Stack Exchange game development discussion, including scenes with superfluous art assets and scripts. Not only do developers advise eliminating unnecessary assets to conserve space, but they also point out potential problems like mismatched names and superfluous animations if this is overlooked.

Comprehending the issues raised, specifically the inability to distinguish between different concerns and the problem of excessively large assets, provides game designers looking to improve their work with insightful knowledge. Not only do these detected "smells" diagnose possible problems, but they also act as a helpful roadmap for better game design. Recognizing the inability to separate concerns as a design challenge invites a game designer to take a more modular and structured approach. The division of various game elements, like controller inputs and object animations, into discrete areas allows designers to simplify code, lower complexity, and improve maintainability. The Stack Exchange discussion on game development, which is cited, emphasizes how crucial this division is to preventing potential problems and unethical behavior. This realization can serve as inspiration for designers of video games, who can use it to create modular systems that support a variety of features.

The issue of bloated assets provides another avenue for constructive improvement. Game designers can leverage the awareness that excessively large assets, containing elements rarely used, are viewed critically by developers. By adopting a more discerning approach to asset creation and management, designers can optimize game performance and streamline the development process. The practical recommendations from developers on Stack Exchange Game Development, emphasizing the removal of unnecessary assets for both space-saving and organizational cleanliness, can guide game designers in creating more efficient and polished games.

Conclusion

It gives game designers more options because it acknowledges that some game engines are actively resolving these problems, as demonstrated by Unity's new input system. Designers can ensure that their games follow industry best practices and utilize state-of-the-art tools by keeping up with evolving engine features and adapting their practices accordingly. The study overall is relevant as it recognizes the bad indications in game designs and therefore validate a better quality of games if these problems are avoided.

[olderthannetfic]

I don't agree with censoring fanworks created by humans but I *am* actually against AI content, especially fanfics and fanart. So you think they'd walk back on allowing it on AO3? Have you heard about the Stack Exchange mods' strike? Meta. stackexchange. Com/questions/389811/moderation-strike-stack-overflow-inc-cannot-consistently-ignore-mistreat-an

--

I hadn't heard about it, no.

[technologywhis]

Ah, shadowbans — that’s a great topic, and it’s more common than many people realize.

A shadowban is when a user is partially or fully blocked from a platform (like a social media site or forum) without being notified. So, everything looks normal to the banned person — they can post, comment, like, etc. — but other users might not actually see their content, or their content gets deprioritized in algorithms (like not showing up in searches or feeds).

Common Platforms and How Shadowbans Work:

• Twitter/X: Your tweets might not show up in search or replies to others.

• Reddit: Your posts or comments might be invisible to everyone but you.

• Instagram/TikTok: Your content might get limited reach or not appear in hashtags.

• Forums (like StackExchange): Your posts might be auto-hidden or downvoted heavily by moderation bots.

Why Do Shadowbans Happen?

Usually for things like:

• Violating community guidelines (spam, hate speech, etc.)

• Using automation or bots

• Getting mass-reported

• Sometimes, mistakenly due to overly strict moderation filters

How Can You Tell If You’re Shadowbanned?

It can be tricky because it’s meant to be stealthy. But some tips:

• Ask a friend to check if they can see your posts.

• Use shadowban checking tools (for Twitter/X, Reddit, etc.)

• Notice sudden drops in engagement (likes, views, comments)

Would you like help checking if you’ve been shadowbanned on a specific platform?

[fishmech]

forums as people know them are inherently web 2.0. user generated content, submittable and live updating with in-browser interfaces, unlike previous forms of forums that most people have never used cuz the web side of them was just a reformatting of data that needed to be submitted from another way, usually email or nntp based. like thats why web forums as we know them barely existed before the late 90s, the technology to support them wasn't there yet.

also reddit was never a forum, it's practically an opposite kind of thing. its the bastard child of slashdot voting and auto-moderation via voting plus usenet. it inherently destroys long-running conversations because of the voting and attempt to bubble up a top set of replies, rather than maintaing conversation flow! that's what makes reddit much more like using slashdot, which is still around, or quora, or digg, or fark, or stackexchange or metafilter (which ok is pretty dead these days, but still).

hyper-reliance on reddit, which by the way is doing all these bullshit moves to its staff and users lately because Conde Nast is trying to get rid of it via selling it in an IPO, is a problem in and of itself. like i get its the easiest way to do things if you only trust and use or 1 or 2 sites for things like this, but it rots people's ability to cope with websites betraying you. which they always will.

It sucks that people are treating the Reddit blackout as a joke or assuming it's impotent rage over a minor decision bc it's Reddit when like. No, a tech company shutting down access to their API by forcing third-party devs to pay completely unreasonable fees ($12,000 per 50 million API requests, which to the largest third party clients would be tens of millions of dollars) and in the process destroying both accessibility apps and moderation tools is Bad Actually

[this-has-returned]

The worst kind of coding problem is the kind that makes all of the "fucking obvious" solutions completely unusable, so the power-tripping shitheads that moderate StackExchange simply refuse to let you receive anyone's assistance.

And if you try to explain your actual engineering constraints, people just go "what the fuck, are you stupid?" and continue to suggest all the "obvious" stuff, as if you're being unreasonable or don't know how coding works.

Like...that does not work. If the obvious solutions worked, I would not still be looking for help. I've already explained that the obvious stuff doesn't work.

Fucking hell, people.

I am tied up in the weirdest goddamn engineering constraints ever, and trying to Google possible solutions just gives me solutions for a completely different problem entirely, which has apparently been answered over 500 times.

All of the people who actually have my same problem have gotten laughed off of 5 different websites without any assistance.

Ugh.

[softgrungeprophet]

OKAY

first thing’s first, credit for the resources i used:

"Soda Bottle Orange Liquid" by smart61, Creative Commons Attribution

to make the clouded liquid i used this set of nodes by this dude on this stackexchange post: https://blender.stackexchange.com/a/190931

for the clear liquid i followed the blender guru glass coffee cup tutorial: https://www.youtube.com/watch?v=fdWUjEMzDhg

I mostly fiddled around with the colors, and while that took.... hours.... especially trying to find a way to render brominated soda that i liked (didn’t figure that one out lol).... most of the hard work was of course already done for me via that 3D model. i didn’t make the model. just to reiterate, it’s by smart61 on sketchfab.

okay so, what is this, you may be asking, or not asking, idk.

It’s soda!

actually i’m gonna put this under a cut so if you want to read my random Soda Worldbuilding for Sulum’s country (called Vadpukh) you can read ahead:

I wanted to do this 3d so i wouldn’t have to draw it by hand because i think that would have been awfully time consuming...... so i did this in Blender rendered in Cycles with the denoise filter on and using the indoors world... thingy for the lighting reflections (cause i felt it looked better with that than the plain render lighting i had)

I wanted to get a better idea of what the soda colors might look like from Vadpukh, where Sulum hails from, and I wanted to be able to have examples of them for reference for.... some reason? idk i got fixated on it. spent all day fucking with blender.

1: TOP row: these are yer red sodas... and watermelon. Why is watermelon soda yellow?! Because where Sulum is from, sweet watermelon is yellow, and citron melon (not used for soda) is white. From left to right: watermelon, strawberry, redcurrant, pomegranate, and blackberry.

Now, Vadpukh is... I suppose the best description is that it’s a semi-temperate coastal area that’s like... if the french riviera was a little more north? There are lots of berries available and so there are probably a lot of other berry flavors using these four shades of red but do not ask me to list them out. These are essentially just the most popular, quintessential Berries. and watermelon.

2: coconut soda comes in four different colors. blue is the classic, cause boy do they love food coloring, something about the ocean being blue, coconut crabs, and: marketing/fun. clear is moderately common but less fun and recognizable than blue. brominated white coconut soda is usually more of a coconut milky type flavor, less coconut water... but really not that much different. PINK coconut soda is (F)AU(X) NATUREL because coconut water turns pink when it oxygenates (i think) and some brands want to emulate that naturalness with food coloring. Though there are totally all-natural sparkling coconut waters that are pink too, so, check the ingredients i guess.

Also right here i’m going to mention it: Brominated Vegetable Oil. It’s not good for you. Most countries--nay, continents--had already banned it in drinks and various US soft drinks have also recently stopped using it (ie mtn dew, powerade, etc.) Because it’s bad for you.

Vadpukh is a fun country. Vadpukh is a semi-socialist democratic country which is attempting to recover from the problem at the root of all problems: Capitalism. It’s sort of working but there are still a lot of problems, notably in the health industry, politics, and the food industry. The country is notorious for a scandal with human flesh from like 20 years ago (one time, someone finds human meat in their pork, and suddenly that’s all anyone talks about.) Anyway. They’re at a very weird juxtaposition of like, still using BVO in a LOT of drinks, using a lot of artificial and natural food coloring alike, but also are really far ahead on the green-packaging front which is why actually these plastic bottles are wrong. Most soda is packaged in metal bottles or cans, or glass bottles, though plastic is used to some extent too. I just needed it to be clear.

Anyway. so yeah. lots of brominated sodas.

3: a whole row of grapes. you got your typical purplish-bluish (depends on manufacturer) grape soda made out of those kinds of grapes that are real dark, usually in a dark brown bottle but not always. then, green grape soda which is often straight up the color of mountain dew but it’s hard to tell cause it’s always in a green bottle. then you have your WHITE grape soda which i believe is just a culinary differentiation in which wine grapes are white and table grapes are green but anyway white grape soda, while also in a green bottle, is usually a light straw color or sometimes a pinkish gold... usually similar flavors tho. then we have our second blue soda: anise grape. It’s hard to tell partially because it’s a very deep blue, but it is, also, brominated. idk how that would look irl. the alt color is teal. why? because i was looking at anise syrups and liqueurs and thought, hm, these are all a kind of cloudy yellow, yellow plus blue equals--well, green, but in smaller amounts, teal... also brominated.

ah, the colors of gatorade fierce grape and gatorade frost arctic blitz.

4: stone fruit sodas, aka, the best kind? i’ve never had plum or apricot soda but it sounds good. Peach-rhubarb, a bit red, a bit peachy. Regular peach soda is also this color sometimes. Plum is yellow! Because many plums are yellow inside. This is a more muted amber color but it can also be as yellow as the watermelon soda depending on who’s making it. There’s also: green plum soda because I tried greengage plums and they are VERY sweet so i figured that would suit a soda well. And apricot. Color ranges from calmer more light oranges to full on orange crush neon orange, and it’s also used for plain peach sodas as well. and maybe some more specific oranges like tangerines? idk

moving on

5: Citrus. ALL the citrus sodas are brominated, which made me suffer because I could not render them to look nice at all. Grapefruit soda is a lovely shade of pink usually but it can sometimes come in more salmony, more red, or rarely yellowish-white (like Squirt). orange soda is supposed to be like, the color of orange juice... i struggled with it and it’s basically impossible to find pictures of how it should look because most orange soda is the color of kraft mac and cheese. maybe the best color comparison would be like.... if you mixed lemonade with a bit of OJ. lemon soda, lime soda, and lemon-lime soda (they all exist) all tend to look the same, generally resembling the cloudy whiteness of... Squirt. again. They can come in clear bottles, but lemon soda sometimes comes in sunny yellow bottles, and lime or lemon-lime can come in green ones. (like i said, usually glass, but sometimes plastic)

6: using citrus as a weird stepping point... the Herbaceous Sodas (and pistachio cause i had nowhere else to put it and it was green) For those who like a bracing soda, juniper-grapefruit. I don’t think I would like it but I’m sure some would. It’s usually the same color as regular grapefruit, sometimes a little more orangey. Pine soda is next, and includes stuff like juniper but also spruce. It was... basically impossible to replicate the spruce beers I was looking at but it’s roughly a sort of semi-cloudy translucent straw color... like--and i am realizing ironically the reference i was using was not spruce beer--spruce soda co’s hopped cola. OOPS.

Pistachio and pistachio-rose are both VERY popular flavors in Vadpukhi food and that extends to soda--both flavors of soda tend to be that pale pistachio green, either way. And the other green soda: cucumber. Like the pic says, sometimes it’s colorless. Basically, comes in the same weird color ranges as irl cucumber soda, ranging from colorless to a light minty green to that hulkly abomination of a color in the pic. flavor i’m sure wildly varies. i’ve tried dry cucumber soda, it’s pretty good (if unusual). I have also tried cucumber-lime gatorade, which tastes like extremely sweet pickle juice (bad) (it COULD be good but the pickle flavor... i can’t do it)

7: FLOWERS. lavender and rose are, again, very common flavors. they smell good, they taste good. Not pictured, but rosehip soda is probably one of the many red sodas you can find. But this floral section is lavender two ways: lavender soda that is a pale purple color itself, and clear lavender soda packaged in a lovely purple bottle. Then, rose, which is generally always a lovely soft shade of pink tho sometimes includes lemon or orange flavoring (at which point it probably is made cloudy to communicate that)

8: NOT colas. Vadpukh doesn’t have cola sodas (as in, based on the flavor of the kola nut) except in some specialty/ethnic shops and restaurants (or made by individuals.) This is the brown soda section. First up we have barley-malt and we have hops soda. They look very similar and are often packaged similarly, ie brown glass bottles or brown cans, and there can be a lot of overlap in ingredients ie a lot of the hops sodas have malt syrup. However most of the time malt soda is used to distinguish like.... malted milk powder flavor... but soda. Hops is beer soda. lol. Then it’s caramel soda which as I wrote on the pic is somewhat like vanilla cream soda but it doesn’t generally have vanilla in it. It’s caramel flavored. Delicious. Cinnamon soda is cinnamon spiced and often includes other spices as well, including ginger...

Then there’s honey soda which I have two nearly identical renders of lol. Most honey soda is a light ambery-yellow color, roughly the color of honey, maybe less warm tones... It can be salted, or spiced, combined with apple flavor... The other honey soda, barely distinguishable in color, is usually a warmer, redder amber tone and is used for things like berry-honey sodas.

-

NOT PICTURED: Tea soda which I completely skipped over on accident, sparkling juices (including apple and pear and everything else, more natural, less sweet, less dyes, no bromination), apple and pear soda which i guess i forgot too for some reason but lbr sparkling apple cider is better anyway. Ginger ales and lemonades. Like. Those are fairly self explanatory. Ginger ale ranges from straw to amber, ranges in spiciness, etc... lemonade can be plain, or pink, or lime, or what have you, similar to lemon or lemon-lime sodas but with extra flavors like pomegranate or peach.

Also not pictured are the Vadpukh “cream” sodas or milk sodas. These are not like American-style creme sodas, there’s no vanilla or anything. They’re more akin to Milkis or like... Calpico if Calpico wasn’t yogurt flavored. They’re carbonated sodas with dairy in them basically, like an ice cream float without the extra work. They are almost universally somewhat opaque, come in many of the same flavors as above, and tend to be like, soft pastels or off-whites. ie, orange milk soda would probably be the same color as mango Calpis.  A lot of people just make milk soda at home either by adding ice cream to soda or by adding heavy cream or milk, but the convenience of pre-bottled is nice. Plus it usually stays carbonated longer than if you pour your soda full of milk...

There is ALSO a sparkling carbonated yogurt drink similar to Calpis btw... which looks similar to the milk sodas but is tangier, fermented... probiotic. (i could drink it without destroying my digestive system)

Sparkling water, mineral water and club soda are also all very common and some come in flavor infusions. The Vadpukh citizens like bubbles. A lot.

There are also non-carbonated drinks like electrolyte drinks (tho those often also have carbonated versions), flavored milk, nesquik-style carob drink powder... and teas and stuff but I didn’t want to spend like, the whole week doing this 😂 Milk tea is real big (but not like boba, no chunks or tapiocas (Tragic), just milk tea and fruit milk teas and stuff)

What else... wine.... kefir... lassi-style drinks... fruit juices (usually unclarified but sometimes clarified)....

Anyway if you want to know, Sulum’s favorite soda flavor is basically any of the berries--blackberry, pomegranate, apple-berry, and darjeeling (rasp)berry especially. they love a blended darjeeling raspberry slush in the hot summer... they also like the simplicity of a nice ice-cold, unfiltered apple juice.

sometimes they like to add spicy pepper flavors. lol.

OH MAN I bet they have spicy pepper sodas...

[lets-dance-with-gpt]

Title: LLaMA - Leaked OpenAI Model for Academic Research

Contrary to Facebook's announcement that LLaMA would only be accessible for academic research purposes, LLaMA was leaked through torrents on the first day of its release.

LLaMA, like last year's Stable Diffusion Moment with image generation models, has generated excitement in the open-source community due to the lack of public access to ChatGPT and DALL-E, which are both works of OpenAI.

LLaMA is based on the Transformer model and has not undergone significant changes since its release in 2017, yet it continues to show impressive performance.

The key differences of LLaMA compared to ChatGPT and DALL-E are the use of Pre-normalization, SwiGLU activation function, and Rotary Embeddings. Pre-normalization normalizes the input with RMSNorm instead of the output of sub-layers in Transformer, SwiGLU activation function replaces ReLU, and Rotary Embeddings apply Positional Encoding. LLaMA also uses Rotary Positional Embeddings.

LLaMA applies the AdamW Optimizer and uses the xformers library, which implements causal multi-head attention, to speed up training. Additionally, manual implementation of backward functions for Transformer layers was done to reduce the re-computation of activations during checkpointing backward pass.

LLaMA is trained with 1.4T Tokens, which is about three times larger than GPT-3's 500B Tokens. BloombergGPT, trained with 700B Tokens, extracted 363B Tokens from Bloomberg's financial data. As a result, the 5B LLaMA model outperforms the 175B model in Finance Tasks.

The training period for LLaMA took 21 days for the entire 1.4T token training on 80GB A100 2,048 GPUs, processing 380 Tokens/1 GPU/1s during the 65B model training.

Models Developed from SFT to RLHF in SFT LLaMA

Alpaca: A Strong, Replicable Instruction-Following Model

Released by Stanford CRFM (Foundation Model Research Institute directed by Percy Liang).

Seed with 175 human-generated instruction-output pairs on the LLaMA 7B model, followed by generating 52K examples using text-davinci-003 for SFT.

Made efforts to ensure diversity in the data.

Trained on 4 A100 GPUs with PyTorch FSDP, using a Hugging Face version with LLaMA PRs that were not yet merged.

Applied OpenAI's Moderations API for content filtering in the demo and added watermarks to all model outputs.

Vicuna: An Open Chatbot Impressing GPT-4

Conducted SFT using 70K conversation pairs collected from ShareGPT.

First version, Vicuna-13B, was trained on LLaMA 13B and later LLaMA 7B was also released.

Created a separate Python package called FastChat and added a demo service using Gradio for convenient usage.

Converted LLaMA to Hugging Face format and applied the provided apply_delta from FastChat to obtain Vicuna weights.

Training was done on 8 A100 GPUs with PyTorch FSDP for one day.

When evaluated with GPT-4, outperformed ChatGPT by 14%, but still suffers from significant Hallucination issues.

Koala: A Dialogue Model for Academic Research

Released by UC Berkeley's BAIR.

Utilized 60K data pairs from ShareGPT, with duplicates and non-English conversations removed, resulting in 30K remaining.

Evaluation was conducted by 100 human evaluators.

Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data

Trained using LoRA, and 100K dialogues were generated using ChatGPT.

Repository available at https://github.com/project-baize/baize-chatbot.

GPT4All

Built 800K samples using gpt-3.5-turbo and conducted SFT.

RLHF

RLHF is a model that combines SFT (Stacked Fusion Transformer) with reinforcement learning using a library called TRL (Transformer Reinforcement Learning). It showed good results with SFT alone, but the model that incorporates RLHF, similar to the InstructGPT paper, is as follows: StackLLaMA. The initial training was done using the StackExchange dataset, and Reward scores were automatically constructed using upvotes. The peft library, which performs LoRA to reduce memory usage, was used. The model was loaded with 8 bits (1 byte of memory per parameter), and for LLaMA 7B, it only occupied 7GB of memory. Thanks to this, SFT can be run on Google Colab as well. RLHF used the TRL library, which is a Transformer Reinforcement Learning library currently being developed in-house.

ColossalChat

ColossalChat is a project that applies RLHF to LLM as part of the Colossal-AI project: ColossalAI.

Open Assistant

Open Assistant, as described on their website open-assistant.io, aims to provide amazing conversational AI that improves the world, just like how Stable Diffusion has helped create art and images in new ways. The project is being carried out by LAION AI, known for the LAION-400M dataset in the computer vision field. They have already released a model based on Pythia 12B with SFT.

Others

trlX: trlX is a project created by CarperAI, a spin-off from EleutherAI, which extends TRL (Transformer Reinforcement Learning) and supports Accelerate by Hugging Face up to 20B parameters and NeMo for more than 20B parameters. The PPO version of NeMo is still under development, but frameworks for ChatGPT RLHF using trlX have been almost completed, including publishing a summary task RLHF using trlX on WandB.

llama.cpp: llama.cpp is a project by Georgi Gerganov from Bulgaria, reminiscent of the talented hacker Fabrice Bellard from France (who is also currently all-in with GPT and LLM), with the goal of making LLaMA runnable on a local MacBook rather than an A100 GPU. Its main objectives are C++ optimization, 4-bit quantization, and CPU support. It has been an ongoing personal project for a long time, developing C++ projects that infer GPT on CPU, and it shows a high level of completion, which suggests that it may be possible to run the service on CPU servers alone in production.

LLaMA 4bit ChatBot Guide v2: LLaMA 4bit ChatBot Guide v2 provides a guide on running LLaMA with 4-bit quantization. It offers various resources that can significantly reduce operating costs by running CPU production.

[homo-lorens]

If you go to official tech support with a problem that's on you. Like every agent under capitalism, they are only motivated to help you up to a certain point. This is why forums like Quora, Reddit, StackExchange exist. Smarter companies have their own forum where users can help each other solve problems, and they employ a handful of people to comb the forum, who both act as moderators and second line tech support. This way they can distance themselves from the suggested diagnosis and solution while still helping users.

Reddit’s tech help threads may heckle you for being an amateur but at least they don’t try to gaslight you like Adobe help threads

[hackernewsrobot]

StackExchange offers 'possible reinstatement' to moderator removed over pronouns

https://tech.slashdot.org/story/19/12/28/0448227/stack-exchange-apologizes-offers-possible-reinstatement-to-moderator-removed-over-pronouns-policy Comments

[thousandmaths]

“and do the exercises”

I’ve always found the common recommendation to “Read this textbook and do the exercises.” to be... kind of unrealistic?

Recently I’ve had the opportunity to be on the other side of this recommendation, and this post is a somewhat winding and personal journey from my old perspective to my new one.

------

Bounding the problem

That was intentionally a bit inflammatory, so let me backpedal a bit. There are certainly times when this kind of approach is appropriate. For instance, I’ve encountered a bunch of geometers who have been given such a recommendation by their advisor about Vakil’s notes. This is a bit of an extreme example, but even in this case, this kind of advice seems justified. Probably most of those notes are going to be important for them at some stage in their career, and doing exercises is the sort of thing that a person is just generally less willing to do as they get older. So you might as well get comfortable with these ideas, in a very concrete, no-details-spared way, while you’re still a student.

On the other hand, if Christine told me to do the same, that seems a little less justified. Yes, a lot of commutative algebra interacts quite strongly with algebraic geometry, and so I wouldn’t be terribly surprised to go through substantial portions of the book at some point. But it’s another thing entirely to do the exercises, and the difference really has to do with time investment.

I always assumed that people giving this kind of recommendation just forgot that exercises are fairly time-consuming. Even if they’re easy, they still take a while. And of course if you’re really trying to do all the exercises, some of them will not be easy. (If you’re not trying to do all the exercises, then you have to guess which ones will be most useful ahead of time! Which is a task I barely trust myself with, and I am a fairly mathematically mature student...)

For me, who uses algebraic geometry as a tool rather than being in the field per se, there’s no way it would be worthwhile to go through Vakil “and do the exercises”. That would take an enormous amount of time, time which would, almost surely, be much better spent thinking about things more directly relevant to my thesis problem. Or, you know, going through Enumerative Combinatorics and doing the exercises :P

------

On the other side of the fence...

I haven’t talked about it much on the blog, but I’m working with a high school student online this summer because of conversations we’ve had through Math StackExchange. They’ve just finished the legally & freely available portion of Hermann and Sally’s Number, Shape, and Symmetry, and we’re now in the position of trying to figure out where to go from here. I have a physical copy of the book and am pretty sure I could get away with anything reasonable while staying well within fair use boundaries. But in particular the publicly available portion doesn’t have the exercises for Chapter 1, so I figured I’d type up the ones that we would be most likely to be useful. And you probably know where this is going.

To be clear: this book is chock-full of exercises: in addition to more routine things posed as “Practice Problems” within the text itself, there are also 42 exercises for Chapter 1 alone. And these are not terribly short exercises, it even takes the authors 10 pages just to write down all of them.

A few were pretty easy to cross off for this particular student, since they pick up on things fast enough that they don’t really need to be computing another union and intersection of elementwise-defined sets, &c.

The first one that I picked up on was 1.7, which is the classic $|2^S|=2^{|S|}$ for finite sets. And then 1.10, about the composability of functions. And then 1.14 which asks to justify a certain argument via the ring axioms.

And then we hit 1.17 and suddenly my job gets fairly difficult because now it’s “here are some unusual but respectable binary operations, which ring axioms do they satisfy” for a solid ten questions, and pretty much all of them are worth it— like at some point we’re going to talk about Pythagorean triples and so it would be good if they had seen 1.25 about Gaussian integers, but can you really justify doing that without 1.24 about $\Bbb Z\times\Bbb Z$, and what about 1.27 which translates the operation to the rational points of the unit circle (which would help for that video) and it keeps going...

And then 1.34 to 1.39 are all about $\Bbb Z/12\Bbb Z$, which, since the textbook has already made a big deal about $\Bbb Z/10\Bbb Z$ and $\Bbb Z/2\Bbb Z$, might be the point where the general notion of modular arithmetic might start taking shape in their mind.

Not to mention 1.31 and 1.40 which start hinting at quadratic residues, and 1.32 gives that idea an interesting connection to Fermat’s Little Theorem, and—

You get the idea. There are just so many worthwhile problems crammed in there.

------

“They advise you on how to fail.”

As it happens, I’ve experienced this phenomenon in another aspect of my life: low-level Starcraft coaching. The exact nature of the problem is slightly different there, but the idea is the same. 

Since probably not many readers of this blog play Starcraft, let me say a few words. I am not a tremendously good player, but I could look at any replay given to me by a silver-leaguer, and we could analyze its problems for multiple hours. Of course I don’t do this in sessions because that’s not productive. But it’s worse than that: it’s even unproductive to condense that analysis down into just the advice portions, which we could get through in 20 minutes easily. Firstly that’s because it would be a massive dump of information. And more importantly, most of that advice— while it will surely help— will only help very little.

So the easy part of my job as a Starcraft coach is really to compile a list of problems in their play. The hard part is to take this long list of problems, throw most of it away, and find the two or maybe three bits of advice that they can walk away with, that will improve their skill most quickly while still being moderately entertaining.

And that, in turn, reminds me of this:

If you ask them what is important to succeed as a junior faculty member, people will tell you everything they did that they think helped them succeed. Plus everything they wish they had done. And all the things their friend did too. They deliver you this list without annotation, a list which no single person could ever accomplish.

So I no longer think that the problem is just that “and do the exercises” recommendations come from a place of failing to remember that solving math problems is time-consuming work. Sometimes it probably is. But it’s probably more common that this advice comes from remembering how much insight they personally have gained from doing just that one exercise, and forgetting how many exercises they did which— while they surely helped!— helped only very little.

[ It is also, of course, much easier to say “and do the exercises” than to say “and do Exercises 1.8, 2.12, 4.6,” etc., which is perhaps an even more direct comparison to the Starcraft situation. It’s not even clear to what extent such advice would be welcomed in the mathematical context— or in Starcraft either, but there it is easy to sidestep that issue by simply not mentioning the hundred other things. This workaround doesn’t translate to reading books, where the constraints of the medium are that, like it or not, you have all the available information presented to you at once. ]

[gywair]

There are a lot of websites for learning new skills. Trying to keep them straight can be a hassle. What are the pros and cons of each one? I picked up a Humble Bundle for Zenva Academy and decided to try it out.

If you’re like me, then you have NO IDEA what you are doing and every day is an anxiety-induced fever dream of terror and uncertainty. Most of this fear comes from the realization that the market’s demand for skills has drastically changed in ways we are still struggling to comprehend or predict. When I entered college, I was told any degree would work for getting a stable office job. Graduating, I was told that I was up a dry creek without a boat.

Luckily, there are several different online learning websites to help me gain an advantage in furthering my career. Most of these services work as a learning management system but paired down from say a higher education environment. Features like assessment, rubrics, and anonymous course evaluations can really bulk out an LMS. Since Zenva is one of the commercial options, like Lynda, Khan, and Udemy, it is much more straight forward than say Blackboard or Canvas. Those are more geared toward a personal instructional environment at a college or corporation. In Zenva, you have a course, videos for each lesson, course files, class discussion, and a review page. In that way, it is very similar to other offerings of its type.

My experiences with these online commercial learning websites have been a mixed bag. Khan is really great but its courses are much more content restricted. It’s mostly for elementary to high school students preparing for college. Udemy has a huge collection of courses but the content is not as supervised. Some courses are great, some are terrible, and some are borderline scammy. Lynda seems like the best of both worlds but it is much more expensive. Khan is free, Udemy is by course, and Lynda is a subscription service.

Zenva is priced by course like Udemy. The content seems more moderated than Udemy but not as strict as Khan. Zenva has similar offerings to Lynda but you don’t have to pay monthly for courses. Lynda is nice in that you get access to everything when you are paying, however. Zenva’s big draw for me was a Humble Bundle deal. I got a huge selection of courses for game development and web design at a pretty nice price. They also have regular sales on their inventory.

The first course I dove into was the Unity Ping Pong class. It was designed so that you could get an introduction to Unity 3D, build a full project, and learn the workflow. Personally, I got lost a few times. The instructor would show you how to do things one way, change their mind, and then show you a new way. I had to do the whole thing a couple of times before I got close to completing it.

You can play the finished product on my itch.io page. Mine has an asset swap. It plays well and it did teach me a whole lot about Unity. The course also offers the completed project files and closed caption notes.

Overall, the experience was good. Starting some of the other courses, I found out that there are several instructors. If one course doesn’t work for you try some of the others. The Intro to Unity has an instructor I can follow much easier. They do have a class discussion page to ask questions if you get lost. I figured out my problem before trying but you can also read other peoples’ questions there.

In addition to the website, there is also a phone app. It links to your Zenva account. I haven’t gotten very far but am pretty impressed with what I have tried.

It has a number of courses that you can purchase by logging in each day. I assume there is also a way to buy in-app currency. Each course restricts you with prerequisites courses. That way you have to build a good foundation before getting ahead of your self. This is nice for a mobile experience since you are going to probably spend shorter periods of time working on it. Each course won’t have to reteach you everything.

It also has assignments and quizzes. The app uses gamified assignments to track your mastery. It seems like a nice compliment to the website.

So far I am pretty happy with the content, pricing, and delivery method of Zenva. It has the best features of its top competitors without glaring drawbacks. I don’t think I would have tried it without the Humble Bundle (or rather having a few courses to compare to one another). It has more specialized courses than Khan. It has better quality than Udemy. I like the price per course sales model. The only real flaw at this point is the number of courses. There are a LOT of courses available but compared to Udemy and Lynda they are more focused on coding. Since their offerings have more support than say Udemy (which relies on the individual instructors) I don’t really mind the smaller catalog.

The art for the applications and courses is a little different. It feels like they wanted the courses to have a uniform look. That’s fine but it’s at the expense of meaningful cover art. So I don’t always know what a course is based on the icon. I don’t hate the art direction but this is something Lynda has really mastered. I can open their catalog and know exactly which course is which without the title.

Something I like is that Zenva does certificates. Lynda does as well but I can never tell if that’s something I should actually be pushing to my curriculum vitae. Zenva goes a step further by adding ‘mini-degrees’ in grouped subjects as well as course certificates. It’s probably not as official as say being A+ Certified but it feels a little more definitive with the URL attached to the file. Someone could actually go out and verify that I finished it online. Rather than just putting it on my resume and hoping I can set expectations of exactly how much I know (cause there is a difference between I made a Hello World website from scratch and I made a new Facebook all by myself).

Closing thoughts: Zenva seems like a solid training site. It works better for me than just YouTube video or StackExchange tutorials. It’s right up there with its competitors. If you like learning through these kinds of websites, then I highly recommend checking them out.

I had some thoughts on Zenva Academy after trying out some of their courses. They are a pretty fair experience overall. There are a lot of websites for learning new skills. Trying to keep them straight can be a hassle.

[topicprinter]

Article URL: https://tech.slashdot.org/story/19/12/28/0448227/stack-exchange-apologizes-offers-possible-reinstatement-to-moderator-removed-over-pronouns-policy

Comments URL: https://news.ycombinator.com/item?id=21900778

Points: 23

# Comments: 2

[e-commerce-magento]

RT @mattantley: Congratulations to @sandermangel and @dev_amitbera for becoming the latest Magento StackExchange Moderators! Well deserved 🙂

Congratulations to @sandermangel and @dev_amitbera for becoming the latest Magento StackExchange Moderators! Well deserved 🙂

— Matt Antley (@mattantley) March 6, 2018

from Twitter https://twitter.com/fbeardev

[longroadmediacom-blog]

Long Road Media

10 Hot Websites for Developers

They  used to call super new apps “unicorns.” Now, that’s so last year! Any app developer looking to revolutionize the market must stay alert and aware in an industry where everything runs faster than unicorns.

Writing for The New York Times, Brian X. Chen remembered, “When the market started to take off, anyone with a bit of tech-savviness could download some tools, read some books or take some lessons and then whip up an app.”

You have choices.

The web serves people who use it well. The list of top websites for developers is long. Here is a list of 10 of the most useful and popular sites among developers, students and pros.

Hacker News (https://news.ycombinator.com/) is a social community of developers, programmers, and startups. Experts drop in and out offering insights and opinions in terms only understood by their peers. Participants vote submissions up; they cannot vote them down until the user builds up points. Geared to reward positive contributions, Hacker News still struggles to find its own voice.

Slashdot (https://slashdot.org/) carries “news for nerds.” Members submit industry news items, and a team of editors publishes a one paragraph summary with an invitation to continue the thread. A points-based karma system moderates offensive and unqualified participants out of community participation.

Reddit Programming (https://www.reddit.com/r/programming/) remains the community standard. The best of the best stay tuned to Reddit for news and trending. Visitors don’t have to be members to read the front page where stories get placement as a result of positive votes. Note: Programming is only one subject category in the broader Reddit.

DZone (https://dzone.com/) is more visually interesting and engaging than the previous communities. It’s an actual website, not just a number of threads and conversations. They market themselves as the leading publisher of software developer content.

StackExchange (http://stackexchange.com/) follows a Question and Answer format instead of forum threads. Users vote on which questions and which answers move to the top in a gaming points system. Questions are fresh and unusual, and on a recent page, there were 239,000 questions evoking 400,000 plus answers. Askers compete to make list of top questionnaires on the landing page.

A List Apart (https://alistapart.com/) is a clever name for a website dedicated to “people who make websites.” A library of informative articles, it invites writers to contribute (and provides extensive directions on submissions). It’s more about content than programming and development.

SitePoint (https://www.sitepoint.com/) posts articles, ebooks videos, and tutorials for designers, developers, entrepreneurs, and programmers. The variety of material has plenty to offer to the novice too.

MIT OpenCourseWare (https://ocw.mit.edu/) brings MIT’s prestige to students everywhere. MIT made the historical decision to makes it work available at no cost to serve its students better, to share their wealth with the world, and to drive its professors to keep their work fresh. It offers active learning, problem solving, digital tools, design processes, and more.

Coursera (https://www.coursera.org/) offers selected courses from many top universities on a variety of subject majors. Currently, courses are priced by individual course, specialization, or degree. That makes it a for-profit education with your freedom to select from the best educators at the best universities.

Code Academy (https://www.codecademy.com/) markets itself as a solution to old and broken education models, “the first truly net native education.” It offers extensive code learning opportunities for beginners. They also offers live sessions and subscription membership at a cost.

Where do web developers go when they need news? Finding the hottest sites in the web developers’ world presents the same problem facing everything hi-tech: There’s just so much out there.