#democratization of AI

[jcmarchi]

From Recurrent Networks to GPT-4: Measuring Algorithmic Progress in Language Models - Technology Org

New Post has been published on https://thedigitalinsider.com/from-recurrent-networks-to-gpt-4-measuring-algorithmic-progress-in-language-models-technology-org/

From Recurrent Networks to GPT-4: Measuring Algorithmic Progress in Language Models - Technology Org

In 2012, the best language models were small recurrent networks that struggled to form coherent sentences. Fast forward to today, and large language models like GPT-4 outperform most students on the SAT. How has this rapid progress been possible? 

Image credit: MIT CSAIL

In a new paper, researchers from Epoch, MIT FutureTech, and Northeastern University set out to shed light on this question. Their research breaks down the drivers of progress in language models into two factors: scaling up the amount of compute used to train language models, and algorithmic innovations. In doing so, they perform the most extensive analysis of algorithmic progress in language models to date.

Their findings show that due to algorithmic improvements, the compute required to train a language model to a certain level of performance has been halving roughly every 8 months. “This result is crucial for understanding both historical and future progress in language models,” says Anson Ho, one of the two lead authors of the paper. “While scaling compute has been crucial, it’s only part of the puzzle. To get the full picture you need to consider algorithmic progress as well.”

The paper’s methodology is inspired by “neural scaling laws”: mathematical relationships that predict language model performance given certain quantities of compute, training data, or language model parameters. By compiling a dataset of over 200 language models since 2012, the authors fit a modified neural scaling law that accounts for algorithmic improvements over time. 

Based on this fitted model, the authors do a performance attribution analysis, finding that scaling compute has been more important than algorithmic innovations for improved performance in language modeling. In fact, they find that the relative importance of algorithmic improvements has decreased over time. “This doesn’t necessarily imply that algorithmic innovations have been slowing down,” says Tamay Besiroglu, who also co-led the paper.

“Our preferred explanation is that algorithmic progress has remained at a roughly constant rate, but compute has been scaled up substantially, making the former seem relatively less important.” The authors’ calculations support this framing, where they find an acceleration in compute growth, but no evidence of a speedup or slowdown in algorithmic improvements.

By modifying the model slightly, they also quantified the significance of a key innovation in the history of machine learning: the Transformer, which has become the dominant language model architecture since its introduction in 2017. The authors find that the efficiency gains offered by the Transformer correspond to almost two years of algorithmic progress in the field, underscoring the significance of its invention.

While extensive, the study has several limitations. “One recurring issue we had was the lack of quality data, which can make the model hard to fit,” says Ho. “Our approach also doesn’t measure algorithmic progress on downstream tasks like coding and math problems, which language models can be tuned to perform.”

Despite these shortcomings, their work is a major step forward in understanding the drivers of progress in AI. Their results help shed light about how future developments in AI might play out, with important implications for AI policy. “This work, led by Anson and Tamay, has important implications for the democratization of AI,” said Neil Thompson, a coauthor and Director of MIT FutureTech. “These efficiency improvements mean that each year levels of AI performance that were out of reach become accessible to more users.”

“LLMs have been improving at a breakneck pace in recent years. This paper presents the most thorough analysis to date of the relative contributions of hardware and algorithmic innovations to the progress in LLM performance,” says Open Philanthropy Research Fellow Lukas Finnveden, who was not involved in the paper.

“This is a question that I care about a great deal, since it directly informs what pace of further progress we should expect in the future, which will help society prepare for these advancements. The authors fit a number of statistical models to a large dataset of historical LLM evaluations and use extensive cross-validation to select a model with strong predictive performance. They also provide a good sense of how the results would vary under different reasonable assumptions, by doing many robustness checks. Overall, the results suggest that increases in compute have been and will keep being responsible for the majority of LLM progress as long as compute budgets keep rising by ≥4x per year. However, algorithmic progress is significant and could make up the majority of progress if the pace of increasing investments slows down.”

Written by Rachel Gordon

Source: Massachusetts Institute of Technology

You can offer your link to a page which is relevant to the topic of this post.

[kakief]

Is Artificial Intelligence (AI) Ruining the Planet—or Saving It?

AI’s Double-Edged Impact: Innovation or Environmental Cost? Have you heard someone say, “AI is destroying the environment” or “Only tech giants can afford to use it”? You’re not alone. These sound bites are making the rounds—and while they come from real concerns, they don’t tell the whole story. I’ve been doing some digging. And what I found was surprising, even to me: AI is actually getting a…

[mysharona1987]

[progressive-memes]

[unitedfrontvarietyhour]

I'M BACK, B!+CHES!!!!

"Communism is the doctrine of the conditions of the liberation of the proletariat." - Engels, Principles of Communism (1847)

(Pt.1)

[political-us]

[beauty-funny-trippy]

"Grok is a generative AI chatbot and large language model launched in 2023 by Elon Musk's xAI. It's designed to be objective and truthful." (source)

(Source: Grok AI)

[jcmarchi]

Beyond Security: How AI-Based Video Analytics Are Enhancing Modern Business Operations

New Post has been published on https://thedigitalinsider.com/beyond-security-how-ai-based-video-analytics-are-enhancing-modern-business-operations/

Beyond Security: How AI-Based Video Analytics Are Enhancing Modern Business Operations

AI-based solutions are becoming increasingly common, but those in the security industry have been leveraging AI for years—they’ve just been using the word “analytics.”  As businesses seek new ways to use AI to create a competitive advantage, many are beginning to recognize that video devices represent an increasingly valuable data source—one that can generate actionable business intelligence insights. As processing power improves and chipsets become more advanced, modern IP cameras and other security devices can support AI-powered analytics capabilities that can do far more than identify trespassers and shoplifters.

Many businesses are already leveraging AI-based analytics to improve efficiency and productivity, reduce liability, and better understand their customers. Video analytics can help enterprises identify ways to improve employee productivity and staffing efficiency, streamline the layout of stores, factories, and warehouses, identify in-demand products and services, detect malfunctioning or poorly maintained equipment before it breaks, and more. These new analytics capabilities are being designed with business intelligence and operational efficiency in mind—and they are increasingly accessible to organizations of all sizes.

The Growing Accessibility of AI in Video Surveillance

Analytics has always had clear applications in the security industry, and the evolution from basic intelligence and video motion detection to more advanced object analytics and deep learning has made it possible for modern analytics to identify suspicious or criminal behavior or to detect suspicious sounds like breaking glass, gunshots, or cries for help. Today’s analytics can detect these events in real time, alerting security teams immediately and dramatically reducing response times. The emergence of AI has allowed security teams to be significantly more proactive, allowing them to make quick decisions based on accurate, real-time information. Not long ago, only the most advanced surveillance devices were powerful enough to run the AI-based analytics needed to enable those capabilities—but today, the landscape has changed.

The advent of deep learning processing units (DLPUs) has significantly enhanced the processing power of surveillance devices, allowing them to run advanced analytics at the network edge. Just a few years ago, the bandwidth and storage required to record, upload, and analyze thousands of hours of video could be prohibitively expensive. Today, that’s no longer the case: modern devices no longer need to send full video recordings to the cloud—only the metadata necessary for classification and analysis. As a result, the bandwidth, storage, and hardware footprint required to take advantage of AI-based analytics capabilities have all dramatically decreased—significantly reducing operational costs and making the technology accessible to businesses of all sizes, whether they employ a network of three cameras or three thousand.

As a result, the range of potential customers has expanded significantly—and those customers aren’t just looking for security applications, but business ones as well. Since DLPUs are effectively standard on modern surveillance devices, customers are increasingly looking to leverage those capabilities to gain a competitive advantage in addition to protecting their locations. The democratization of AI in the security industry has led to a significant expansion of use cases as developers look to satisfy businesses turning to video analytics to address a wider range of security and non-security challenges.

How Organizations Are Using AI to Enhance Their Operations

It’s important to emphasize that part of what makes the emergence of more business-focused use cases for AI-based video analytics notable is the fact that most businesses are already familiar with the basic technology. For example, retailers already using video analytics to protect their stores from shoplifters will be delighted to learn that they can use similar capabilities to monitor customers entering and leaving the store, identify high- and low-traffic periods, and use that data to adjust their staffing needs accordingly. They can use video analytics to alert employees when a lengthy queue is forming, when an empty shelf needs to be restocked, or if the layout of the store is causing unnecessary congestion. By embracing business-focused analytics alongside security-focused ones, retailers can improve staffing efficiency, create more effective store layouts, and enhance the customer experience.

Of course, retailers are just the tip of the iceberg. Businesses in nearly every industry can benefit from modern video analytics use cases. Manufacturers, for example, can monitor factory floors to identify inefficiencies and choke points. They can use thermal cameras to detect overheating machinery, allowing maintenance personnel to address problems before they can cause significant damage. In many cases, they can even monitor assembly lines for defective or poorly made products, providing an additional layer of quality assurance protection. Some devices may even be able to monitor for chemical leaks, overheating equipment, smoke, and other signs of danger, saving organizations from potentially dangerous (and costly) incidents. This has clear applications in industries ranging from manufacturing and healthcare to housing and critical infrastructure.

The ability to generate insights and improve operations extends beyond traditional businesses and into areas like healthcare. Hospitals and healthcare providers are now leveraging analytics to engage in virtual patient monitoring, allowing them to have eyes on their patients on a 24-hour basis. Using a combination of video and audio analytics, they can automatically detect signs of distress such as coughing, labored breathing, and cries of pain. They can also generate an alert if a high-risk patient attempts to leave their bed or exit the room, allowing caregivers or security teams to respond immediately. Not only does this improve patient outcomes, but it can also significantly reduce liability on slip/trip/fall cases. Similar technology can also be used to improve compliance outcomes, ensuring emergency exits remain clear and avoiding other potentially finable offenses in healthcare and other industries. The opportunities to reduce costs and improve outcomes are expanding every day.

Maximizing AI in the Present and Future

The shift toward leveraging surveillance devices for business intelligence and operations purposes has happened quickly, driven by the fact that most organizations are already familiar with the equipment they need to take advantage. And with businesses of all sizes—and in nearly every industry—increasingly turning to video analytics to enhance both their security capabilities and their business operations, the development of new, AI-based analytics is unlikely to slow anytime soon.

Best of all, the market is still growing. Even today, roughly 80% of security budgets are spent on human labor, including monitoring, guarding, and maintenance capabilities. As AI-based video analytics become increasingly widespread, that will change quickly—and businesses will be able to streamline their business intelligence and operations capabilities in a similar manner. As AI development continues and new, business-focused use cases emerge, organizations should ensure they are positioned to get the most out of analytics—both now and into the future.

[contemplatingoutlander]

It is disturbing that Musk's AI chatbot is spreading false information about the 2024 election. "Free speech" should not include disinformation. We cannot survive as a nation if millions of people live in an alternative, false reality based on disinformation and misinformation spread by unscrupulous parties. The above link is from the Internet Archive, so anyone can read the entire article. Below are some excerpts:

Five secretaries of state plan to send an open letter to billionaire Elon Musk on Monday, urging him to “immediately implement changes” to X’s AI chatbot Grok, after it shared with millions of users false information suggesting that Kamala Harris was not eligible to appear on the 2024 presidential ballot. The letter, spearheaded by Minnesota Secretary of State Steve Simon and signed by his counterparts Al Schmidt of Pennsylvania, Steve Hobbs of Washington, Jocelyn Benson of Michigan and Maggie Toulouse Oliver of New Mexico, urges Musk to “immediately implement changes to X’s AI search assistant, Grok, to ensure voters have accurate information in this critical election year.” [...] The secretaries cited a post from Grok that circulated after Biden stepped out of the race: “The ballot deadline has passed for several states for the 2024 election,” the post read, naming nine states: Alabama, Indiana, Michigan, Minnesota, New Mexico, Ohio, Pennsylvania, Texas and Washington. Had the deadlines passed in those states, the vice president would not have been able to replace Biden on the ballot. But the information was false. In all nine states, the ballot deadlines have not passed and upcoming ballot deadlines allow for changes to candidates. [...] Musk launched Grok last year as an anti-“woke” chatbot, professing to be frustrated by what he says is the liberal bias of ChatGPT. In contrast to AI tools built by Open AI, Microsoft and Google, which are trained to carefully navigate controversial topics, Musk said he wanted Grok to be unfiltered and “answer spicy questions that are rejected by most other AI systems.” [...] Secretaries of state are grappling with an onslaught of AI-driven election misinformation, including deepfakes, ahead of the 2024 election. Simon testified on the subject before the Senate Rules and Administration Committee last year. [...] “It’s important that social media companies, especially those with global reach, correct mistakes of their own making — as in the case of the Grok AI chatbot simply getting the rules wrong,” Simon added. “Speaking out now will hopefully reduce the risk that any social media company will decline or delay correction of its own mistakes between now and the November election.” [color emphasis added]

[mysharona1987]

They have turned the Palestinians into actual Guinea pigs for the military industrial complex.

We will see the robots and miserable remote controlled dogs at the next big BLM protest on American soil soon enough.

[progressive-memes]

[undergroundusa]

"I think that the $50 million spent on condoms for Palestinians in Gaza or free benefits for people who have broken the law to be here in the first place, could be better spent on Americans in need right here in the US. But hey, I’m not an appointed spendthrift, so what do I know…"

READ & LISTEN NOW: https://www.undergroundusa.com/p/taxpayer-dollars-cant-be-considered

[aiandhunks]

The Electoral Race

[relaxedstyles]

[crow-made-of-onyx]

today

todays the day

and i'm being told its only 4 years

but look at history

this may never end.